| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/queens: GATE CHECK for TripleO https://review.openstack.org/567224 | 00:00 |
|---|---|---|
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/560445 | 00:00 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates master: GATE CHECK for TripleO https://review.openstack.org/604298 | 00:00 |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-heat-templates stable/rocky: GATE CHECK for TripleO https://review.openstack.org/604293 | 00:00 |
| *** rascasoft has quit IRC | 00:04 | |
| *** wolverineav has quit IRC | 00:15 | |
| *** wolverineav has joined #tripleo | 00:21 | |
| *** cfontain_ has quit IRC | 00:21 | |
| *** ansmith has joined #tripleo | 00:22 | |
| *** tosky has quit IRC | 00:23 | |
| *** ansmith has quit IRC | 00:27 | |
| *** dciabrin__ has quit IRC | 00:29 | |
| *** dciabrin__ has joined #tripleo | 00:30 | |
| *** jobewan has quit IRC | 00:45 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: container image build: introduce --use-buildah https://review.openstack.org/634322 | 00:46 |
| *** thrash is now known as thrash|g0ne | 00:55 | |
| *** dprince has quit IRC | 01:16 | |
| *** wolverineav has quit IRC | 01:18 | |
| *** jaosorior has quit IRC | 01:20 | |
| *** wolverineav has joined #tripleo | 01:24 | |
| *** rascasoft has joined #tripleo | 01:26 | |
| *** jaosorior has joined #tripleo | 01:32 | |
| *** dmacpher_ has joined #tripleo | 01:33 | |
| *** dmacpher has quit IRC | 01:34 | |
| openstackgerrit | Steve Baker proposed openstack-infra/tripleo-ci master: Ensure cirros-0.3.6 is referenced consistently https://review.openstack.org/639008 | 01:40 |
| *** rascasoft has quit IRC | 01:40 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Add LVM option for ceph-loop-device role https://review.openstack.org/637196 | 01:47 |
| *** artom has joined #tripleo | 02:04 | |
| *** _fragatina has quit IRC | 02:05 | |
| *** wolverineav has quit IRC | 02:05 | |
| *** wolverineav has joined #tripleo | 02:08 | |
| *** bfournie has joined #tripleo | 02:09 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] rhel8 beta support https://review.openstack.org/623137 | 02:11 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] shortcut pip-and-virtualenv for later redhat distros https://review.openstack.org/639220 | 02:11 |
| *** mschuppert has quit IRC | 02:15 | |
| *** cfontain has joined #tripleo | 02:18 | |
| *** wolverineav has quit IRC | 02:26 | |
| *** wolverineav has joined #tripleo | 02:27 | |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: Rename errors file if it's big https://review.openstack.org/639178 | 02:27 |
| *** wolverineav has quit IRC | 02:32 | |
| *** cfontain_ has joined #tripleo | 02:44 | |
| *** cfontain has quit IRC | 02:47 | |
| openstackgerrit | Rafael Folco proposed openstack-infra/tripleo-ci master: Add timestamp to containers' build error log https://review.openstack.org/639089 | 02:49 |
| *** psachin has joined #tripleo | 02:56 | |
| *** cfontain_ has quit IRC | 02:57 | |
| *** wolverineav has joined #tripleo | 03:02 | |
| *** wolverineav has quit IRC | 03:07 | |
| *** ykarel has joined #tripleo | 03:19 | |
| *** cfontain has joined #tripleo | 03:24 | |
| *** abishop has quit IRC | 03:27 | |
| *** rlandy has quit IRC | 03:28 | |
| *** cfontain has quit IRC | 03:29 | |
| *** ade_lee has quit IRC | 03:33 | |
| *** skramaja has joined #tripleo | 03:34 | |
| *** ramishra has joined #tripleo | 03:36 | |
| *** raukadah is now known as chandankumar | 03:41 | |
| *** sdake has joined #tripleo | 03:49 | |
| *** sdake has quit IRC | 03:49 | |
| *** udesale has joined #tripleo | 03:54 | |
| *** cfontain has joined #tripleo | 03:56 | |
| *** ratailor has joined #tripleo | 03:56 | |
| *** shyamb has joined #tripleo | 03:59 | |
| *** shyamb has quit IRC | 04:06 | |
| *** sdake has joined #tripleo | 04:06 | |
| *** sdake has quit IRC | 04:10 | |
| *** sdake has joined #tripleo | 04:11 | |
| *** sdake has quit IRC | 04:13 | |
| *** cfontain has quit IRC | 04:13 | |
| *** sdake has joined #tripleo | 04:14 | |
| *** sdake has quit IRC | 04:23 | |
| *** wolverineav has joined #tripleo | 04:33 | |
| *** rascasoft has joined #tripleo | 04:41 | |
| openstackgerrit | yatin proposed openstack/tripleo-quickstart-extras master: Move 'extra_tht_configs' tasks to overcloud-deploy role https://review.openstack.org/621473 | 04:46 |
| *** rascasoft has quit IRC | 04:53 | |
| *** shyamb has joined #tripleo | 05:05 | |
| *** wolverineav has quit IRC | 05:09 | |
| *** wolverineav has joined #tripleo | 05:11 | |
| *** _fragatina has joined #tripleo | 05:12 | |
| *** shyam89 has joined #tripleo | 05:14 | |
| *** shyam89 has quit IRC | 05:16 | |
| *** shyam89 has joined #tripleo | 05:17 | |
| *** shyamb has quit IRC | 05:17 | |
| openstackgerrit | Merged openstack/ansible-role-tripleo-modify-image master: Speed up execution time of yum_update.sh https://review.openstack.org/635398 | 05:17 |
| *** janki has joined #tripleo | 05:26 | |
| *** jaganathan has joined #tripleo | 05:27 | |
| *** ramishra has quit IRC | 05:32 | |
| *** saneax has joined #tripleo | 05:36 | |
| *** ramishra has joined #tripleo | 05:38 | |
| openstackgerrit | Merged openstack/tripleo-common master: Allow tripleo-admin user to have a keypair https://review.openstack.org/638316 | 05:44 |
| openstackgerrit | Merged openstack/tripleo-quickstart-extras master: update the reproducer doc, with image import https://review.openstack.org/639208 | 05:44 |
| *** _fragatina has quit IRC | 05:47 | |
| *** shyam89 has quit IRC | 05:48 | |
| *** wolverineav has quit IRC | 06:01 | |
| *** wolverineav has joined #tripleo | 06:02 | |
| *** wolverineav has quit IRC | 06:07 | |
| *** wolverineav has joined #tripleo | 06:14 | |
| *** shyamb has joined #tripleo | 06:14 | |
| *** shyam89 has joined #tripleo | 06:17 | |
| shyam89 | Hi | 06:19 |
| *** shyamb has quit IRC | 06:19 | |
| *** shyam89 has quit IRC | 06:19 | |
| *** shyamb has joined #tripleo | 06:20 | |
| shyamb | Hi | 06:20 |
| *** shyamb has quit IRC | 06:27 | |
| *** shyamb has joined #tripleo | 06:27 | |
| *** jchhatbar has joined #tripleo | 06:43 | |
| *** janki has quit IRC | 06:45 | |
| *** cfontain has joined #tripleo | 07:01 | |
| *** ksambor has joined #tripleo | 07:01 | |
| *** florianf has joined #tripleo | 07:03 | |
| *** wolverineav has quit IRC | 07:06 | |
| *** wolverineav has joined #tripleo | 07:06 | |
| *** shyam89 has joined #tripleo | 07:07 | |
| *** yprokule has joined #tripleo | 07:07 | |
| *** rascasoft has joined #tripleo | 07:09 | |
| *** cfontain has quit IRC | 07:09 | |
| *** shyamb has quit IRC | 07:10 | |
| *** skramaja has quit IRC | 07:11 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/python-tripleoclient master: Ensure created symlink is in $PATH https://review.openstack.org/639077 | 07:13 |
| *** wolverineav has quit IRC | 07:13 | |
| *** shyam89 has quit IRC | 07:13 | |
| jaosorior | Tengu: did you check why https://review.openstack.org/#/c/616945/5 failed? | 07:14 |
| Tengu | hey :)0 | 07:16 |
| Tengu | not yet, have some other patches in the pipe | 07:16 |
| ykarel | fultonj, seeing ceph_disk.main.Error: Error: Device /dev/loop3 is in use by a device-mapper mapping (dm-crypt?): dm-0,dm-1,dm-2 in all releases ceph jobs | 07:16 |
| ykarel | fultonj, post https://review.openstack.org/#/c/637196/ merged ^^ | 07:17 |
| ykarel | fultonj, example logs:- https://logs.rdoproject.org/92/18992/2/check/rdoinfo-tripleo-master-centos-7-scenario001-standalone/9ccec7e/logs/undercloud/var/log/extra/docker/containers/ceph-osd-prepare-standalone-loop3/stdout.log.txt.gz | 07:17 |
| *** jfrancoa has joined #tripleo | 07:19 | |
| *** kopecmartin|off is now known as kopecmartin | 07:23 | |
| *** shyam89 has joined #tripleo | 07:24 | |
| *** mschuppert has joined #tripleo | 07:25 | |
| *** jtomasek has joined #tripleo | 07:26 | |
| *** skramaja has joined #tripleo | 07:26 | |
| *** ramishra has quit IRC | 07:31 | |
| *** ramishra has joined #tripleo | 07:32 | |
| *** yprokule_ has joined #tripleo | 07:35 | |
| *** yprokule has quit IRC | 07:35 | |
| *** yprokule_ is now known as yprokule | 07:35 | |
| *** _fragatina has joined #tripleo | 07:41 | |
| *** cylopez has joined #tripleo | 07:43 | |
| ykarel | fultonj, so i rechecked https://review.openstack.org/#/c/547682/, looks like master will be clear post this, but need to fix older releases | 07:46 |
| *** wolverineav has joined #tripleo | 07:47 | |
| *** shyam89 has quit IRC | 07:51 | |
| *** shyam89 has joined #tripleo | 07:51 | |
| *** wolverineav has quit IRC | 07:52 | |
| *** chem` has joined #tripleo | 07:54 | |
| *** ykarel is now known as ykarel|lunch | 07:56 | |
| *** rcernin has quit IRC | 07:56 | |
| *** chem has quit IRC | 07:57 | |
| openstackgerrit | RedHat RDO CI proposed openstack/tripleo-quickstart-extras master: GATE CHECK for quickstart-extras https://review.openstack.org/560445 | 08:00 |
| openstackgerrit | Cédric Jeanneret proposed openstack/python-tripleoclient master: Ensure created symlink is in $PATH https://review.openstack.org/639077 | 08:01 |
| *** shyam89 has quit IRC | 08:01 | |
| *** shyam89 has joined #tripleo | 08:01 | |
| *** agurenko has joined #tripleo | 08:01 | |
| *** ccamacho has joined #tripleo | 08:02 | |
| *** holser_ has joined #tripleo | 08:03 | |
| *** quiquell is now known as quiquell|brb | 08:03 | |
| openstackgerrit | Merged openstack/tripleo-repos master: Add fedora support to tripleo-repos https://review.openstack.org/637212 | 08:05 |
| *** avivgt has joined #tripleo | 08:07 | |
| *** shyam89 has quit IRC | 08:07 | |
| jbadiapa | could any core review https://review.openstack.org/#/c/638143/ to fix the gate, please? | 08:09 |
| openstackgerrit | Cédric Jeanneret proposed openstack/python-tripleoclient master: Ensure created symlink is in $PATH https://review.openstack.org/639077 | 08:15 |
| *** tkajinam has quit IRC | 08:16 | |
| *** shardy has joined #tripleo | 08:16 | |
| openstackgerrit | Merged openstack/ansible-role-openstack-operations master: Fix the gate https://review.openstack.org/638143 | 08:20 |
| openstackgerrit | waleed mousa proposed openstack/os-net-config master: Configure switchdev mode in SR-IOV in os-net-config https://review.openstack.org/633885 | 08:22 |
| *** quiquell|brb is now known as quiquell | 08:23 | |
| *** _fragatina has quit IRC | 08:23 | |
| *** aufi has joined #tripleo | 08:27 | |
| jbadiapa | jaosorior: thx | 08:28 |
| *** wolverineav has joined #tripleo | 08:29 | |
| jaosorior | np | 08:30 |
| *** hberaud|gone is now known as hberaud | 08:31 | |
| *** xek has joined #tripleo | 08:31 | |
| *** holser_ has quit IRC | 08:32 | |
| *** holser_ has joined #tripleo | 08:33 | |
| *** wolverineav has quit IRC | 08:33 | |
| *** pcaruana has joined #tripleo | 08:35 | |
| *** aufi_ has joined #tripleo | 08:35 | |
| *** aufi has quit IRC | 08:38 | |
| *** tosky has joined #tripleo | 08:39 | |
| *** jpena|off is now known as jpena | 08:42 | |
| *** wolverineav has joined #tripleo | 08:44 | |
| *** dtantsur|afk is now known as dtantsur | 08:46 | |
| *** wolverineav has quit IRC | 08:49 | |
| *** shyam89 has joined #tripleo | 08:52 | |
| *** jpich has joined #tripleo | 08:53 | |
| *** _fragatina has joined #tripleo | 08:54 | |
| *** shardy has quit IRC | 08:54 | |
| *** dciabrin__ is now known as dciabrin | 08:58 | |
| *** ccamacho has quit IRC | 08:58 | |
| openstackgerrit | Merged openstack/tripleo-docs master: Add service template sections walkthough (part 3) https://review.openstack.org/638388 | 08:58 |
| *** bogdando has joined #tripleo | 09:02 | |
| Hobbestigrou | hi | 09:07 |
| dtantsur | morning folks! wasn't the problem with removing IPA fixed in DIB long ago? https://logs.rdoproject.org/91/637191/10/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/a92e90b/logs/undercloud/home/zuul/overcloud_image_build.log.txt.gz#_2019-02-25_17_51_35 | 09:09 |
| *** ykarel|lunch is now known as ykarel | 09:10 | |
| *** maufart__ has joined #tripleo | 09:11 | |
| *** aufi_ has quit IRC | 09:13 | |
| *** jchhatbar has quit IRC | 09:16 | |
| openstackgerrit | Martin Mágr proposed openstack/ansible-role-openstack-operations master: Pacemaker backup task https://review.openstack.org/635506 | 09:16 |
| *** jchhatbar has joined #tripleo | 09:16 | |
| *** jchhatbar has quit IRC | 09:22 | |
| *** jchhatbar has joined #tripleo | 09:23 | |
| holser_ | Tengu Hobbestigrou https://review.openstack.org/#/c/638399/ | 09:24 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: DNM: Add py3/f28 support to tripleo-container-build-base https://review.openstack.org/636160 | 09:24 |
| Tengu | holser_: hey :). yep, we just discussed that, it's now sorted ;) | 09:25 |
| holser_ | Ahh, thanks :D | 09:25 |
| Tengu | holser_: so now https://review.openstack.org/639077 does all the things | 09:26 |
| *** ksambor has quit IRC | 09:30 | |
| *** gpocentek has quit IRC | 09:30 | |
| *** Hobbestigrou has quit IRC | 09:30 | |
| *** tbarron has quit IRC | 09:30 | |
| openstackgerrit | Lukas Bezdicka proposed openstack/tripleo-heat-templates master: [FFU] Ensure compatibility with ansible 2.6. https://review.openstack.org/639169 | 09:30 |
| *** Hobbestigrou has joined #tripleo | 09:30 | |
| shyam89 | Hi | 09:31 |
| shyam89 | Any ideas to debug this issue: http://paste.openstack.org/show/746147/ | 09:31 |
| shyam89 | debugging overcloud deployment is toughest part | 09:32 |
| shyam89 | I went into nested stacks and resources | 09:32 |
| shyam89 | still it just one line of error: stack create cancelled | 09:32 |
| jaosorior | shyam89: stack CREATE cancelled usually means that you pressed Ctrl + C while the deployment was running. | 09:33 |
| openstackgerrit | Sergii Golovatiuk proposed openstack/tripleo-heat-templates master: [FFU] Ensure compatibility with ansible 2.6. https://review.openstack.org/639169 | 09:34 |
| shyam89 | jaosorior: I have not pressed it | 09:34 |
| jaosorior | shyam89: is there anything in the heat-engine logs? | 09:34 |
| shyam89 | It hangs and timeout | 09:34 |
| *** ekultails has quit IRC | 09:35 | |
| shyam89 | jaosorior: in osp14 undercloud is containerized | 09:35 |
| shyam89 | where can I find heat logs | 09:35 |
| Tengu | shyam89: /var/log/containers/ | 09:35 |
| *** ekultails has joined #tripleo | 09:35 | |
| shyam89 | ok | 09:35 |
| shyam89 | thanks | 09:35 |
| *** _fragatina has quit IRC | 09:39 | |
| *** ccamacho has joined #tripleo | 09:39 | |
| shyam89 | jaosorior: logs are rotated | 09:40 |
| shyam89 | and cleaned also | 09:40 |
| *** wolverineav has joined #tripleo | 09:41 | |
| shyam89 | my stack failed yesterday evening | 09:41 |
| shyam89 | now I don't see logs of that time | 09:41 |
| *** ksambor has joined #tripleo | 09:41 | |
| paramite | Hey guys, can we please merge this backport: https://review.openstack.org/#/c/633510/ ? | 09:41 |
| shyam89 | jaosorior: I see vms are in BUILD state | 09:41 |
| shyam89 | and their console show no device to boot | 09:42 |
| jaosorior | shyam89: it could have been that it timedout waiting for those nodes to build | 09:42 |
| shyam89 | possible | 09:42 |
| jaosorior | shyam89: so your next bet is to check the ironic logs | 09:42 |
| shyam89 | ok | 09:42 |
| openstackgerrit | Bogdan Dobrelya proposed openstack-infra/tripleo-ci master: Delay deployment jobs until the pep8/unit passes.. https://review.openstack.org/568543 | 09:43 |
| shyam89 | jaosorior: ironic conductor? | 09:43 |
| shyam89 | or api | 09:43 |
| openstackgerrit | Cédric Jeanneret proposed openstack/tripleo-common master: Make PythonImageUploader understand insecure_registries https://review.openstack.org/639037 | 09:44 |
| jaosorior | shyam89: I think it would be conductor | 09:45 |
| *** derekh has joined #tripleo | 09:46 | |
| shyam89 | jaosorior: looked into logs | 09:48 |
| shyam89 | I see ipmitool commands there | 09:48 |
| shyam89 | returning 0 | 09:48 |
| shyam89 | it means they are successful | 09:49 |
| shyam89 | no 'error' or 'failed' found in the log file | 09:49 |
| jaosorior | you could try to execute them yourself to verify. but it sure seems like the nodes are not booting | 09:49 |
| shyam89 | ok | 09:50 |
| shyam89 | jaosorior: command is working well | 09:51 |
| shyam89 | http://paste.openstack.org/show/746151/ | 09:51 |
| shyam89 | it is able to get power status of vm | 09:51 |
| shyam89 | it means power managment is working | 09:51 |
| shyam89 | boot process is not | 09:52 |
| shyam89 | I have single disk for each overcloud node | 09:53 |
| shyam89 | so not set root disk | 09:53 |
| shyam89 | I don't think it is required | 09:53 |
| openstackgerrit | Bogdan Dobrelya proposed openstack-infra/tripleo-ci master: Delay deployment jobs until the pep8/unit passes.. https://review.openstack.org/568543 | 09:56 |
| shyam89 | jaosorior: Should I re-run the deployment and monitor heat engine logs? | 09:56 |
| shyam89 | any other logs need to monitor | 09:56 |
| shyam89 | ? | 09:56 |
| jaosorior | shyam89: heat-engine, nova-compute and ironic are the most relevant in this case | 09:58 |
| shyam89 | ok | 09:59 |
| shyam89 | great | 09:59 |
| shyam89 | thanks | 09:59 |
| shyam89 | jaosorior: One more things | 10:02 |
| shyam89 | I have create new flavors for deployment with new names | 10:03 |
| shyam89 | not delete old | 10:03 |
| shyam89 | and then added property to those flavors | 10:03 |
| shyam89 | to link profiles | 10:03 |
| shyam89 | control, compute | 10:03 |
| *** sai_p_ has quit IRC | 10:04 | |
| shyam89 | and also mentioned --control-flavor and --compute-flavor | 10:04 |
| shyam89 | in deployment command | 10:04 |
| shyam89 | jaosorior: is it enough to make custom flavors work? | 10:04 |
| paramite | guys, can we please merge this backport: https://review.openstack.org/#/c/633510/ ? | 10:06 |
| openstackgerrit | Sergii Golovatiuk proposed openstack/tripleo-heat-templates master: [FFU] Ensure compatibility with ansible 2.6. https://review.openstack.org/639169 | 10:06 |
| jaosorior | shyam89: I'm not sure... I think so | 10:08 |
| openstackgerrit | Cédric Jeanneret proposed openstack/python-tripleoclient master: Ensure created symlink is in $PATH https://review.openstack.org/639077 | 10:10 |
| *** shyam89 has quit IRC | 10:10 | |
| openstackgerrit | Sergii Golovatiuk proposed openstack/tripleo-heat-templates master: [FFU] Ensure compatibility with ansible 2.6. https://review.openstack.org/639169 | 10:11 |
| *** shyam89 has joined #tripleo | 10:11 | |
| *** wolverineav has quit IRC | 10:14 | |
| *** skramaja_ has joined #tripleo | 10:27 | |
| dtantsur | hi ovn folks. what is broken here? resources.ServiceChain: Property error: resources[37].properties: Property DockerOvnControllerImage not assigned | 10:28 |
| dtantsur | (essentially the default deployed-server installation) | 10:28 |
| *** skramaja has quit IRC | 10:29 | |
| dtantsur | jaosorior: can it be that the deployed-server roles were not updated ^^^? | 10:30 |
| openstackgerrit | Natal Ngétal proposed openstack/tripleo-puppet-elements master: [Documentation] Migrate the format. https://review.openstack.org/627165 | 10:31 |
| jaosorior | dtantsur: where are you seeing that error? | 10:33 |
| dtantsur | jaosorior: a deployed-server deployment locally | 10:33 |
| jaosorior | dtantsur: are you using a custom role? | 10:33 |
| dtantsur | jaosorior: nope. line 23 in https://etherpad.openstack.org/p/tripleo-nova-less-deploy | 10:33 |
| dtantsur | this used to work on an a bit older undercloud. rebuilt mine today and now getting this. | 10:34 |
| jaosorior | dtantsur: uhm... maybe you have an older tripleo-common | 10:38 |
| *** _fragatina has joined #tripleo | 10:39 | |
| jaosorior | dtantsur: Ovn controller should be part of the newer one | 10:39 |
| dtantsur | jaosorior: rebased yesterday, can rebase again | 10:39 |
| jaosorior | dtantsur: what did you rebase? t-h-t? | 10:39 |
| dtantsur | jaosorior: tripleo-common is the only custom thing I have here | 10:39 |
| jaosorior | oho | 10:39 |
| jaosorior | dtantsur: how do you update it? | 10:39 |
| dtantsur | you shouldn't have asked ;) lemme show you | 10:40 |
| dtantsur | jaosorior: http://paste.openstack.org/show/742646/ | 10:40 |
| jaosorior | haha wowo | 10:40 |
| jaosorior | well... maybe rebasing works | 10:40 |
| jaosorior | dtantsur: you might also need a newer t-h-t | 10:40 |
| dtantsur | jaosorior: won't I get today's t-h-t if I build oooq with master-tripleo-ci? | 10:41 |
| jaosorior | right, you should | 10:41 |
| *** florianf has quit IRC | 10:45 | |
| openstackgerrit | Cédric Jeanneret proposed openstack/puppet-tripleo master: Ensure we keep enough files, while ensuring their content is clean https://review.openstack.org/616945 | 10:46 |
| dtantsur | jaosorior: I wild guess: I already use a checkout of tripleoclient, maybe I should rebase it as well? | 10:47 |
| openstackgerrit | Sergii Golovatiuk proposed openstack/tripleo-heat-templates master: [FFU] Ensure compatibility with ansible 2.6. https://review.openstack.org/639169 | 10:48 |
| dtantsur | hmm, seems recent enough as well | 10:48 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: [DNM] Force periodic https://review.openstack.org/639301 | 10:50 |
| *** dciabrin has quit IRC | 10:50 | |
| *** dciabrin has joined #tripleo | 10:50 | |
| *** holser_ is now known as holser|lunch | 10:52 | |
| openstackgerrit | Lukas Bezdicka proposed openstack/tripleo-heat-templates stable/rocky: [FFU] Ensure compatibility with ansible 2.6. https://review.openstack.org/639303 | 10:56 |
| openstackgerrit | Jakub Libosvar proposed openstack-infra/tripleo-ci master: DNM: Trigger 012 scenario https://review.openstack.org/639304 | 10:56 |
| *** shyamb has joined #tripleo | 10:57 | |
| openstackgerrit | Lukas Bezdicka proposed openstack/tripleo-heat-templates stable/queens: [FFU] Ensure compatibility with ansible 2.6. https://review.openstack.org/639305 | 10:57 |
| ramishra | dtantsur: May be you're missing https://review.openstack.org/#/c/604953/ or something related to it, your issue seems similar to https://bugzilla.redhat.com/show_bug.cgi?id=1629599 | 10:59 |
| openstack | bugzilla.redhat.com bug 1629599 in openstack-tripleo-common "OVN CI jobs failed because DockerOvnControllerImage not assigned" [Urgent,Closed: errata] - Assigned to sbaker | 10:59 |
| *** shyam89 has quit IRC | 10:59 | |
| *** bfournie has quit IRC | 11:00 | |
| shyamb | Hi jaosorior: ironic and heat logs does not show anything | 11:01 |
| shyamb | I mean any failures | 11:01 |
| shyamb | boot is not working | 11:01 |
| shyamb | Do we still support kvm based overcloud vms? | 11:02 |
| shyamb | using vbmc driver? | 11:02 |
| *** gfidente has joined #tripleo | 11:02 | |
| shyamb | osp13 was working well | 11:02 |
| dtantsur | ramishra: I definitely have patches from 5 months ago :) | 11:02 |
| shyamb | osp14 documentation does not mention about this driver | 11:02 |
| ramishra | dtantsur: sure, I thought your custom rebase may have messed it up;) | 11:03 |
| ramishra | but yeah could be some new issue | 11:04 |
| dtantsur | I'll check, but I haven't had any merge conflicts. My code touches files of its own. | 11:04 |
| openstackgerrit | Quique Llorente proposed openstack-infra/tripleo-ci master: DNM: Add py3/f28 support to tripleo-container-build-base https://review.openstack.org/636160 | 11:06 |
| *** shyamb has quit IRC | 11:08 | |
| paramite | skramaja_, thanks | 11:08 |
| *** florianf has joined #tripleo | 11:08 | |
| openstackgerrit | Chandan Kumar proposed openstack/tripleo-quickstart-extras master: Standardize os_tempest playbook and set vars dynamically https://review.openstack.org/639310 | 11:17 |
| *** dciabrin has quit IRC | 11:24 | |
| *** dciabrin_ has joined #tripleo | 11:24 | |
| *** shyamb has joined #tripleo | 11:24 | |
| *** thrash|g0ne is now known as thrash | 11:28 | |
| *** udesale has quit IRC | 11:29 | |
| *** hberaud is now known as hberaud|lunch | 11:29 | |
| openstackgerrit | waleed mousa proposed openstack/os-net-config master: Configure switchdev mode in SR-IOV in os-net-config https://review.openstack.org/633885 | 11:30 |
| *** ratailor has quit IRC | 11:31 | |
| *** dciabrin_ is now known as dciabrin | 11:32 | |
| *** psachin has quit IRC | 11:40 | |
| *** wolverineav has joined #tripleo | 11:41 | |
| openstackgerrit | Marios Andreou proposed openstack-infra/tripleo-ci master: WIP: Add override repos into tripleo-ci tripleo-repos role https://review.openstack.org/638652 | 11:43 |
| *** wolverineav has quit IRC | 11:46 | |
| *** ccamacho has quit IRC | 11:50 | |
| EmilienM | weshay: when you have time today, please review https://review.openstack.org/#/c/635991 | 11:53 |
| EmilienM | marios: ^ | 11:53 |
| *** jpena is now known as jpena|lunch | 11:57 | |
| openstackgerrit | Dan Macpherson proposed openstack/ansible-role-openstack-operations master: Redis backup, restore, and validation tasks https://review.openstack.org/637467 | 11:57 |
| bandini | EmilienM: https://review.openstack.org/#/c/638518/ quick one / tested and acked by the networking folks, please | 11:58 |
| EmilienM | bandini: ack | 11:58 |
| bandini | EmilienM: merci ;) | 11:58 |
| marios | EmilienM: ack adding in next reviews | 11:59 |
| EmilienM | thx | 12:00 |
| panda|ruck | can someone from neutron take a look at https://bugs.launchpad.net/tripleo/+bug/1817705 ? blocking master promotion. | 12:02 |
| openstack | Launchpad bug 1817705 in tripleo "periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master fails tempest test related to DHCP agent" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 12:02 |
| openstackgerrit | Gael Chamoulaud proposed openstack/tripleo-common master: Use ANSIBLE_BECOME_FLAGS instead of ANSIBLE_SUDO_FLAGS https://review.openstack.org/639320 | 12:03 |
| *** quiquell is now known as quiquell|lunch | 12:04 | |
| panda|ruck | mmmh lunch | 12:05 |
| Tengu | have a good one :) | 12:06 |
| Tengu | hey folks, btw, PTAL? https://review.openstack.org/#/c/637729/ note: the new check isn't yet active in the current state, would love to get some inputs as to WHERE I can get it working (CI folks, mostly ;)) | 12:06 |
| *** rfolco|rover has quit IRC | 12:11 | |
| *** rfolco|rover has joined #tripleo | 12:11 | |
| *** ccamacho has joined #tripleo | 12:12 | |
| *** shyamb has quit IRC | 12:13 | |
| *** janki has joined #tripleo | 12:14 | |
| *** jchhatbar has quit IRC | 12:15 | |
| *** quiquell|lunch is now known as quiquell | 12:15 | |
| *** raildo has joined #tripleo | 12:16 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Switch scenario00{1,4}-standalone to Ceph bluestore https://review.openstack.org/547682 | 12:22 |
| *** holser|lunch is now known as holser_ | 12:22 | |
| *** janki has quit IRC | 12:24 | |
| *** janki has joined #tripleo | 12:24 | |
| *** holser__ has joined #tripleo | 12:25 | |
| *** srini_ has joined #tripleo | 12:26 | |
| *** sanjayu_ has joined #tripleo | 12:26 | |
| *** paramite|clone has joined #tripleo | 12:26 | |
| *** rf0lc0 has joined #tripleo | 12:26 | |
| *** dmellado_ has joined #tripleo | 12:28 | |
| *** janki has quit IRC | 12:29 | |
| *** beekneemech has joined #tripleo | 12:29 | |
| *** dprince has joined #tripleo | 12:30 | |
| *** gouthamr has quit IRC | 12:30 | |
| *** rnoriega has quit IRC | 12:30 | |
| *** avivgt has quit IRC | 12:30 | |
| *** cschwede has quit IRC | 12:30 | |
| *** haleyb has quit IRC | 12:30 | |
| *** cgoncalves has quit IRC | 12:30 | |
| *** owalsh has quit IRC | 12:30 | |
| *** toure has quit IRC | 12:30 | |
| *** sri_ has quit IRC | 12:30 | |
| *** dpeacock has quit IRC | 12:30 | |
| *** dmellado has quit IRC | 12:30 | |
| *** zzzeek has quit IRC | 12:30 | |
| *** rfolco|rover has quit IRC | 12:30 | |
| *** gfidente has quit IRC | 12:30 | |
| *** florianf has quit IRC | 12:30 | |
| *** holser_ has quit IRC | 12:30 | |
| *** ramishra has quit IRC | 12:30 | |
| *** saneax has quit IRC | 12:30 | |
| *** jaganathan has quit IRC | 12:30 | |
| *** bnemec has quit IRC | 12:30 | |
| *** paramite has quit IRC | 12:30 | |
| *** hjensas has quit IRC | 12:30 | |
| *** openstackgerrit has quit IRC | 12:30 | |
| *** dmellado_ is now known as dmellado | 12:30 | |
| *** owalsh_ has joined #tripleo | 12:31 | |
| *** gfidente has joined #tripleo | 12:31 | |
| *** hjensas has joined #tripleo | 12:31 | |
| *** zzzeek has joined #tripleo | 12:31 | |
| *** avivgt has joined #tripleo | 12:31 | |
| *** rnoriega has joined #tripleo | 12:31 | |
| *** dxiri has joined #tripleo | 12:31 | |
| *** cschwede_ has joined #tripleo | 12:32 | |
| *** cfontain has joined #tripleo | 12:32 | |
| *** cgoncalves has joined #tripleo | 12:32 | |
| *** gouthamr has joined #tripleo | 12:33 | |
| *** rf0lc0 is now known as rfolco|rover | 12:33 | |
| *** ramishra_ has joined #tripleo | 12:38 | |
| *** shyamb has joined #tripleo | 12:41 | |
| *** dpeacock has joined #tripleo | 12:42 | |
| *** _fragatina has quit IRC | 12:44 | |
| *** shyamb has quit IRC | 12:46 | |
| *** morazi has joined #tripleo | 12:47 | |
| *** hberaud|lunch is now known as hberaud | 12:48 | |
| *** shyamb has joined #tripleo | 12:50 | |
| *** ccamacho has quit IRC | 12:52 | |
| weshay | jaosorior I think tihs channel lost it's alerts | 12:55 |
| Tengu | ah, that's why it's sooo silent? :) bot's dead? | 13:01 |
| jaosorior | weshay: did it? who can we poke? | 13:01 |
| weshay | jaosorior whomever has ops | 13:03 |
| *** gfidente has quit IRC | 13:04 | |
| *** beagles has left #tripleo | 13:04 | |
| *** beagles has joined #tripleo | 13:04 | |
| jaosorior | mwhahaha, EmilienM ^^ | 13:05 |
| *** ccamacho has joined #tripleo | 13:05 | |
| EmilienM | ops to do what | 13:05 |
| *** EmilienM sets mode: +o jaosorior | 13:05 | |
| *** EmilienM sets mode: +o weshay | 13:05 | |
| *** EmilienM sets mode: +o Tengu | 13:05 | |
| * EmilienM hides | 13:05 | |
| weshay | EmilienM the list the alert bugs | 13:06 |
| * Tengu ducks | 13:06 | |
| EmilienM | weshay: isn't it a script run by someone | 13:06 |
| EmilienM | I'm wondering if beekneemech or dprince know | 13:06 |
| dprince | yeah, not sure. mwhahaha? | 13:08 |
| EmilienM | alex isn't running it for sure | 13:08 |
| *** agurenko has quit IRC | 13:08 | |
| *** abishop has joined #tripleo | 13:08 | |
| jaosorior | owalsh_: is this still an issue https://bugs.launchpad.net/tripleo/+bug/1759945 ? | 13:11 |
| openstack | Launchpad bug 1759945 in tripleo "Review wsgi process/thread config" [High,Triaged] - Assigned to Oliver Walsh (owalsh) | 13:11 |
| jaosorior | quiquell: is this still an issue https://bugs.launchpad.net/tripleo/+bug/1762351 ? | 13:12 |
| openstack | Launchpad bug 1762351 in tripleo "periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-queens-upload is timeout in virt-customize step" [High,Triaged] - Assigned to Quique Llorente (quiquell) | 13:12 |
| jaosorior | arxcruz, quiquell: What about this https://bugs.launchpad.net/tripleo/+bug/1762419 ? still an issue? | 13:14 |
| openstack | Launchpad bug 1762419 in tripleo "Collect logs task are not collecting the virt-customize logs from periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-$release-upload " [High,Triaged] - Assigned to Matt Young (halcyondude) | 13:14 |
| *** mcornea has joined #tripleo | 13:14 | |
| bogdando | PTAL https://review.openstack.org/#/c/635161/ | 13:14 |
| *** sdake has joined #tripleo | 13:15 | |
| jaosorior | jfrancoa, jistr: Do you guys know if this is an issue still https://bugs.launchpad.net/tripleo/+bug/1764698 ? | 13:17 |
| openstack | Launchpad bug 1764698 in tripleo "Deal with "TLS by default" change on Q->R upgrade" [High,Triaged] | 13:17 |
| quiquell | jaosorior: the issue with nester virtualization I think is still there | 13:17 |
| quiquell | jaosorior: very transitory though | 13:17 |
| quiquell | jaosorior: we have same issue with CI at current reproducer | 13:17 |
| jaosorior | quiquell: uuuuh... which bug number again? | 13:17 |
| * jaosorior too many tabs open | 13:18 | |
| quiquell | jaosorior: https://bugs.launchpad.net/tripleo/+bug/1762351 | 13:18 |
| openstack | Launchpad bug 1762351 in tripleo "periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset002-queens-upload is timeout in virt-customize step" [High,Triaged] - Assigned to Quique Llorente (quiquell) | 13:18 |
| jistr | jaosorior: re TLS i don't know. Personally i haven't hit anything, so from my perspective we could close and reopen if we discover some issue. | 13:18 |
| jaosorior | jistr: thanks. | 13:19 |
| *** owalsh_ is now known as owalsh | 13:19 | |
| quiquell | jaosorior: some RDO hypervisors have problem with nester virtualization and centos | 13:19 |
| quiquell | jaosorior: https://bugzilla.redhat.com/show_bug.cgi?id=1565179 | 13:19 |
| openstack | bugzilla.redhat.com bug 1565179 in qemu-kvm-rhev "qemu-kvm fails when run with -machine accel=kvm:tcg nested KVM" [Unspecified,Closed: insufficient_data] - Assigned to pbonzini | 13:19 |
| owalsh | jaosorior: it's more of a "we should take a look at this" than a bug | 13:20 |
| shyamb | Hi | 13:20 |
| shyamb | on osp14, during overcloud deployment kvm based vms are not getting powered on | 13:20 |
| jaosorior | owalsh: anybody planning to do so? | 13:20 |
| shyamb | I am using vbmc driver | 13:20 |
| owalsh | jaosorior, not that I know of | 13:21 |
| jaosorior | ok, I'll leave that bug be for now. | 13:21 |
| shyamb | Does rocky tripleo supports vbmc based power management? | 13:21 |
| Tengu | hey folks, quick reminder about the Validation Framework meeting in 8 minutes :) | 13:22 |
| jaosorior | mcornea: hey, do you know if this is still an issue https://bugs.launchpad.net/tripleo/+bug/1765552 ? | 13:23 |
| openstack | Launchpad bug 1765552 in tripleo "FFU: deploy_steps_playbook.yaml playbook fails because rabbitmq_init_bundle container is unable to successfully run Executing: 'rabbitmqctl status | grep -F "{rabbit,"'" [High,Triaged] | 13:23 |
| *** trown|outtypewww is now known as trown | 13:24 | |
| *** florianf has joined #tripleo | 13:24 | |
| *** agopi has quit IRC | 13:29 | |
| jaosorior | weshay, marios: I'm checking this bug out https://bugs.launchpad.net/tripleo/+bug/1771082 . Aren't we already collecting console logs? | 13:30 |
| openstack | Launchpad bug 1771082 in tripleo "[rfe] TripleO CI must collect and publish console logs from fake overcloud nodes during introspection and deployment" [High,Triaged] | 13:30 |
| panda|ruck | jaosorior: yes, we do now | 13:31 |
| jaosorior | panda|ruck: thanks | 13:31 |
| marios | jaosorior: ack thanks panda looks like old bug then | 13:33 |
| *** jpena|lunch is now known as jpena | 13:33 | |
| fultonj | ykarel: saw your ping from 6 hours ago | 13:33 |
| *** rlandy has joined #tripleo | 13:33 | |
| fultonj | ykarel: the commit message for https://review.openstack.org/#/c/638363 explains what you saw. it was expected behavior, thus the jobs were set to non-voting temporarily so both jobs could land. | 13:34 |
| fultonj | both jobs have merged, we now just need https://review.openstack.org/#/c/638364 to land so 001 and 004 can vote again | 13:34 |
| *** ansmith has joined #tripleo | 13:35 | |
| ykarel | fultonj, see https://bugs.launchpad.net/tripleo/+bug/1817688 and https://review.openstack.org/#/c/637196/6/roles/ceph-loop-device/tasks/main.yml | 13:35 |
| openstack | Launchpad bug 1817688 in tripleo "job that test ceph in periodic pipeline for stable branches are getting the wrong configuration" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 13:35 |
| ykarel | fultonj, master is fine, issue is in stable branches | 13:35 |
| *** suuuper has joined #tripleo | 13:35 | |
| owalsh | shyamb: vbmc is used for CI so it should work. If the VM is being created in libvirt but failing to start then there is a debugging guide here - https://kashyapc.fedorapeople.org/virt/Debugging-libvirt-QEMU-in-OpenStack-DevConf-2016.pdf | 13:35 |
| fultonj | ykarel: ok, let me dig into this to fix stable | 13:36 |
| ykarel | fultonj, sync with panda|ruck i think he is already on it | 13:37 |
| ykarel | just to avoid duplicate work | 13:37 |
| fultonj | ykarel: ack | 13:37 |
| owalsh | shyamb: I'd start by looking at the logs on the host - could be something obvious like not enough RAM/DISK etc... | 13:37 |
| *** dtantsur is now known as dtantsur|afk | 13:38 | |
| *** shyamb has quit IRC | 13:39 | |
| fultonj | panda|ruck: for 1817688 are you working on making the release variable available earlier? | 13:40 |
| panda|ruck | fultonj: yes | 13:41 |
| fultonj | panda|ruck: anything i can do to help? | 13:41 |
| panda|ruck | fultonj: still not sure if it's possible | 13:41 |
| fultonj | panda|ruck: for what versions do we have stable jobs? | 13:41 |
| fultonj | rocky, queens could use bluestore | 13:42 |
| fultonj | do we also need pike though? | 13:42 |
| fultonj | i thought i saw ci_branch working before it merged; is the issue that it only worked for master? | 13:43 |
| panda|ruck | fultonj: yes | 13:44 |
| fultonj | is that yes for all 3 questions? | 13:45 |
| panda|ruck | fultonj: yes :) sorry I saw only the last one | 13:46 |
| panda|ruck | fultonj: so anything you can do to help, probably not, I think we can move this role around, since it' used really only during deployment | 13:47 |
| panda|ruck | fultonj: we have stable jobs down to pike | 13:47 |
| fultonj | down to pike | 13:47 |
| panda|ruck | fultonj: and yes, the job passed because the ci_branch will always be master for changes in tripleo-ci | 13:48 |
| panda|ruck | fultonj: rocky queens pike | 13:48 |
| panda|ruck | ykarel: why is pike still relevant ? seems EOL pretty everywhere ? | 13:49 |
| ykarel | panda|ruck, in upstream it's not EOL afaik | 13:49 |
| fultonj | rocky and queens could go to bluestore as both are being used with ceph-ansible 3.2 | 13:49 |
| *** haleyb has joined #tripleo | 13:50 | |
| panda|ruck | fultonj: I think it's a start then, I can help you test the bluestore with queens and rocky branches | 13:50 |
| fultonj | panda|ruck: ok, i can backport | 13:50 |
| fultonj | panda|ruck: you think it's too dificult to move that var? | 13:51 |
| fultonj | "move it" as in set it earlier in the process | 13:51 |
| ykarel | https://releases.openstack.org/ | 13:51 |
| panda|ruck | ykarel: you're right, but extended maintenance stops 3 match | 13:51 |
| panda|ruck | March | 13:51 |
| fultonj | 5 days | 13:52 |
| ykarel | yeah only few days left | 13:52 |
| panda|ruck | fultonj: not "too" difficult, but the release variable is pivot to a lot of thinks I have to rotate around it. | 13:52 |
| panda|ruck | things* | 13:52 |
| panda|ruck | fultonj: what you need to do to backport ? in tripleo-ci should be enough to remove the branch restriction, and add branchful tests. | 13:53 |
| fultonj | https://review.openstack.org/#/c/547682/ | 13:53 |
| fultonj | since LVM is there i backport ^ to rocky and queens and they will work | 13:54 |
| fultonj | panda|ruck: this is just a proposal to solve the problem differently ^ | 13:55 |
| mcornea | jaosorior: hey o/ I think it's not anymore, it's pretty old one | 13:56 |
| *** mjturek has joined #tripleo | 13:56 | |
| jaosorior | mcornea: ack, will close it. | 13:56 |
| panda|ruck | fultonj: in rocky and queens standalones are not the main test jobs, we still rely on multinode jobs. You'll need to change scenario001/4-multinode. | 13:56 |
| *** agopi has joined #tripleo | 13:58 | |
| *** beekneemech is now known as bnemec | 13:59 | |
| bnemec | EmilienM: weshay: I think it was being run by derekh | 14:00 |
| jaosorior | #startmeeting tripleo | 14:00 |
| openstack | Meeting started Tue Feb 26 14:00:55 2019 UTC and is due to finish in 60 minutes. The chair is jaosorior. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| *** openstack changes topic to " (Meeting topic: tripleo)" | 14:00 | |
| openstack | The meeting name has been set to 'tripleo' | 14:00 |
| jaosorior | #topic agenda | 14:01 |
| *** openstack changes topic to "agenda (Meeting topic: tripleo)" | 14:01 | |
| jaosorior | * Review past action items | 14:01 |
| jaosorior | * One off agenda items | 14:01 |
| jaosorior | * Squad status | 14:01 |
| jaosorior | * Bugs & Blueprints | 14:01 |
| jaosorior | * Projects releases or stable backports | 14:01 |
| jaosorior | * Specs | 14:01 |
| jaosorior | * open discussion | 14:01 |
| jaosorior | Anyone can use the #link, #action and #info commands, not just the moderatorǃ | 14:01 |
| jaosorior | Hey folks! who's around? | 14:01 |
| beagles | o/ | 14:01 |
| dpeacock | o/ | 14:01 |
| Tengu | «o/ | 14:01 |
| bogdando | o/ | 14:01 |
| chandankumar | \o/ | 14:01 |
| ksambor | o/ | 14:01 |
| panda|ruck | o/ | 14:02 |
| marios | o/ | 14:02 |
| dprince | hi | 14:03 |
| rfolco|rover | o/ | 14:03 |
| fmount | o/ | 14:04 |
| jfrancoa | o/ | 14:04 |
| fultonj | o/ | 14:04 |
| *** aissat-mus has joined #tripleo | 14:04 | |
| weshay | 0/ | 14:04 |
| jaosorior | #topic review past action items | 14:04 |
| *** openstack changes topic to "review past action items (Meeting topic: tripleo)" | 14:04 | |
| jaosorior | None. | 14:04 |
| *** zaneb has quit IRC | 14:04 | |
| jaosorior | #topic one off agenda items | 14:05 |
| *** openstack changes topic to "one off agenda items (Meeting topic: tripleo)" | 14:05 | |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-meeting-items | 14:05 |
| jaosorior | The first topic is: (rfolco): Quick updates on Fedora 28 containers build: tripleo-repos tool now supports Fedora 28 for setting repos and mirrors. Check job to be added to RDO given the nuances around unmergeable Kolla patches. Epic: https://tree.taiga.io/project/tripleo-ci-board/epic/445. | 14:05 |
| jaosorior | rfolco|rover: ^^ | 14:06 |
| weshay | marios, ^ | 14:06 |
| rfolco|rover | The goal is to have a check jon in rdo at the end of this sprint to track progress on f28 containers build. That's it. Happy to provide more detail if any questions. | 14:06 |
| derekh | bnemec: EmilienM ya it was me, a old HD died where I was running it, I'll sort it out and get it back, | 14:06 |
| rfolco|rover | check job* | 14:06 |
| marios | jaosorior: rfolco|rover is rover this sprint so we quiquell zbr|out and i are trying to get those merged see https://tree.taiga.io/project/tripleo-ci-board/us/650?milestone=220350 for info and we can discuss some more in the ci meeting next if anyeone is interested in more | 14:06 |
| marios | jaosorior: quiquell has been trying to get the job to run in rdo jobs for the upstream at https://review.openstack.org/#/c/636160 | 14:07 |
| *** zaneb has joined #tripleo | 14:07 | |
| marios | jaosorior: main news item compared to last week is that we will only be runing in in check non voting and in rdo for now not upstream | 14:07 |
| rfolco|rover | thanks marios | 14:09 |
| *** vinaykns has joined #tripleo | 14:10 | |
| jaosorior | marios: thanks for the update | 14:10 |
| jaosorior | #topic Squad status | 14:11 |
| *** openstack changes topic to "Squad status (Meeting topic: tripleo)" | 14:11 | |
| jaosorior | ci | 14:11 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-ci-squad-meeting | 14:12 |
| jaosorior | upgrade | 14:12 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-upgrade-squad-status | 14:12 |
| jaosorior | containers | 14:12 |
| jaosorior | #link https://trello.com/b/S8TmOU0u/tripleo-podman | 14:12 |
| jaosorior | integration | 14:12 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-integration-squad-status | 14:12 |
| jaosorior | ui/cli | 14:12 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-ui-cli-squad-status | 14:12 |
| jaosorior | validations | 14:12 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-validations-squad-status | 14:12 |
| jaosorior | networking | 14:12 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-networking-squad-status | 14:12 |
| jaosorior | security | 14:12 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-security-squad | 14:12 |
| jaosorior | edge | 14:12 |
| jaosorior | #link https://etherpad.openstack.org/p/tripleo-edge-squad-status | 14:12 |
| jaosorior | Any squad wanting to bring up their status, or a topic for the general public? | 14:12 |
| Tengu | Validation: we need reviewers on https://review.openstack.org/636930 and https://review.openstack.org/636581 - those two are the base of the whole work, and it would be good to get them merged sooner than later. | 14:12 |
| *** ykarel is now known as ykarel|away | 14:13 | |
| Tengu | As next steps will involve the addition of ansible-runner to those two changes, we're stuck for now :). | 14:13 |
| Tengu | help is really appreciate. | 14:13 |
| Tengu | (end of communication ;)) | 14:13 |
| *** florianf_ has joined #tripleo | 14:14 | |
| jaosorior | Tengu: added myself as a reviewer. Thanks for bringing that up | 14:15 |
| Tengu | :+1: | 14:15 |
| Tengu | :) | 14:15 |
| *** florianf has quit IRC | 14:17 | |
| jaosorior | #topic bugs & blueprints | 14:17 |
| *** openstack changes topic to "bugs & blueprints (Meeting topic: tripleo)" | 14:17 | |
| jaosorior | #link https://launchpad.net/tripleo/+milestone/stein-3 | 14:17 |
| jaosorior | For Stein we currently have 21 (for stein-3) blueprints open in Launchpad. | 14:17 |
| jaosorior | Bugs: 522 (-45) stein-3 in Launchpad. 104 (0) open Storyboard bugs. | 14:17 |
| openstack | bug 522 in Launchpad itself "translation percentage off the scale" [Medium,Fix released] https://launchpad.net/bugs/522 - Assigned to Carlos Perelló Marín (carlos) | 14:17 |
| jaosorior | #link https://storyboard.openstack.org/#!/project_group/76 | 14:17 |
| *** jcoufal has joined #tripleo | 14:18 | |
| jaosorior | #topic projects releases or stable backports | 14:20 |
| *** openstack changes topic to "projects releases or stable backports (Meeting topic: tripleo)" | 14:20 | |
| *** sdake has quit IRC | 14:21 | |
| jaosorior | #topic projects releases or stable backports | 14:23 |
| *** openstack changes topic to "projects releases or stable backports (Meeting topic: tripleo)" | 14:23 | |
| *** gfidente has joined #tripleo | 14:24 | |
| *** manojawa has joined #tripleo | 14:24 | |
| manojawa | Hello All... My existing cluster has - 1 X Director, 3 X Controllers & 10 X Compute nodes. All Baremetal. When I tried to add 11th node, it is failing with error "resources.AllNodesDeploySteps: resources.ControllerDeployment_Step5: Error: resources[2]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 6 ". Need help in understanding and fixing the issue. | 14:25 |
| *** ade_lee has joined #tripleo | 14:25 | |
| manojawa | I can provide any required details about my setup | 14:25 |
| *** tzumainn has joined #tripleo | 14:26 | |
| manojawa | I searched a lot. I see lot of links with the same error "Deployment exited with non-zero status code: 6", but none of them have any resolution | 14:27 |
| jaosorior | #topic specs | 14:28 |
| *** openstack changes topic to "specs (Meeting topic: tripleo)" | 14:28 | |
| jaosorior | #link https://review.openstack.org/#/q/project:openstack/tripleo-specs+status:open | 14:28 |
| manojawa | or any root cause | 14:28 |
| jaosorior | Any specs folks want to bring attention to? | 14:28 |
| fultonj | manojawa: we're in a meeting ATM | 14:28 |
| manojawa | Oh.. I am sorry | 14:28 |
| fultonj | manojawa: no prob, you can ask when we're done :) | 14:29 |
| jaosorior | #topic open discussion | 14:29 |
| *** openstack changes topic to "open discussion (Meeting topic: tripleo)" | 14:29 | |
| jaosorior | Anything else that folks want to bring up to the meeting? | 14:29 |
| fultonj | jaosorior: https://bugs.launchpad.net/tripleo/+bug/1817688 | 14:30 |
| openstack | Launchpad bug 1817688 in tripleo "job that test ceph in periodic pipeline for stable branches are getting the wrong configuration" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 14:30 |
| fultonj | jaosorior: panda|ruck and i are working on this | 14:30 |
| fultonj | jobs which are broken are not voting ATM | 14:30 |
| manojawa | fultonj: Thank you ! By what time will the meeting be done from now? | 14:30 |
| fultonj | plan is in bug linked above , if any questions we're here but wanted to point it out | 14:31 |
| fultonj | jaosorior: that's all (unless there's questions) | 14:31 |
| jaosorior | ack, thanks for the update. | 14:31 |
| weshay | fultonj I'm going to raise this in the ci community mtg in a few.. if you could join that would be super | 14:31 |
| panda|ruck | fultonj: jaosorior we'll discuss the bug more in the community meeting | 14:31 |
| fultonj | weshay: ok | 14:31 |
| bogdando | Wrt ordering our CI jobs on the linting/unit results, I revived https://review.openstack.org/#/c/568543/ see also ML topic http://lists.openstack.org/pipermail/openstack-discuss/ | 14:33 |
| bogdando | 2019-February/003142.html . My patch basically fails on zuul's "graphs freezing" and I don't know how to proceed with it. So if anyone wish to handover/assist... | 14:33 |
| EmilienM | o/ (late) | 14:33 |
| bogdando | #link http://lists.openstack.org/pipermail/openstack-discuss/003142.html | 14:33 |
| *** sdake has joined #tripleo | 14:33 | |
| bogdando | #link http://lists.openstack.org/pipermail/openstack-discuss/2019-February/003142.html | 14:34 |
| weshay | ah thanks | 14:34 |
| bogdando | IMO we can make openstack-infra folks happy dancing if introduced such an ordering for tripleo CI pools) | 14:35 |
| weshay | I'll raise this as well in the ci community | 14:36 |
| weshay | mtg | 14:36 |
| bogdando | thanks | 14:36 |
| weshay | bogdando++ | 14:36 |
| marios | bogdando: seems sensible btw first link is 404 for me | 14:37 |
| bogdando | marios: yeah, sorry for that | 14:37 |
| marios | bogdando: but yeah if we can have same for py27/pep8 etc for example right | 14:37 |
| marios | before even running jobs | 14:37 |
| bogdando | right | 14:37 |
| marios | there may be more to do but that is just the simple/obvious case | 14:37 |
| *** mjturek has quit IRC | 14:38 | |
| panda|ruck | I have concens, we don't run lin and tox for every change, only the ones in emit-releases-script | 14:38 |
| panda|ruck | also the templates are used in other repos than tripleo-ci | 14:39 |
| bogdando | panda|ruck: sure, the details should be adjusted | 14:39 |
| bogdando | just that the very idea seems doesn't work for zuul, or that was me doing it wrong | 14:39 |
| marios | bogdando: we use it in periodics | 14:40 |
| marios | bogdando: like the container build job is dependency for all the others | 14:40 |
| bogdando | marios: oh, then should be not a biggie to catch up that for standalone jobs at least | 14:40 |
| marios | https://github.com/rdo-infra/review.rdoproject.org-config/blob/e1d81ff7a4e1f19eb4ce1577f0f370acec3e0adc/zuul.d/tripleo.yaml#L41-L106 | 14:41 |
| marios | bogdando: eg ^ | 14:41 |
| panda|ruck | marios: yes, but in tripleo-ci it's very complicated, we serve multiple repos, and the jobs listed in the current dependencies are not run every time | 14:41 |
| marios | panda|ruck: ack sure i just agreed with the idea, doesn't mean it will work everywhere ack on us not running th elinting etc for each change | 14:42 |
| weshay | panda|ruck are other projects using our project-template? | 14:42 |
| marios | panda|ruck: but in general i like the idea | 14:42 |
| weshay | or just the jobs.. I think just the job | 14:42 |
| panda|ruck | let's discuss in the community meeting | 14:42 |
| panda|ruck | weshay: templates also | 14:42 |
| weshay | panda|ruck hrm.. k.. please grab me a link where a project outside of tripleo is using tripleo-multinode-container-minimal | 14:43 |
| weshay | or other example | 14:43 |
| panda|ruck | weshay: tht | 14:43 |
| panda|ruck | weshay: https://github.com/openstack/tripleo-heat-templates/blob/master/zuul.d/layout.yaml#L4 | 14:43 |
| bogdando | so more places to amend | 14:44 |
| weshay | hrm | 14:44 |
| weshay | that's not outside of tripleo though | 14:44 |
| bogdando | lets just to fix that freezer cuz warm beer is not cool | 14:44 |
| weshay | lolz | 14:45 |
| panda|ruck | motion to move to community meeting | 14:46 |
| *** ykarel|away is now known as ykarel | 14:46 | |
| weshay | aye | 14:47 |
| *** yprokule has quit IRC | 14:49 | |
| jaosorior | #endmeeting | 14:49 |
| *** openstack changes topic to "CI Status: Should be green, recheck if needed | https://docs.openstack.org/tripleo-docs/latest/" | 14:49 | |
| openstack | Meeting ended Tue Feb 26 14:49:44 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:49 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/tripleo/2019/tripleo.2019-02-26-14.00.html | 14:49 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/tripleo/2019/tripleo.2019-02-26-14.00.txt | 14:49 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/tripleo/2019/tripleo.2019-02-26-14.00.log.html | 14:49 |
| weshay | thanks.. | 14:50 |
| *** mjturek has joined #tripleo | 14:50 | |
| rfolco|rover | CI community meeting (office hours) starts now at https://bluejeans.com/4113567798 | 14:51 |
| *** mjturek has quit IRC | 14:52 | |
| *** sdake has quit IRC | 14:55 | |
| *** aissat-mus has quit IRC | 14:55 | |
| *** ykarel_ has joined #tripleo | 14:59 | |
| *** sdake has joined #tripleo | 15:00 | |
| *** ykarel has quit IRC | 15:00 | |
| *** aedc has joined #tripleo | 15:02 | |
| *** mjturek has joined #tripleo | 15:03 | |
| *** mjturek has quit IRC | 15:03 | |
| manojawa | Hello All... My existing cluster has - 1 X Director, 3 X Controllers & 10 X Compute nodes. All Baremetal. When I tried to add 11th node, it is failing with error "resources.AllNodesDeploySteps: resources.ControllerDeployment_Step5: Error: resources[2]: Deployment to server failed: deploy_status_code : Deployment exited with non-zero status code: 6 ". Need help in understanding and fixing the issue. | 15:04 |
| manojawa | I searched a lot. I see lot of links with the same error "Deployment exited with non-zero status code: 6", but none of them have any resolution | 15:04 |
| manojawa | So, need help in fixing the issue | 15:04 |
| manojawa | Please let me know what all logs/command outputs I need to provide | 15:05 |
| aedc | Hello, where can we create a ticket on a bug, apart from https://bugzilla.redhat.com (tripleo-rdo) ? | 15:06 |
| aedc | PMD: net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory | 15:06 |
| aedc | (tripleomaster/centos-binary-ovn-controller:54c5a6de8ce5b9cfae83632a7d81000721d56071_786d88d2) | 15:07 |
| aedc | => latest | 15:07 |
| *** Vorrtex has joined #tripleo | 15:11 | |
| jistr | aedc: hi, the upstream tracker is https://bugs.launchpad.net/tripleo/+filebug | 15:18 |
| *** rbrady has joined #tripleo | 15:19 | |
| jistr | manojawa: hi, there should be some better output somewhere which should help you uncover the root cause | 15:19 |
| aedc | jistr: thks | 15:19 |
| jistr | http://tripleo.org/install/troubleshooting/troubleshooting-overcloud.html | 15:20 |
| jistr | http://tripleo.org/install/advanced_deployment/ansible_config_download.html | 15:20 |
| *** gbarros has joined #tripleo | 15:20 | |
| jistr | manojawa: ^ some troubleshooting links | 15:20 |
| *** owalsh_ has joined #tripleo | 15:24 | |
| *** owalsh has quit IRC | 15:25 | |
| *** gbarros has quit IRC | 15:26 | |
| *** owalsh_ is now known as owalsh | 15:26 | |
| *** gbarros has joined #tripleo | 15:29 | |
| aedc | https://bugs.launchpad.net/tripleo/+bug/1817743 | 15:30 |
| openstack | Launchpad bug 1817743 in tripleo "PMD: net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory" [Undecided,New] | 15:30 |
| bogdando | weshay, marios, panda|ruck: pushed some updates here https://review.openstack.org/#/q/topic:ci_pipelines+(status:open+OR+status:merged) , but that freezer still fails :/ | 15:31 |
| *** ykarel_ is now known as ykarel|away | 15:32 | |
| *** sanjayu_ has quit IRC | 15:34 | |
| *** sanjayu_ has joined #tripleo | 15:35 | |
| aedc | https://bugs.launchpad.net/tripleo/+bug/1789236 | 15:38 |
| openstack | Launchpad bug 1789236 in tripleo "OVN deployment is failing when used with OVS 2.10" [High,Fix released] - Assigned to Numan Siddique (numansiddique) | 15:38 |
| *** Chaserjim has joined #tripleo | 15:41 | |
| chandankumar | Hello | 15:42 |
| larsks | mwhahaha: does the ftype=0 check in https://github.com/openstack/ansible-role-container-registry/blob/master/tasks/docker.yml#L28 be limited to just checking /var/lib/docker? | 15:42 |
| larsks | Well, the first half and the second half of that sentence obviously didn't talk to eachother before hand, but you get the idea. | 15:43 |
| chandankumar | is there a way to export these values in tqe OS::TripleO::Services::CephMgr in playbook? | 15:43 |
| chandankumar | during runtime | 15:43 |
| mwhahaha | larsks: yes it just affects docker storage | 15:44 |
| mwhahaha | But that is the old default for <7.3 I think | 15:45 |
| mwhahaha | New systems shouldn't run into that problem | 15:45 |
| *** ccamacho has quit IRC | 15:51 | |
| *** ccamacho has joined #tripleo | 15:55 | |
| *** maufart__ has quit IRC | 15:56 | |
| *** ksambor has quit IRC | 15:56 | |
| *** sdake has quit IRC | 15:59 | |
| *** sdake has joined #tripleo | 16:01 | |
| *** skramaja_ has quit IRC | 16:01 | |
| *** sdake has quit IRC | 16:02 | |
| fultonj | regarding bug 1817688 proposal in comment #5 | 16:06 |
| openstack | bug 1817688 in tripleo "job that test ceph in periodic pipeline for stable branches are getting the wrong configuration" [Critical,Triaged] https://launchpad.net/bugs/1817688 - Assigned to Gabriele Cerami (gcerami) | 16:06 |
| fultonj | jaosorior: ^ | 16:06 |
| fultonj | mwhahaha: ^ | 16:06 |
| fultonj | weshay: (as per our conversation) | 16:07 |
| gfidente | chandankumar not sure what you're asking exactly | 16:08 |
| gfidente | chandankumar what is it you need to export? | 16:08 |
| *** gbarros has quit IRC | 16:10 | |
| chandankumar | gfidente: I want to set these values dynamically based on services imported https://review.openstack.org/#/c/639310/3/playbooks/tempest.yml@4 | 16:10 |
| chandankumar | gfidente: by the way congrats :-) | 16:10 |
| chandankumar | we are moving to a new role os_tempest for running tempest in future developed by OSA | 16:11 |
| *** gbarros has joined #tripleo | 16:16 | |
| *** gbarros has quit IRC | 16:16 | |
| *** paramite|clone has quit IRC | 16:17 | |
| *** sdake has joined #tripleo | 16:18 | |
| *** openstackgerrit has joined #tripleo | 16:20 | |
| openstackgerrit | Sagi Shnaidman proposed openstack/tripleo-quickstart master: WIP: run novajoin tempest tests https://review.openstack.org/639324 | 16:20 |
| *** jfrancoa has quit IRC | 16:20 | |
| *** suuuper has quit IRC | 16:20 | |
| *** gbarros has joined #tripleo | 16:21 | |
| openstackgerrit | Marios Andreou proposed openstack-infra/tripleo-ci master: WIP: Add override repos into tripleo-ci tripleo-repos role https://review.openstack.org/638652 | 16:24 |
| weshay | panda|ruck please add a skip list review for https://bugs.launchpad.net/tripleo/+bug/1817705 | 16:24 |
| openstack | Launchpad bug 1817705 in tripleo "periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master fails tempest test related to DHCP agent" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 16:24 |
| weshay | panda|ruck and then continue to debug w/ rfolco|rover | 16:24 |
| *** itlinux has joined #tripleo | 16:25 | |
| *** sdake has quit IRC | 16:25 | |
| *** sdake has joined #tripleo | 16:26 | |
| panda|ruck | weshay: ack | 16:27 |
| gfidente | chandankumar so I am not sure what the os_tempest role does | 16:28 |
| *** quiquell is now known as quiquell|off | 16:28 | |
| gfidente | chandankumar but it looks like data should flow the other way around, you enable particular parts of tempest based on how the heat stack is configured | 16:28 |
| gfidente | instead of enabling/disabling services based on tempest config | 16:29 |
| *** gbarros has quit IRC | 16:30 | |
| *** pcaruana has quit IRC | 16:32 | |
| *** sdake has quit IRC | 16:35 | |
| *** sdake has joined #tripleo | 16:35 | |
| *** cylopez has quit IRC | 16:36 | |
| *** sdake has quit IRC | 16:40 | |
| *** sdake_ has joined #tripleo | 16:41 | |
| *** ykarel|away has quit IRC | 16:41 | |
| *** ykarel has joined #tripleo | 16:42 | |
| *** sdake_ has quit IRC | 16:45 | |
| *** sdake has joined #tripleo | 16:46 | |
| *** sdake has quit IRC | 16:50 | |
| *** sdake has joined #tripleo | 16:52 | |
| *** _fragatina has joined #tripleo | 16:52 | |
| *** gfidente is now known as gfidente|afk | 16:56 | |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart-extras master: Adding test to the skiplist https://review.openstack.org/639386 | 16:58 |
| *** vinaykns has quit IRC | 16:59 | |
| *** moguimar has quit IRC | 16:59 | |
| *** bogdando has quit IRC | 17:00 | |
| *** wolverineav has joined #tripleo | 17:00 | |
| *** sdake has quit IRC | 17:00 | |
| *** ccamacho has quit IRC | 17:01 | |
| *** sdake_ has joined #tripleo | 17:02 | |
| *** vinaykns has joined #tripleo | 17:03 | |
| *** cfontain has quit IRC | 17:03 | |
| *** cfontain has joined #tripleo | 17:05 | |
| *** aedc has quit IRC | 17:07 | |
| *** _fragatina has quit IRC | 17:08 | |
| *** itlinux_ has joined #tripleo | 17:10 | |
| *** _fragatina has joined #tripleo | 17:10 | |
| *** cfontain has quit IRC | 17:10 | |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart-extras master: Adding test to the skiplist for OVN related updates https://review.openstack.org/639386 | 17:13 |
| *** itlinux has quit IRC | 17:13 | |
| *** sshnaidm is now known as sshnaidm|afk | 17:17 | |
| *** jpich has quit IRC | 17:17 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Drop the role service override for ovb https://review.openstack.org/637604 | 17:21 |
| *** morazi has quit IRC | 17:21 | |
| openstackgerrit | Gabriele Cerami proposed openstack/tripleo-quickstart-extras master: Adding test to the skiplist for OVN related updates https://review.openstack.org/639386 | 17:22 |
| openstackgerrit | Nicolas PLANEL proposed openstack/tripleo-heat-templates master: Fix skydive_vars evaluation https://review.openstack.org/639394 | 17:25 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Use the default ContainerImagePrepare if not provided https://review.openstack.org/638286 | 17:27 |
| panda|ruck | fultonj: question about https://review.openstack.org/639351, we are effectively switching the default to bluestore here, do we need to keep a test for the old method in stable branches ? | 17:31 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: Remove environments/baremetal-services.yaml https://review.openstack.org/627632 | 17:33 |
| fultonj | panda|ruck: ceph-ansible 3.2 defaults to bluestore. | 17:33 |
| fultonj | panda|ruck: if we could switch on version, that would be nice to keep testing the older filestore | 17:33 |
| fultonj | but we can't switch on version | 17:34 |
| fultonj | "do we need to keep a test for the old method", no. it would be nice, so if we get the ability to switch on version then we could in a follow up. | 17:34 |
| fultonj | panda|ruck: ^ | 17:34 |
| panda|ruck | fultonj: ok so you're just updating our method to the current ceph default | 17:34 |
| fultonj | panda|ruck: yes | 17:34 |
| panda|ruck | fultonj: ok thanks. going to +W the patches then | 17:35 |
| fultonj | panda|ruck: did they come out green yet? | 17:35 |
| fultonj | i expect, they will but it's nice to be sure first | 17:35 |
| *** cfontain has joined #tripleo | 17:35 | |
| panda|ruck | fultonj: yes sorry I meant +2 | 17:35 |
| fultonj | panda|ruck: ack | 17:35 |
| *** mjturek has joined #tripleo | 17:36 | |
| panda|ruck | fultonj: also it's on THT which I'm not allowed to +W :P | 17:37 |
| *** avivgt has quit IRC | 17:37 | |
| openstackgerrit | Sergii Golovatiuk proposed openstack/python-tripleoclient stable/queens: Use a single queue for update/upgrade https://review.openstack.org/634190 | 17:39 |
| *** paramite|clone has joined #tripleo | 17:40 | |
| panda|ruck | fultonj: mmh looks like it failed for queens at least http://logs.openstack.org/52/639352/2/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/6b0ab2d/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 17:41 |
| * fultonj looks | 17:41 | |
| panda|ruck | and rocky also ... | 17:42 |
| openstackgerrit | Lukas Bezdicka proposed openstack/tripleo-heat-templates master: [FFU] Ensure compatibility with ansible 2.6. https://review.openstack.org/639169 | 17:42 |
| fultonj | http://logs.openstack.org/51/639351/2/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/d5d0dee/logs/undercloud/var/lib/mistral/overcloud/ceph-ansible/ceph_ansible_command.log.txt.gz#_2019-02-26_16_57_44_446 | 17:47 |
| *** holser__ has quit IRC | 17:48 | |
| fultonj | http://logs.openstack.org/52/639352/2/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/6b0ab2d/logs/undercloud/var/log/mistral/ceph-install-workflow.log.txt.gz#_2019-02-26_16_56_37_250 | 17:48 |
| fultonj | panda|ruck: ^ it's as if the LVs were not created. /me looking | 17:49 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/ansible-role-container-registry master: restrict docker xfs ftype check to /var/lib/docker https://review.openstack.org/639400 | 17:50 |
| openstackgerrit | Merged openstack/python-tripleoclient master: Remove redundancy in _launch_ansible* methods. https://review.openstack.org/636920 | 17:50 |
| larsks | mwhahaha: https://review.openstack.org/#/c/639400/ restricts the fstype check. | 17:53 |
| panda|ruck | fultonj: http://logs.openstack.org/52/639352/2/check/tripleo-ci-centos-7-scenario001-multinode-oooq-container/6b0ab2d/job-output.txt.gz#_2019-02-26_15_02_29_330577 | 17:53 |
| panda|ruck | fultonj: creation was skipped | 17:53 |
| weshay | panda|ruck so for https://bugs.launchpad.net/tripleo/+bug/1817705 I think we can only skip when ovn is in use ya? | 17:53 |
| openstack | Launchpad bug 1817705 in tripleo "periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-master fails tempest test related to DHCP agent" [Critical,Triaged] - Assigned to Gabriele Cerami (gcerami) | 17:53 |
| openstackgerrit | Lars Kellogg-Stedman proposed openstack/ansible-role-container-registry master: restrict docker xfs ftype check to /var/lib/docker https://review.openstack.org/639400 | 17:53 |
| mwhahaha | larsks: why do you have ftype not enabled somewhere? | 17:53 |
| larsks | mwhahaha: I am working with a partner who has root filesystems with ftype=0 which is constantly tripping things up. | 17:54 |
| panda|ruck | weshay: yes, means everywhere in master now, except for scenario007 | 17:54 |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-common master: Skip ssh_known_hosts tasks in check mode https://review.openstack.org/639402 | 17:54 |
| larsks | Just checking /var/lib/docker seems better than telling someone to reformat all their servers. | 17:54 |
| mwhahaha | larsks: we added that logic for some reason | 17:54 |
| mwhahaha | basically if we ever move the container bits (which we will) | 17:54 |
| larsks | Yes, it's because docker requires ftype=1 and won't run without it. | 17:54 |
| larsks | ...but the existing check causes everything to blow in cases where the system is actually configured appropriately. | 17:55 |
| larsks | It will trigger on *any* filesystem that happens to have ftype=0. | 17:55 |
| mwhahaha | larsks: why are they running ftype=0 at all | 17:55 |
| larsks | The change adds a variable (container_registry_docker_fs) which is the directory path we care about. | 17:55 |
| weshay | panda++ | 17:55 |
| larsks | mwhahaha: I cannot answer that question. | 17:55 |
| *** bfournie has joined #tripleo | 17:55 | |
| mwhahaha | larsks: well you should answer it, because unless this is an upgrade they shouldn't be running it | 17:56 |
| mwhahaha | they can't do anything contrainers that on those disks | 17:56 |
| mwhahaha | er containers | 17:56 |
| larsks | The *root* filesystem as ftype=0, but /var/lib/docker does not. | 17:56 |
| larsks | Everything works *fine* on these systems. | 17:56 |
| mwhahaha | well that's dangerous because it'll need to be /var/lib/containers in a future release | 17:56 |
| larsks | ...in which case we would just change the continer_registry_docker_fs variable. | 17:56 |
| mwhahaha | the point is that it's going to cause them problems and i'd rather not open that up unless there' sa techincal reason | 17:56 |
| *** trown is now known as trown|lunch | 17:57 | |
| larsks | This just seems *really* broad. What if you have an old disk mounted with critical data that you can't reformat to be ftype=1? It has nothing to do with docker -- maybe it's /srv/somethingorother -- but it will cause the deploy to bail. | 17:57 |
| mwhahaha | that would be the upgrade problem | 17:58 |
| larsks | I wouldn't mind if it were possible to upgrade an xfs filesyste, but I don't think it is. | 17:58 |
| mwhahaha | there isn't which is why it's important not to install on a system that isn't ftype=1 | 17:58 |
| mwhahaha | so no, this is critical for the container stuff and we cannot be certain we won't need other paths | 17:58 |
| larsks | I honestly don't see the problem here: with this change, we're still checking for an appropriate filesystem configuration. We're just targeting the filesystem on which we're installing containers, rather than looking at *all* mounted filesystems. | 17:59 |
| larsks | If the container location changes, we can update the check to match. | 17:59 |
| mwhahaha | but it's too late for customers who already installed | 18:00 |
| mwhahaha | that's the problem | 18:00 |
| mwhahaha | updating the path solves nothing because there is no fix | 18:00 |
| mwhahaha | this is a UX thing to prevent folks from being screwed in teh future | 18:00 |
| larsks | I'm not following you there. This check will cause the deploy to fail in situtations in which it should fail. If someone has already installed...I don't get it; either their install is already broken or I am missing something? | 18:00 |
| mwhahaha | i'm sorry but you should ask the partner why ftype=0, it's been =1 since 7.3 | 18:00 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: container-image-prepare: redirect all output to logfile https://review.openstack.org/639405 | 18:01 |
| mwhahaha | larsks: if i create a /var/lib/docker that's ftype=1 and everything is ftype=0, upgrades to podman will fail in stein | 18:01 |
| mwhahaha | and user cannot fix their environment | 18:01 |
| larsks | Are we only running this check on initial install and not on upgrade? | 18:01 |
| larsks | Or wouldn't we catch it during the upgrade (if we were to check the new container location)? | 18:01 |
| mwhahaha | we run it all the time, it's part of the container setup | 18:01 |
| larsks | Okay, so the upgrade to podman should be fine, right? Because we'd check the new location and it would either have ftype=1 or not, and in that case it would fail. | 18:02 |
| *** wolverineav has quit IRC | 18:02 | |
| mwhahaha | but there's no recovery for folks who don't have more disks | 18:02 |
| *** wolverineav has joined #tripleo | 18:03 | |
| mwhahaha | sorry but i don't think we should be adjusting this to be less restrictive | 18:03 |
| larsks | I apologize, I may have misunderstood your initial response. I asked if this check could be limited to /var/lib/docker and you said "yes", and I took that as an indication that you would be okay with the change. I'll drop it. | 18:03 |
| *** wolverineav has quit IRC | 18:03 | |
| mwhahaha | we went thought many iterations of this check, can you get away with it in your case? sure. should we be lifting the restrictions for everyone? no. | 18:05 |
| mwhahaha | larsks: for the sake of the partner it would be better to find out if there's an actual reason for their disk format (performance/security/etc) and then we could investigate more. but an end user can get in a tricky/unrecoverable scenario | 18:06 |
| *** thrash is now known as thrash|biab | 18:06 | |
| mwhahaha | since you can't upgrade xfs, it's really not a good idea to leave it on ftype=0 for new systems | 18:07 |
| *** cfontain has quit IRC | 18:09 | |
| *** florianf_ has quit IRC | 18:12 | |
| *** itlinux_ has quit IRC | 18:13 | |
| *** kopecmartin is now known as kopecmartin|off | 18:14 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: add image sharing instructions https://review.openstack.org/639407 | 18:14 |
| hamzy | I am running RHOSP14. If docker ps says nova_libvirt is up and healthy, then systemd doesn't control libvirtd right? What is necessary to restart libvirt? I did "for i in nova_compute nova_libvirt; do sudo docker restart $i; done" on the compute node and "for i in nova_conductor nova_compute nova_api nova_scheduler; do sudo docker restart $i; done" on the controller node. | 18:15 |
| fultonj | panda|ruck: thanks for explaining "branchful jobs" it's a good horror story. i'll tell my kids at night | 18:15 |
| *** jpena is now known as jpena|off | 18:15 | |
| *** chandankumar is now known as raukadah | 18:16 | |
| *** chem` has quit IRC | 18:16 | |
| fultonj | hamzy: it's better to use paunch to start and stop the containers | 18:16 |
| mwhahaha | waht no | 18:17 |
| mwhahaha | docker restart should be sufficient | 18:17 |
| fultonj | https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/tips_tricks.html#debugging-with-paunch | 18:17 |
| *** itlinux has joined #tripleo | 18:17 | |
| mwhahaha | that's for running containers that don't exist | 18:17 |
| mwhahaha | once they exist, docker restart should be fine | 18:17 |
| fultonj | that's true | 18:18 |
| hamzy | hrm... I am seeing "error : virNetSocketNewConnectUNIX:712 : Failed to connect socket to '/var/run/libvirt/virtlogd-sock': Connection refused" in the syslog | 18:18 |
| fultonj | he wanted to modify the container and see a change to it | 18:18 |
| fultonj | _I assumed_ he wanted to modify the container and see a change to it | 18:18 |
| hamzy | well restart really | 18:18 |
| fultonj | sorry | 18:18 |
| *** wolverineav has joined #tripleo | 18:18 | |
| *** rbrady has quit IRC | 18:18 | |
| hamzy | I see libvirt complaining "error: kvm run failed Device or resource busy. This is probably because your SMT is enabled." so I disabled SMT and am trying to restart libvirt | 18:19 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: Removing queens and pike from the negative match for ceph-loop role https://review.openstack.org/639408 | 18:19 |
| panda|ruck | fultonj: ^ | 18:20 |
| fultonj | panda|ruck: ack | 18:20 |
| mwhahaha | hamzy: owalsh / mschuppert may be able to help you with libvirt problems | 18:20 |
| hamzy | thanks! | 18:21 |
| *** wolverineav has quit IRC | 18:22 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates stable/rocky: Switch scenario00{1,4}-standalone to Ceph bluestore https://review.openstack.org/639351 | 18:22 |
| *** wolverineav has joined #tripleo | 18:22 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates stable/queens: Switch scenario00{1,4}-standalone to Ceph bluestore https://review.openstack.org/639352 | 18:22 |
| fultonj | panda|ruck: ^ | 18:22 |
| *** wolverineav has quit IRC | 18:23 | |
| openstackgerrit | Nagasai Vinaykumar Kapalavai proposed openstack/puppet-tripleo master: metrics_qdr: TLS support. https://review.openstack.org/631297 | 18:23 |
| openstackgerrit | Nagasai Vinaykumar Kapalavai proposed openstack/tripleo-heat-templates master: metrics_qdr: Add TLS support https://review.openstack.org/630736 | 18:23 |
| hamzy | ooo... mwhahaha owalsh mschuppert, I saw something about virtlogd being a requirement so it looks like "for i in nova_compute nova_virtlogd nova_libvirt; do sudo docker restart $i; done" worked! | 18:24 |
| *** derekh has quit IRC | 18:25 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates stable/queens: Switch scenario00{1,4}-multinode-containers to Ceph bluestore https://review.openstack.org/639352 | 18:26 |
| *** wolverineav has joined #tripleo | 18:26 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-heat-templates stable/rocky: Switch scenario00{1,4}-multinode-containers to Ceph bluestore https://review.openstack.org/639351 | 18:26 |
| panda|ruck | fultonj: thanks! | 18:28 |
| fultonj | thank you panda|ruck | 18:28 |
| *** panda|ruck is now known as panda|ruck|off | 18:28 | |
| *** sdake_ has quit IRC | 18:30 | |
| *** sdake has joined #tripleo | 18:32 | |
| openstackgerrit | Nagasai Vinaykumar Kapalavai proposed openstack/puppet-tripleo master: metrics_qdr: TLS support. https://review.openstack.org/631297 | 18:32 |
| *** sdake has quit IRC | 18:35 | |
| *** sdake has joined #tripleo | 18:37 | |
| openstackgerrit | Nagasai Vinaykumar Kapalavai proposed openstack/tripleo-heat-templates master: metrics_qdr: Add TLS support https://review.openstack.org/630736 | 18:39 |
| openstackgerrit | Merged openstack/puppet-tripleo master: Support cephfs_volume_mode parameter https://review.openstack.org/638773 | 18:43 |
| openstackgerrit | Merged openstack/tripleo-puppet-elements master: Release note for chrony/ntp switch https://review.openstack.org/639163 | 18:43 |
| *** sdake has quit IRC | 18:46 | |
| *** rfolco|rover has quit IRC | 18:46 | |
| *** dprince has quit IRC | 18:48 | |
| *** _fragatina has quit IRC | 18:50 | |
| *** sdake has joined #tripleo | 18:51 | |
| *** thrash|biab is now known as thrash | 18:52 | |
| *** sdake has quit IRC | 18:55 | |
| *** sdake_ has joined #tripleo | 18:56 | |
| *** sdake_ has quit IRC | 18:56 | |
| openstackgerrit | Marc Methot proposed openstack/tripleo-heat-templates master: Adding configurability to the amphora image format https://review.openstack.org/639416 | 18:57 |
| *** sdake has joined #tripleo | 18:58 | |
| *** sanjayu_ has quit IRC | 18:59 | |
| openstackgerrit | Tom Barron proposed openstack/puppet-tripleo stable/rocky: Support cephfs_volume_mode parameter https://review.openstack.org/639417 | 18:59 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: Script to identify tests passing on FS021 but still on skip list https://review.openstack.org/637524 | 19:00 |
| *** sdake has quit IRC | 19:00 | |
| *** sdake_ has joined #tripleo | 19:02 | |
| *** wolverineav has quit IRC | 19:03 | |
| openstackgerrit | Tom Barron proposed openstack/tripleo-heat-templates master: Support cephfs_volume_mode parameter https://review.openstack.org/638775 | 19:03 |
| *** wolverineav has joined #tripleo | 19:04 | |
| *** sdake_ has quit IRC | 19:06 | |
| *** sdake has joined #tripleo | 19:06 | |
| *** ramishra_ has quit IRC | 19:09 | |
| *** sdake has quit IRC | 19:10 | |
| Vorrtex | Hey all, I'm trying to get tox running smoothly for tripleo-common on my mac, and I'm not sure how to approach this error: "ERROR: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError("Failed to get version_info for python3.6: pyenv: python3.6: command not found\n\nThe `python3.6' command exists in these Python | 19:11 |
| Vorrtex | versions:\n 3.6.3\n\n", None)" | 19:11 |
| *** rfolco has joined #tripleo | 19:12 | |
| *** sdake has joined #tripleo | 19:12 | |
| Vorrtex | py37 works fine, my local version of python is installed via pyenv and is version 3.5.5, and I'm finding that a lot of posts are talking about pyenv having local version clashes, but I can't seem to find where exactly that is happening. Any ideas? | 19:13 |
| mwhahaha | sounds like an error in tox | 19:14 |
| mwhahaha | do you have some weird path that you're keeping tripleo-common in? | 19:14 |
| Vorrtex | I place it here: ~/workspace/openstack/tripleo-common | 19:14 |
| Vorrtex | and run tox there as well. | 19:15 |
| Vorrtex | I've done tox -r -e py36 (fails), and if I do just a tox -r, it completes everything successfully EXCEPT py36 | 19:15 |
| mwhahaha | no idea | 19:15 |
| Vorrtex | I suppose I could try again with a fresh clone of master or something, but I haven't changed anything on my master branch, outside of pulling latest this morning | 19:16 |
| mwhahaha | it doesn't sound like it's getting to the code itself, but rather than environment setup | 19:16 |
| mwhahaha | do you have python3.6 installed? | 19:16 |
| mwhahaha | that error seems to point that you're pyenv isn't currently set to 3. | 19:17 |
| mwhahaha | 3.6 | 19:17 |
| Vorrtex | My pyenv is locally set to 3.5. Not sure why 3.7 would pass if that was an issue. Also, I thought tox installed its own python virtual environments for whatever version was needed to run? Bad assumption on my part? | 19:18 |
| *** trown|lunch is now known as trown | 19:18 | |
| mwhahaha | is 3.7 the system python | 19:18 |
| mwhahaha | I bet if you pyenv local 3.6.3 in tripleo-common and then tox -e py36 it'd work | 19:18 |
| Vorrtex | 3.6.3 is the global | 19:18 |
| mwhahaha | right but the local wins | 19:19 |
| Vorrtex | Okay, but that doesn't match up then, why does 3.7 work if the local would win? | 19:19 |
| mwhahaha | magic? no idea what your system setup is | 19:19 |
| *** itlinux_ has joined #tripleo | 19:19 | |
| Vorrtex | ha ha, sorry, I don't mean to attack, just looking for understanding | 19:20 |
| *** sdake has quit IRC | 19:20 | |
| *** gouthamr has quit IRC | 19:20 | |
| *** gouthamr_ has joined #tripleo | 19:20 | |
| *** itlinux has quit IRC | 19:21 | |
| *** sdake_ has joined #tripleo | 19:22 | |
| Vorrtex | I'm removing my global definition to see if anything changes... Maybe it was clashing with that one. | 19:22 |
| *** sdake_ has quit IRC | 19:25 | |
| Vorrtex | Doesn't seem to have changed anything... I'll try cleaning up some other virtual environments. | 19:28 |
| *** sdake has joined #tripleo | 19:28 | |
| *** rfolco is now known as rfolco|rover | 19:28 | |
| *** ykarel has quit IRC | 19:33 | |
| *** sdake has quit IRC | 19:35 | |
| *** sdake_ has joined #tripleo | 19:36 | |
| *** wolverineav has quit IRC | 19:42 | |
| weshay | cgoncalves can we get a wf on https://review.openstack.org/#/c/639083/ ? | 19:44 |
| *** sdake_ has quit IRC | 19:45 | |
| *** sdake has joined #tripleo | 19:47 | |
| *** ekultails has quit IRC | 19:47 | |
| *** hberaud is now known as hberaud|gone | 19:48 | |
| openstackgerrit | Nagasai Vinaykumar Kapalavai proposed openstack/tripleo-heat-templates master: metrics_qdr: Add TLS support https://review.openstack.org/639423 | 19:48 |
| *** dprince has joined #tripleo | 19:48 | |
| weshay | rlandy mwhahaha https://review.openstack.org/#/c/639341/ | 19:50 |
| weshay | saves about 25mb of space | 19:50 |
| mwhahaha | too bad atop is actually useful | 19:51 |
| mwhahaha | but sure | 19:51 |
| weshay | mwhahaha we're going to keep in containers-multinode and ovb on rdo | 19:51 |
| weshay | so if you hit perf issues just check the 3rd party jobs | 19:51 |
| *** itlinux_ has quit IRC | 19:52 | |
| *** sdake has quit IRC | 19:55 | |
| *** itlinux has joined #tripleo | 19:56 | |
| *** wolverineav has joined #tripleo | 19:56 | |
| *** sdake has joined #tripleo | 19:57 | |
| *** derekh has joined #tripleo | 19:58 | |
| *** wolverineav has quit IRC | 20:01 | |
| *** sdake has quit IRC | 20:03 | |
| Vorrtex | mwhahaha so now I blew away my virtual environment with 3.6.3 (and any venvs stemming from it), and ended up with the following error: "ERROR: py36: InterpreterNotFound: python3.6" | 20:03 |
| Vorrtex | So now I'm really at a loss... its like tox doesn't know what python3.6 is. | 20:03 |
| *** wolverineav has joined #tripleo | 20:04 | |
| *** ekultails has joined #tripleo | 20:06 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Modify roles to remove unused services https://review.openstack.org/623012 | 20:07 |
| *** wolverineav has quit IRC | 20:09 | |
| *** derekh has quit IRC | 20:09 | |
| *** itlinux has quit IRC | 20:16 | |
| *** itlinux has joined #tripleo | 20:20 | |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: clean up the doc footer in tripleo jobs https://review.openstack.org/639430 | 20:26 |
| *** dciabrin has quit IRC | 20:27 | |
| *** dciabrin has joined #tripleo | 20:28 | |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart-extras master: Adding test to the skiplist for OVN related updates https://review.openstack.org/639386 | 20:31 |
| openstackgerrit | Emilien Macchi proposed openstack/tripleo-heat-templates master: upgrade2podman: prune everything in Docker https://review.openstack.org/620405 | 20:32 |
| *** wolverineav has joined #tripleo | 20:37 | |
| *** gouthamr_ is now known as gouthamr | 20:42 | |
| *** wolverineav has quit IRC | 20:42 | |
| mwhahaha | weshay: did we ever log about for the image building failing on OVB jobs? https://logs.rdoproject.org/86/638286/4/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/b59490e/logs/undercloud/home/zuul/overcloud_image_build.log.txt.gz#_2019-02-26_18_42_28 | 20:43 |
| mwhahaha | seems to be happening quite frequently as of late | 20:44 |
| mwhahaha | i think it's on the ironic-python-agent | 20:44 |
| mwhahaha | wonder if there's a real bug tehre | 20:44 |
| weshay | so.. rdo and especially fs001 where we see that was all messed up yesterday.. jobs were running at like 20% success rate | 20:44 |
| weshay | I've added the error signature to sova to help figure any patterns this morning.. | 20:45 |
| weshay | I will continue to debug it w/ rfolco|rover and panda|ruck|off | 20:45 |
| mwhahaha | k | 20:45 |
| weshay | mwhahaha we have it in alerts.. but alerts are not working in the channel atm | 20:46 |
| openstackgerrit | Harry Rybacki proposed openstack/tripleo-heat-templates stable/queens: Move ipa enrollment to host_prep_tasks https://review.openstack.org/638523 | 20:56 |
| *** itlinux has quit IRC | 20:59 | |
| openstackgerrit | Lee Yarwood proposed openstack/tripleo-heat-templates master: WIP nova: Add nova-status upgrade check call during deployment https://review.openstack.org/635139 | 21:02 |
| *** spsurya has quit IRC | 21:03 | |
| openstackgerrit | Steve Baker proposed openstack/tripleo-common master: Expunge the docker uploader https://review.openstack.org/639438 | 21:07 |
| *** itlinux has joined #tripleo | 21:07 | |
| *** cfontain has joined #tripleo | 21:08 | |
| *** hjensas has quit IRC | 21:19 | |
| openstackgerrit | Nagasai Vinaykumar Kapalavai proposed openstack/puppet-tripleo master: metrics_qdr: TLS support. https://review.openstack.org/631297 | 21:20 |
| *** wolverineav has joined #tripleo | 21:21 | |
| *** wolverineav has quit IRC | 21:21 | |
| openstackgerrit | Nagasai Vinaykumar Kapalavai proposed openstack/tripleo-heat-templates master: metrics_qdr: Add TLS support https://review.openstack.org/639423 | 21:21 |
| *** wolverineav has joined #tripleo | 21:21 | |
| *** hjensas has joined #tripleo | 21:22 | |
| *** dxiri has quit IRC | 21:23 | |
| *** derekh has joined #tripleo | 21:24 | |
| *** derekh has quit IRC | 21:24 | |
| *** zbr|out has quit IRC | 21:27 | |
| *** ansmith has quit IRC | 21:31 | |
| *** raildo has quit IRC | 21:33 | |
| *** jcoufal has quit IRC | 21:37 | |
| fultonj | weshay: the patches to fix 001 and 004 are now green in CI https://review.openstack.org/#/c/639351 https://review.openstack.org/#/c/639352 | 21:38 |
| weshay | fultonj nice | 21:38 |
| fultonj | i think we just need some reviews | 21:40 |
| *** cfontain has quit IRC | 21:41 | |
| weshay | aye | 21:41 |
| weshay | mwhahaha might hook you up ^ :) | 21:43 |
| *** jtomasek has quit IRC | 21:43 | |
| mwhahaha | changing stable ci :/ | 21:44 |
| mwhahaha | such is life i guess | 21:44 |
| fultonj | thank you both | 21:45 |
| *** wolverineav has quit IRC | 21:46 | |
| openstackgerrit | Merged openstack/python-tripleoclient stable/rocky: Check proxy env vars https://review.openstack.org/638710 | 21:46 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Make heat yaql limits configurable https://review.openstack.org/639450 | 21:47 |
| *** wolverineav has joined #tripleo | 21:48 | |
| *** wolverineav has quit IRC | 21:48 | |
| *** wolverineav has joined #tripleo | 21:48 | |
| fultonj | weshay: do you want to review (the depends on of those)? https://review.openstack.org/#/c/639408/ | 21:48 |
| weshay | I'm a +2 whore | 21:49 |
| openstackgerrit | Merged openstack/python-tripleoclient master: container image build: introduce --use-buildah https://review.openstack.org/634322 | 21:50 |
| weshay | oh snap ^ | 21:50 |
| fultonj | thanks weshay | 21:51 |
| weshay | fultonj I may lean on you a bit on where to put some extra coverage upstream | 21:52 |
| *** paramite|clone has quit IRC | 21:52 | |
| fultonj | weshay: ok | 21:52 |
| *** itlinux has quit IRC | 21:53 | |
| *** mjturek has quit IRC | 21:54 | |
| openstackgerrit | David Peacock proposed openstack/tripleo-heat-templates master: flatten database service Redis https://review.openstack.org/635930 | 21:55 |
| *** dprince has quit IRC | 21:59 | |
| *** cfontain has joined #tripleo | 22:02 | |
| *** itlinux has joined #tripleo | 22:04 | |
| weshay | mwhahaha that image build is working for me locally atm.. I guess I may have to rerun, rerun | 22:05 |
| weshay | just the image build | 22:05 |
| *** rcernin has joined #tripleo | 22:05 | |
| *** abishop has quit IRC | 22:05 | |
| weshay | it's never worked on your patch I see 638286 | 22:06 |
| mwhahaha | which is weird cause that's a patch for containers not overcloud images | 22:08 |
| weshay | ya.. totally.. rerunning.. few min | 22:08 |
| *** xek has quit IRC | 22:08 | |
| *** wolverineav has quit IRC | 22:08 | |
| *** itlinux has quit IRC | 22:08 | |
| weshay | mwhahaha got a hit on another box | 22:10 |
| *** wolverineav has joined #tripleo | 22:10 | |
| weshay | recreated.. can tmate if you want | 22:10 |
| mwhahaha | seems like a race condition or something | 22:10 |
| weshay | mwhahaha I suppose I can just run the disk-image-create command | 22:12 |
| weshay | this is from some tmp dir? '-o', './ironic-python-agent' | 22:12 |
| mwhahaha | likely | 22:13 |
| mwhahaha | you could use the image build command to rerun | 22:13 |
| * mwhahaha digs up docs | 22:13 | |
| mwhahaha | https://docs.openstack.org/tripleo-docs/latest/install/basic_deployment/basic_deployment_cli.html#get-images | 22:14 |
| weshay | disk-image-create -a amd64 -o ./ironic-python-agent -t qcow2 -p yum-plugin-prioritiespython-hardware-detect --min-tmpfs=7 centos7 selinux-permissive ironic-agent ironic-agent-multipath dynamic-login element-manifest network-gateway enable-packages-install override-pip-and-virtualenv | 22:14 |
| * weshay reads | 22:14 | |
| *** wolverineav has quit IRC | 22:14 | |
| mwhahaha | yea you could try the disk-image-create | 22:15 |
| *** Vorrtex has quit IRC | 22:15 | |
| mwhahaha | wonder which element it's failing tho | 22:16 |
| mwhahaha | seems to be the ironic-agent one | 22:16 |
| *** itlinux has joined #tripleo | 22:17 | |
| *** wolverineav has joined #tripleo | 22:17 | |
| weshay | ya.. -o is output.. I need the right dib path | 22:17 |
| *** slaweq has quit IRC | 22:18 | |
| weshay | that has to be in the log https://logs.rdoproject.org/86/638286/4/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/b59490e/logs/undercloud/home/zuul/overcloud_image_build.log.txt.gz#_2019-02-26_18_42_32 | 22:18 |
| mwhahaha | it seems like it's rdocloud | 22:20 |
| mwhahaha | cause i don't think i've ever seen that error for the buildimage job | 22:21 |
| mwhahaha | https://github.com/openstack-infra/tripleo-ci/blob/master/playbooks/tripleo-buildimage/run-v3.yaml#L40-L61 | 22:21 |
| mwhahaha | elemens are tripleo-common/t-p-e/t-i-e/heat-agents | 22:21 |
| * weshay is missing shit http://pastebin.test.redhat.com/724103 | 22:22 | |
| * weshay reads | 22:22 | |
| weshay | mwhahaha it's not rdo cloud.. I'm on a libvirt node | 22:22 |
| mwhahaha | o | 22:22 |
| mwhahaha | ok then | 22:22 |
| mwhahaha | techincally it's openstack overcloud image build | 22:22 |
| mwhahaha | we might do some stuff under the covers in tripleoclient | 22:23 |
| weshay | sure.. that's what I started w/ | 22:23 |
| * weshay trying to get a real trace | 22:23 | |
| *** bfournie has quit IRC | 22:23 | |
| weshay | or log or something | 22:23 |
| weshay | been awhile since I messed w/ the image builds | 22:23 |
| mwhahaha | pfffft useful logging | 22:23 |
| mwhahaha | who does that | 22:23 |
| weshay | lolz | 22:23 |
| * weshay tries to set the ELEMENT path | 22:23 | |
| *** itlinux has quit IRC | 22:24 | |
| *** itlinux has joined #tripleo | 22:26 | |
| *** slaweq has joined #tripleo | 22:26 | |
| mwhahaha | i thought there was a patch to DIB to not erase git | 22:28 |
| mwhahaha | from sorin | 22:28 |
| mwhahaha | https://logs.rdoproject.org/86/638286/4/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/b59490e/logs/undercloud/home/zuul/overcloud_image_build.log.txt.gz#_2019-02-26_18_42_24 | 22:28 |
| mwhahaha | wonder if we need promotion | 22:28 |
| *** sdake has joined #tripleo | 22:29 | |
| weshay | there was | 22:29 |
| weshay | I remember that patch | 22:29 |
| mwhahaha | cause that's what it's failing on | 22:29 |
| mwhahaha | was from quiquell|off | 22:29 |
| mwhahaha | https://review.openstack.org/#/c/637162/. | 22:29 |
| weshay | missing heat-agents path atm | 22:29 |
| mwhahaha | its i think heat-agents is /usr/share/openstack-heat-agents or something | 22:29 |
| weshay | hrm.. it's not on my box atm | 22:30 |
| * weshay pulls it | 22:30 | |
| mwhahaha | which would be another reason why it's only ovb | 22:30 |
| mwhahaha | cause the other job uses source | 22:30 |
| *** slaweq has quit IRC | 22:30 | |
| weshay | https://github.com/openstack/heat-agents | 22:30 |
| *** sdake has quit IRC | 22:30 | |
| *** itlinux has quit IRC | 22:31 | |
| *** sdake has joined #tripleo | 22:32 | |
| weshay | k... it's running | 22:32 |
| mwhahaha | it's failing somewhere around https://github.com/openstack/diskimage-builder/blob/e796b3bc1884cbb0a7259be486d835ca114cca9e/diskimage_builder/elements/yum/bin/install-packages#L145 | 22:32 |
| mwhahaha | which seems weird cause it's the end | 22:33 |
| mwhahaha | so something else is failing | 22:33 |
| weshay | fak.. http://paste.openstack.org/show/746371/ | 22:35 |
| weshay | worked | 22:35 |
| * weshay checks image | 22:35 | |
| mwhahaha | remove the image and try again | 22:35 |
| mwhahaha | maybe? | 22:35 |
| weshay | running again.. because I don't see the qcow | 22:37 |
| weshay | lolz | 22:37 |
| weshay | wtf | 22:37 |
| *** cfontain has quit IRC | 22:37 | |
| weshay | it HAS ONE JOB | 22:37 |
| mwhahaha | :D | 22:37 |
| weshay | wtf | 22:38 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-heat-templates master: Make heat yaql limits configurable https://review.openstack.org/639450 | 22:39 |
| mwhahaha | dat race condition | 22:39 |
| weshay | is that a clean exit? | 22:39 |
| mwhahaha | yea | 22:39 |
| weshay | no image created | 22:39 |
| mwhahaha | oh no it's a fail | 22:40 |
| *** _fragatina has joined #tripleo | 22:40 | |
| mwhahaha | yum-plugin-prioritiespython-hardware-detect | 22:40 |
| mwhahaha | missing a space | 22:40 |
| mwhahaha | somewhere | 22:40 |
| mwhahaha | in your cmdline | 22:40 |
| weshay | ah see that | 22:40 |
| weshay | thanks! | 22:40 |
| *** sdake has quit IRC | 22:40 | |
| *** sdake_ has joined #tripleo | 22:41 | |
| * weshay missing element python-hardware-detect | 22:41 | |
| * weshay looks | 22:41 | |
| weshay | http://git.openstack.org/cgit/openstack/tripleo-common/tree/image-yaml/overcloud-images-python3.yaml#n56 | 22:42 |
| weshay | that package is not avail to me | 22:43 |
| *** cfontain has joined #tripleo | 22:43 | |
| mwhahaha | don't build python3? | 22:43 |
| weshay | ya.. just codesearching | 22:43 |
| mwhahaha | http://git.openstack.org/cgit/openstack/tripleo-common/tree/image-yaml/overcloud-images.yaml | 22:43 |
| mwhahaha | also includes it, weird | 22:43 |
| mwhahaha | anyway i'm kikcing off a full build i guess | 22:43 |
| weshay | k | 22:43 |
| weshay | mwhahaha where is the element for that though? | 22:44 |
| mwhahaha | actually i'll just cut it down to the ironic-python-agent | 22:44 |
| mwhahaha | that's not an element, it's a package | 22:44 |
| *** sdake_ has quit IRC | 22:45 | |
| weshay | ok.. trying w/o | 22:46 |
| mwhahaha | http://paste.openstack.org/show/746372/ | 22:47 |
| *** sdake has joined #tripleo | 22:47 | |
| mwhahaha | is how to use the regular build process and only build the ipa | 22:47 |
| * weshay is an idiot | 22:47 | |
| weshay | that's the easy way :) | 22:48 |
| mwhahaha | set -ex; while true; do openstack overcloud image build --config ipa.yaml; done | 22:48 |
| mwhahaha | :D | 22:48 |
| mwhahaha | actually i think you'd need to remove the output but anyway | 22:49 |
| weshay | weird.. it throws this.. | 22:49 |
| weshay | diskimage_builder.element_dependencies.MissingElementException: Element 'yum-plugin-priorities' not found | 22:49 |
| weshay | lolz | 22:49 |
| *** cfontain has quit IRC | 22:49 | |
| *** itlinux has joined #tripleo | 22:50 | |
| weshay | I need a comman, not a space or something | 22:50 |
| mwhahaha | oh it's a comma | 22:50 |
| mwhahaha | yea i just looked in my output | 22:50 |
| mwhahaha | failed first try fo rme | 22:51 |
| mwhahaha | lovely | 22:51 |
| weshay | decent trace or no? | 22:51 |
| mwhahaha | of course not | 22:51 |
| weshay | k.. /me trying again | 22:52 |
| weshay | return code 1, no image no error | 22:53 |
| * weshay checks the command for verbose | 22:53 | |
| mwhahaha | Operation failed: No such file or directory | 22:53 |
| mwhahaha | my least favorite error | 22:53 |
| mwhahaha | WHAT FILE | 22:53 |
| weshay | srsly | 22:54 |
| weshay | no debug no verbose? | 22:54 |
| mwhahaha | not for that command | 22:54 |
| mwhahaha | i think dib has one | 22:54 |
| weshay | https://docs.openstack.org/diskimage-builder/latest/ | 22:54 |
| weshay | I'm looking the help and doc | 22:55 |
| *** sdake has quit IRC | 22:56 | |
| mwhahaha | i wonder if it's the rm -rf /tmp/ironic-python-agent | 22:56 |
| * mwhahaha tries || true | 22:56 | |
| * weshay has to go | 22:56 | |
| weshay | bbl | 22:56 |
| mwhahaha | DIB_DEBUG_TRACE | 22:56 |
| mwhahaha | btw | 22:56 |
| mwhahaha | export DIB_DEBUG_TRACE=1 | 22:56 |
| *** itlinux has quit IRC | 22:57 | |
| *** sdake has joined #tripleo | 22:58 | |
| *** agopi has quit IRC | 22:59 | |
| mwhahaha | systemctl enable openstack-ironic-python-agent.service | 23:00 |
| *** Chaserjim has quit IRC | 23:00 | |
| mwhahaha | failing on that | 23:00 |
| mwhahaha | weird | 23:00 |
| *** sdake has quit IRC | 23:00 | |
| *** sdake has joined #tripleo | 23:02 | |
| *** dciabrin has quit IRC | 23:02 | |
| *** tkajinam has joined #tripleo | 23:02 | |
| *** dciabrin has joined #tripleo | 23:02 | |
| *** sdake has quit IRC | 23:06 | |
| *** sdake_ has joined #tripleo | 23:07 | |
| mwhahaha | ianw: you around? | 23:07 |
| mwhahaha | ianw: we're getting ipa failures on build looks like it's erroring enabling the serivce which is weird. not sure if you have any idea, see https://bugs.launchpad.net/tripleo/+bug/1817370 | 23:08 |
| openstack | Launchpad bug 1817370 in tripleo "disk-image-create fails in ovb jobs intermittently" [Critical,Triaged] | 23:08 |
| *** ansmith has joined #tripleo | 23:09 | |
| *** sdake_ has quit IRC | 23:10 | |
| *** sdake has joined #tripleo | 23:11 | |
| mwhahaha | k so it's failing somewhere around https://github.com/openstack/diskimage-builder/blob/master/diskimage_builder/elements/ironic-agent/post-install.d/80-ironic-agent#L26 | 23:12 |
| mwhahaha | 4 year old code | 23:14 |
| * mwhahaha sighs | 23:14 | |
| *** rascasoft has quit IRC | 23:15 | |
| *** dciabrin has quit IRC | 23:15 | |
| *** dciabrin_ has joined #tripleo | 23:15 | |
| *** sanjayu_ has joined #tripleo | 23:19 | |
| *** sdake has quit IRC | 23:20 | |
| *** gfidente|afk has quit IRC | 23:22 | |
| *** sdake has joined #tripleo | 23:23 | |
| *** sdake has quit IRC | 23:25 | |
| *** sdake has joined #tripleo | 23:26 | |
| *** ade_lee has quit IRC | 23:27 | |
| *** sdake has quit IRC | 23:30 | |
| *** sdake has joined #tripleo | 23:33 | |
| *** sdake has quit IRC | 23:35 | |
| mwhahaha | weshay: we need a promotion, it's the git removal | 23:36 |
| *** cfontain has joined #tripleo | 23:37 | |
| *** sdake has joined #tripleo | 23:37 | |
| mwhahaha | well that solves the one | 23:38 |
| mwhahaha | annoying | 23:38 |
| mwhahaha | and likely why we used to have diskimage-builder in the ci excludes so we always got the latest version | 23:38 |
| *** aedc has joined #tripleo | 23:39 | |
| *** mwhahaha changes topic to "CI Status: GREENish but OVB broken building images (need promotion) | https://docs.openstack.org/tripleo-docs/latest/" | 23:39 | |
| *** sdake has quit IRC | 23:40 | |
| *** sdake has joined #tripleo | 23:42 | |
| *** sdake has quit IRC | 23:50 | |
| *** sdake has joined #tripleo | 23:52 | |
| *** vinaykns has quit IRC | 23:54 | |
| *** sdake has quit IRC | 23:55 | |
| *** ade_lee has joined #tripleo | 23:56 | |
| *** sdake has joined #tripleo | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!