| dking_desktop | mwhahaha: Oh, thanks! It didn't seem to generate one for me the first time, and thought it wasn't there. | 00:01 |
|---|---|---|
| dking_desktop | It's a bit intense signing up to contribute. :) | 00:01 |
| *** ansmith_ has quit IRC | 00:08 | |
| *** ansmith_ has joined #tripleo | 00:09 | |
| openstackgerrit | Tom Barron proposed openstack/tripleo-heat-templates stable/stein: Fix Manila when run without pacemaker https://review.opendev.org/685097 | 00:11 |
| *** dsneddon has joined #tripleo | 00:11 | |
| *** morazi has quit IRC | 00:17 | |
| *** morazi has joined #tripleo | 00:23 | |
| *** morazi has quit IRC | 00:37 | |
| *** gbarros has joined #tripleo | 00:38 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: yum-minimal : update mirrors for Centos 8 https://review.opendev.org/684439 | 00:41 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: dib-python : handle centos 8 https://review.opendev.org/684441 | 00:41 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Update locales for Centos 8 https://review.opendev.org/684442 | 00:41 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Update redhat-common pkg-map for centos 8 https://review.opendev.org/684449 | 00:41 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add environment switch for centos8 to use dnf https://review.opendev.org/684459 | 00:41 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Use $YUM instead of direct calls in more places https://review.opendev.org/684460 | 00:41 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: yum-minimal: Don't install yum, install libcurl https://review.opendev.org/684461 | 00:41 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] pip-and-virtualenv : fix for centos 8 https://review.opendev.org/684462 | 00:41 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] don't create interfaces on centos-8 https://review.opendev.org/684513 | 00:41 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] simple-init : don't use network.service for centos8 https://review.opendev.org/684514 | 00:41 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] centos 8 minimal testing https://review.opendev.org/684860 | 00:41 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: WIP: CentOS 8 support https://review.opendev.org/684308 | 00:41 |
| *** morazi has joined #tripleo | 00:42 | |
| *** gbarros has quit IRC | 00:42 | |
| openstackgerrit | Tom Barron proposed openstack/tripleo-heat-templates master: DNM - test only https://review.opendev.org/682248 | 00:52 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: yum-minimal : update mirrors for Centos 8 https://review.opendev.org/684439 | 00:58 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: dib-python : handle centos 8 https://review.opendev.org/684441 | 00:58 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Update locales for Centos 8 https://review.opendev.org/684442 | 00:58 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Update redhat-common pkg-map for centos 8 https://review.opendev.org/684449 | 00:58 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Add environment switch for centos8 to use dnf https://review.opendev.org/684459 | 00:58 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Use $YUM instead of direct calls in more places https://review.opendev.org/684460 | 00:58 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: yum-minimal: Don't install yum, install libcurl https://review.opendev.org/684461 | 00:58 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] pip-and-virtualenv : fix for centos 8 https://review.opendev.org/684462 | 00:59 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] don't create interfaces on centos-8 https://review.opendev.org/684513 | 00:59 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] simple-init : don't use network.service for centos8 https://review.opendev.org/684514 | 00:59 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] centos 8 minimal testing https://review.opendev.org/684860 | 00:59 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: WIP: CentOS 8 support https://review.opendev.org/684308 | 00:59 |
| *** ansmith_ has quit IRC | 01:07 | |
| *** mschuppert has quit IRC | 01:12 | |
| *** dsneddon has quit IRC | 01:16 | |
| *** dsneddon has joined #tripleo | 01:17 | |
| *** dsneddon has quit IRC | 01:22 | |
| *** Goneri has joined #tripleo | 01:30 | |
| *** morazi has quit IRC | 01:31 | |
| *** Goneri has quit IRC | 01:39 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [wip] centos 8 minimal testing https://review.opendev.org/684860 | 01:42 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: WIP: CentOS 8 support https://review.opendev.org/684308 | 01:42 |
| *** morazi has joined #tripleo | 01:43 | |
| stevebaker | dking_desktop: the first time you call "git review" then the hook gets set up. It will also add the Change-Id to the last commit at that time. | 01:48 |
| dking_desktop | Thanks! I'm still new with the the workflow for tripleo. I finally created a user and submitted my first bug. Unfortunately, I don't seem to be able to edit it. The message says to "Please ping the PTL", but I'm not exactly sure how to do that yet. | 01:49 |
| dking_desktop | I'm hoping I did it right. I started with what I believe should just be a documentation update: https://bugs.launchpad.net/tripleo/+bug/1845574 | 01:51 |
| openstack | Launchpad bug 1845574 in tripleo "tripleo-centos-ceph-jewel.repo is not available for stein but listed in install_overcloud.rst" [Undecided,New] | 01:51 |
| stevebaker | dking_desktop: OK, I've triaged that bug. Add this to your commit message | 01:55 |
| stevebaker | Closes-Bug: #1845574 | 01:55 |
| openstack | bug 1845574 in tripleo "tripleo-centos-ceph-jewel.repo is not available for stein but listed in install_overcloud.rst" [Medium,Triaged] https://launchpad.net/bugs/1845574 - Assigned to Daniel King (dankingtech) | 01:55 |
| *** dsneddon has joined #tripleo | 01:55 | |
| stevebaker | dking_desktop: did you find out what you needed about the upload command? | 01:57 |
| *** morazi has quit IRC | 01:58 | |
| dking_desktop | That was the next issue I'm about to file. Yes, I discovered that the problem was due to a change a few months ago. The upload is no longer expected to upload the image to glance. Instead, DiscoverNode never had the default image location updated. | 01:58 |
| *** rh-jelabarre has quit IRC | 01:58 | |
| stevebaker | dking_desktop: ok, I don't know exactly what is going on there, but I can say that in the next cycle some time that glance and nova will be switched off on the undercloud, and images will be referenced by file path from that point on. | 02:01 |
| dking_desktop | That's how it's now working for the kernel and ramdisk. | 02:01 |
| stevebaker | its the future! :) | 02:02 |
| dking_desktop | I live the future. :) | 02:02 |
| stevebaker | but yeah, maybe the defaults for those should be a discovered file path instead of a glance image reference | 02:03 |
| dking_desktop | Once I finish this first one and get the hang of things, and after I file the second issue, here's the fix I'm proposing: http://paste.openstack.org/show/779526/ | 02:03 |
| dking_desktop | I'm still pretty new, so I could have missed something, but I can say that at least it doesn't complain any longer, and I get my nodes discovered and into manageable. | 02:04 |
| stevebaker | dking_desktop: ok, definitely add dtantsur|afk onto that review | 02:04 |
| stevebaker | dtantsur | 02:05 |
| openstackgerrit | Merged openstack/puppet-pacemaker master: Update the constraints url https://review.opendev.org/684873 | 02:08 |
| dking_desktop | Okay, I'll do that. It might take me a bit as I'm still trying to get the hang of things. | 02:08 |
| *** morazi has joined #tripleo | 02:10 | |
| *** Goneri has joined #tripleo | 02:13 | |
| stevebaker | dking_desktop: ok, just keep asking here for help | 02:14 |
| *** weshay has quit IRC | 02:17 | |
| openstackgerrit | Daniel King proposed openstack/tripleo-docs master: Removed reference to tripleo-centos-ceph-jewel.repo https://review.opendev.org/685193 | 02:21 |
| *** morazi has quit IRC | 02:37 | |
| *** dsneddon has quit IRC | 02:42 | |
| *** morazi has joined #tripleo | 02:42 | |
| *** artom has quit IRC | 02:42 | |
| *** dsneddon has joined #tripleo | 02:45 | |
| *** Goneri has quit IRC | 02:47 | |
| *** ricolin has joined #tripleo | 02:49 | |
| *** ykarel has joined #tripleo | 02:55 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/queens: HA: fix <service>_restart_bundle with minor update workflow https://review.opendev.org/682315 | 03:03 |
| *** rfolco has quit IRC | 03:13 | |
| *** rohit02 has joined #tripleo | 03:47 | |
| *** dsneddon has quit IRC | 03:50 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Fix networking for CentOS 8 https://review.opendev.org/684513 | 03:52 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: CentOS 8 minimal testing and support https://review.opendev.org/684860 | 03:53 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: WIP: CentOS 8 support https://review.opendev.org/684308 | 03:53 |
| *** dsneddon has joined #tripleo | 03:57 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: CentOS 8 minimal testing and support https://review.opendev.org/684860 | 03:58 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: WIP: CentOS 8 support https://review.opendev.org/684308 | 03:58 |
| *** dsneddon has quit IRC | 04:02 | |
| *** morazi has quit IRC | 04:11 | |
| *** morazi has joined #tripleo | 04:13 | |
| *** ratailor has joined #tripleo | 04:14 | |
| *** ramishra has joined #tripleo | 04:16 | |
| *** surpatil has joined #tripleo | 04:19 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add "--rm" to the container "run" command https://review.opendev.org/680435 | 04:23 |
| openstackgerrit | Merged openstack/tripleo-common stable/stein: Set display_skipped_host to no https://review.opendev.org/684717 | 04:26 |
| openstackgerrit | Merged openstack/tripleo-common stable/queens: Add profile_tasks to ansible runs https://review.opendev.org/684310 | 04:26 |
| openstackgerrit | Merged openstack/tripleo-common stable/rocky: Set display_skipped_host to no https://review.opendev.org/684719 | 04:26 |
| openstackgerrit | Merged openstack/python-tripleoclient stable/stein: [stein-only] Fix incomplete backport https://review.opendev.org/684748 | 04:26 |
| openstackgerrit | Merged openstack/paunch stable/stein: Add --cpuset-cpus support for both Docker and Podman https://review.opendev.org/684189 | 04:26 |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/stein: Fix selinux context for glance-api https://review.opendev.org/683960 | 04:26 |
| openstackgerrit | Merged openstack/tripleo-upgrade stable/stein: Handle when ceph_osds is a dictionary or a list. https://review.opendev.org/684725 | 04:26 |
| openstackgerrit | Merged openstack/tripleo-upgrade stable/stein: Add new workarounds mechanism to apply workarounds via Ansible. https://review.opendev.org/684969 | 04:26 |
| openstackgerrit | Merged openstack/tripleo-upgrade master: Enable system upgrade and upgrade run per host. https://review.opendev.org/674050 | 04:26 |
| openstackgerrit | Merged openstack/ansible-role-tripleo-modify-image master: Add support for yum caching when buildah updates https://review.opendev.org/683100 | 04:26 |
| *** janki has joined #tripleo | 04:26 | |
| *** rlandy|biab has quit IRC | 04:30 | |
| *** soniya29 has joined #tripleo | 04:35 | |
| *** skramaja has joined #tripleo | 04:36 | |
| *** dsneddon has joined #tripleo | 04:38 | |
| *** ratailor has quit IRC | 04:42 | |
| *** ratailor has joined #tripleo | 04:42 | |
| *** radeks has joined #tripleo | 04:55 | |
| *** tkajinam has quit IRC | 05:00 | |
| *** udesale has joined #tripleo | 05:00 | |
| *** tkajinam has joined #tripleo | 05:02 | |
| *** morazi has quit IRC | 05:07 | |
| *** morazi has joined #tripleo | 05:11 | |
| *** morazi has quit IRC | 05:20 | |
| *** lmiccini has joined #tripleo | 05:35 | |
| *** tkajinam_ has joined #tripleo | 05:36 | |
| *** morazi has joined #tripleo | 05:37 | |
| *** tkajinam has quit IRC | 05:38 | |
| *** ykarel is now known as ykarel|afk | 05:40 | |
| *** ratailor has quit IRC | 05:43 | |
| *** ratailor has joined #tripleo | 05:44 | |
| *** ykarel|afk has quit IRC | 05:45 | |
| *** ykarel|afk has joined #tripleo | 06:00 | |
| *** ykarel|afk is now known as ykarel | 06:08 | |
| *** ratailor_ has joined #tripleo | 06:09 | |
| *** yprokule has joined #tripleo | 06:11 | |
| *** ratailor has quit IRC | 06:11 | |
| *** aakarsh has quit IRC | 06:17 | |
| *** cylopez has joined #tripleo | 06:19 | |
| *** cylopez has quit IRC | 06:23 | |
| *** dking_desktop has quit IRC | 06:24 | |
| *** slaweq has joined #tripleo | 06:27 | |
| *** marios has joined #tripleo | 06:28 | |
| *** rcernin has quit IRC | 06:29 | |
| *** cylopez has joined #tripleo | 06:40 | |
| openstackgerrit | Andreas Jaeger proposed openstack/tripleo-ci master: Added support for train for emit release file https://review.opendev.org/684260 | 06:41 |
| *** cylopez has quit IRC | 06:55 | |
| *** hberaud|gone is now known as hberaud | 06:58 | |
| *** rpittau|afk is now known as rpittau | 07:05 | |
| *** ratailor_ has quit IRC | 07:05 | |
| *** matbu has joined #tripleo | 07:06 | |
| *** bogdando has joined #tripleo | 07:07 | |
| *** ratailor has joined #tripleo | 07:08 | |
| *** avivgt has joined #tripleo | 07:12 | |
| *** suuuper has joined #tripleo | 07:14 | |
| marios | bogdando: sorry didn't get to it on time | 07:18 |
| *** amoralej|off is now known as amoralej | 07:18 | |
| marios | bogdando: (/#/c/683100/) | 07:19 |
| *** tesseract has joined #tripleo | 07:20 | |
| *** pcaruana has joined #tripleo | 07:21 | |
| *** aedc has joined #tripleo | 07:21 | |
| *** matbu has quit IRC | 07:22 | |
| *** matbu has joined #tripleo | 07:24 | |
| *** abishop has joined #tripleo | 07:25 | |
| bogdando | marios: no worries | 07:27 |
| *** avivgt has quit IRC | 07:28 | |
| *** cylopez has joined #tripleo | 07:31 | |
| *** holser has joined #tripleo | 07:32 | |
| *** paramite has joined #tripleo | 07:32 | |
| *** kopecmartin|off is now known as kopecmartin | 07:36 | |
| *** tosky has joined #tripleo | 07:37 | |
| *** apetrich has quit IRC | 07:40 | |
| openstackgerrit | Bogdan Dobrelya (bogdando) proposed openstack/tripleo-common master: Always close src/dst sessions and raise to retry https://review.opendev.org/684786 | 07:40 |
| *** jpena|off is now known as jpena | 07:41 | |
| *** jfrancoa has joined #tripleo | 07:47 | |
| *** jaosorior has quit IRC | 07:48 | |
| *** alexmcleod has joined #tripleo | 07:51 | |
| *** soniya29 is now known as soniya29|lunch | 07:52 | |
| *** jpich has joined #tripleo | 07:52 | |
| *** apetrich has joined #tripleo | 07:52 | |
| *** surpatil is now known as surpatil|lunch | 07:53 | |
| *** matbu has quit IRC | 07:59 | |
| *** holser has quit IRC | 08:01 | |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade stable/stein: Enable system upgrade and upgrade run per host. https://review.opendev.org/685218 | 08:04 |
| *** matbu has joined #tripleo | 08:06 | |
| *** holser has joined #tripleo | 08:11 | |
| Tengu | hmmm. there's an issue in zuul status page - link to the review is wrong, it's lacking the leading "https://" (or, even, "//"), meaning it points to http://zuul.opendev.org/t/openstack/review.opendev.org/XXX instead of review.opendev.org/XXX | 08:14 |
| Tengu | who should I ping about that? | 08:14 |
| Tengu | marios: -^^ ? | 08:15 |
| *** matbu has quit IRC | 08:16 | |
| *** tkajinam_ has quit IRC | 08:16 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-ansible master: Optional flag file for tripleo-transfer to prevent overwriting https://review.opendev.org/684678 | 08:17 |
| *** matbu has joined #tripleo | 08:18 | |
| *** marios has quit IRC | 08:19 | |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates master: Prevent running mysql data transfer twice by accident https://review.opendev.org/685226 | 08:21 |
| openstackgerrit | Jiri Stransky proposed openstack/tripleo-heat-templates stable/stein: Prevent running mysql data transfer twice by accident https://review.opendev.org/685229 | 08:23 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-upgrade stable/rocky: Handle when ceph_osds is a dictionary or a list. https://review.opendev.org/685231 | 08:23 |
| *** soniya29|lunch is now known as soniya29 | 08:28 | |
| *** aedc has quit IRC | 08:29 | |
| *** cylopez has quit IRC | 08:30 | |
| *** aedc has joined #tripleo | 08:31 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart master: Added release file for train https://review.opendev.org/683932 | 08:32 |
| *** derekh has joined #tripleo | 08:33 | |
| *** surpatil|lunch is now known as surpatil | 08:34 | |
| Tengu | bogdando: do you want some help on the close session thing? | 08:35 |
| bogdando | Tengu: hi, I think it's done | 08:36 |
| bogdando | just CI randomly failing | 08:36 |
| Tengu | bogdando: ok! | 08:36 |
| Tengu | lemme test it in my env, I wanted to check something else in relation to the image-serve thing :) | 08:36 |
| bogdando | IIRC, the first revision already wsa passing | 08:36 |
| bogdando | Tengu: sure, go for it | 08:37 |
| Tengu | would be good to get mwhahaha patch merged today.. apparently it's shortening by 30 minutes. really good improvement | 08:38 |
| bogdando | Tengu: hm, that actually was the 3rd one passing see Zuul | 08:38 |
| bogdando | 26. Sep 03:28 Patch Set 3 | 08:38 |
| bogdando | Tengu: indeed | 08:38 |
| Tengu | add the fact we should gain about 10-15 minutes with your patch, we should be fine regarding timeous. | 08:38 |
| Tengu | *timeouts | 08:38 |
| bogdando | hope so | 08:38 |
| Tengu | we already shortened a bit with the duplicated loop removal. a couple of minutes overall. Not much, but that's already something. | 08:39 |
| Tengu | aaaand we have to get EmilienM patch about the inflight-validations merged as well. | 08:39 |
| Tengu | we might get 45 minutes back with all of that. | 08:39 |
| Tengu | and that would be fantastic :) | 08:39 |
| bogdando | yea | 08:40 |
| *** ykarel is now known as ykarel|lunch | 08:41 | |
| *** hberaud has quit IRC | 08:42 | |
| *** hberaud has joined #tripleo | 08:44 | |
| *** ratailor has quit IRC | 08:48 | |
| *** chem has joined #tripleo | 08:48 | |
| *** ratailor has joined #tripleo | 08:51 | |
| *** tosky has quit IRC | 08:51 | |
| *** pcaruana has quit IRC | 08:52 | |
| *** rcernin has joined #tripleo | 08:54 | |
| Tengu | bogdando: quick question: do you know if we actually re-use existing layers within the image-serve? Since the whole image building is done in order to make reusable layer a thing, it would be bad to actually duplicate them on the disk.... right? | 08:59 |
| Tengu | iirc docker registry uses symlink in such case. | 08:59 |
| bogdando | Tengu: nope, I don't know that | 09:00 |
| bogdando | should be not hard to check it live tho | 09:00 |
| bogdando | f.e. using export vs save commands and comparint the size of tarballs | 09:00 |
| Tengu | bogdando: running "find /var/lib/image-serve/ -type l" gives a frightening answer | 09:01 |
| bogdando | comparing* | 09:01 |
| bogdando | the one of those deduplicate layers | 09:01 |
| bogdando | not sure it would be right method although, may be just looking into httpd contents? | 09:01 |
| bogdando | yea | 09:01 |
| Tengu | will wait until the end of the image-prepare thing | 09:01 |
| bogdando | dups all around? | 09:01 |
| Tengu | but apparently...... NO symlink in there. | 09:02 |
| bogdando | nice | 09:02 |
| Tengu | maybe hard-link? let's check the code. | 09:02 |
| *** pcaruana has joined #tripleo | 09:03 | |
| *** ykarel|lunch is now known as ykarel | 09:12 | |
| Tengu | bogdando: ah, hard-link are present. | 09:12 |
| Tengu | harder to find, but it's OK. | 09:12 |
| Tengu | some layer is linked 22 times for instance :). | 09:12 |
| bogdando | Tengu: that's better then, but isn't hard link do not dedup the used space?.. | 09:13 |
| owalsh | Tengu: hey, I'm seeing failure in standalone jobs because the 120 healthcheck delay is being exceeded | 09:13 |
| Tengu | bogdando: I think it's ok because it's duping the inode | 09:13 |
| bogdando | ok | 09:13 |
| Tengu | owalsh: https://review.opendev.org/685063 | 09:13 |
| Tengu | it wasn't properly deactivated last time. | 09:13 |
| owalsh | Tengu: yea, but... isn't it an issue even if we ignore it? | 09:14 |
| bogdando | owalsh: got link? | 09:14 |
| bogdando | let's see why it happens | 09:14 |
| Tengu | owalsh: well, it might? but since decision was made to not test that (for now....) it will be hard to dig deeper into that ;_; | 09:14 |
| owalsh | bogdando: https://b6174f913a50d8c1a7b9-bfbfd8db497e7d54c8b9e3b8f437fb98.ssl.cf1.rackcdn.com/681042/16/check/tripleo-ci-centos-7-standalone/0c8b172/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz | 09:15 |
| Tengu | one more reason to give us some CI job. | 09:15 |
| owalsh | bogdando: container started at 2019-09-26T20:42:09.005178216Z https://b6174f913a50d8c1a7b9-bfbfd8db497e7d54c8b9e3b8f437fb98.ssl.cf1.rackcdn.com/681042/16/check/tripleo-ci-centos-7-standalone/0c8b172/logs/undercloud/var/log/extra/podman/containers/nova_api/podman_info.log.txt.gz | 09:15 |
| owalsh | bogdando: nova-api logs start at 2019-09-26 20:44:50.479 | 09:16 |
| Tengu | owalsh: weird: TASK [Get nova-api healthcheck status] is OK after 3 retries, but [Fail if nova-api healthcheck report failed status] does fail in the end. | 09:16 |
| owalsh | Tengu: that's what it's designed to do - wait for a healthcheck to run then fail if the healthcheck failed | 09:16 |
| Tengu | so it's not a timeout right? | 09:16 |
| owalsh | Tengu: healthcheck failed because nova-api wasn't running after 120s | 09:16 |
| Tengu | https://b6174f913a50d8c1a7b9-bfbfd8db497e7d54c8b9e3b8f437fb98.ssl.cf1.rackcdn.com/681042/16/check/tripleo-ci-centos-7-standalone/0c8b172/logs/undercloud/var/log/containers/stdouts/nova_api.log.txt.gz apparently it starts earlier than you said? | 09:17 |
| *** dtantsur|afk is now known as dtantsur | 09:18 | |
| owalsh | Tengu: I said container started at 2019-09-26T20:42:09.005178216Z | 09:18 |
| owalsh | nova-api isn't accepting connection for another couple of minutes | 09:19 |
| Tengu | maybe there's something to do about that delay? | 09:19 |
| Tengu | why does it take that long? | 09:19 |
| owalsh | probably just load... I really don't like how racy this is | 09:20 |
| Tengu | hm. wait. this is step 5 ?? isn't nova started, like, at step 4 or 3? | 09:20 |
| owalsh | Tengu: step4 | 09:20 |
| owalsh | Tengu: validation run in the next step | 09:21 |
| owalsh | for now anyway... probably need to add some sort of post_deply_step_tasks for this | 09:21 |
| owalsh | or a step_6 to validate the step_5 services but yuck | 09:22 |
| Tengu | hmm.... it appears nova-api image is pulled multiple time?! | 09:23 |
| Tengu | INFO:nova_wait_for_api_service:Nova-api service active // python2 /container-config-scripts/nova_wait_for_api_service.py | 09:23 |
| Tengu | what's this? | 09:23 |
| owalsh | Tengu: it's a deploy task that I'm adding | 09:23 |
| Tengu | that task should prevent going to the next step until it's over? | 09:24 |
| *** marios has joined #tripleo | 09:24 | |
| *** rpittau is now known as rpittau|bbl | 09:24 | |
| owalsh | Tengu: it doesn't necessarily work :-) Still a WIP | 09:24 |
| *** ricolin has quit IRC | 09:24 | |
| *** dtantsur is now known as dtantsur|lunch | 09:24 | |
| Tengu | hence the "should" :) | 09:25 |
| Tengu | owalsh: if we're to add a 6th step in order to validate step5 services, that's not an inflight-validation anymore, right? | 09:25 |
| Tengu | that's more a "post-deploy task that should be launched by operator", using the new "openstack tripleo validator run ...."... ? | 09:26 |
| owalsh | Tengu: I need to move nova-compute to step_5 so I'll need somewhere to run the inflight-validation tasks.... | 09:26 |
| Tengu | inflight shouldn't have a negative impact on the deploy time, it's already long enough :/. if we have to add waiters/timers/retries, there's something wrong somewhere imho | 09:26 |
| owalsh | Tengu: that's a contradiction IMO | 09:27 |
| owalsh | validating that the service is up and running is going to add time unless the service tasks 0s to initialize | 09:27 |
| owalsh | s/tasks/takes/ | 09:27 |
| Tengu | yes. but maybe adding a 5 minutes delay is a bit too much? | 09:27 |
| owalsh | there isn't 5 minutes delay | 09:28 |
| Tengu | 2 minutes for now, but since it appears to be too short... ;) | 09:28 |
| Tengu | I know how delays go in the code. | 09:28 |
| *** iurygregory has joined #tripleo | 09:28 | |
| owalsh | Tengu: I'm thinking it might be best to just drop the 120s delay in the timer and just deal with the fact that the initial healthcheck fails when running the inflight-valations | 09:29 |
| Tengu | hm, so what's the point of the inflight validation there? if we ignore its failed state... ? or did I misundertand you? | 09:30 |
| Tengu | if load is an issue, maybe we should reduce the amount of parallel tasks? | 09:30 |
| Tengu | multi-processing is nice, but to some extends it makes things slower than plain sequential | 09:31 |
| Tengu | (and adds race conditions capabilities) | 09:31 |
| owalsh | Tengu: I'm not saying ignore the failed state - retry a number of times and if it's still failed we fail the validation | 09:31 |
| owalsh | if a service comes up really quickly then this should minimise the impact on deploy time | 09:32 |
| Tengu | owalsh: retry with some delay between each retries, and 0 initial delay then? with a max number of retry, allowing to know the max wait_time it induces? | 09:32 |
| Tengu | sounds better indeed. | 09:32 |
| Tengu | and this might even give some air to a loaded node, since it souldn't start other tasks during those retries... ? | 09:32 |
| openstackgerrit | Sergii Golovatiuk proposed openstack/tripleo-heat-templates master: Fix typo https://review.opendev.org/685107 | 09:33 |
| owalsh | Tengu: indeed. Could keep the delay small but make retry pretty high since we don't want to be too eager to kill a slow deployment | 09:35 |
| Tengu | #food - brb | 09:37 |
| owalsh | Tengu: ack, I gotta go afk too. ttyl | 09:37 |
| *** mschuppert has joined #tripleo | 09:38 | |
| bogdando | owalsh, Tengu: looking onto dstat it doesn't look like high load https://b6174f913a50d8c1a7b9-bfbfd8db497e7d54c8b9e3b8f437fb98.ssl.cf1.rackcdn.com/681042/16/check/tripleo-ci-centos-7-standalone/0c8b172/logs/undercloud/var/log/extra/dstat.html.gz | 09:40 |
| owalsh | bogdando: maybe waiting on another service? e.g db or mq | 09:41 |
| bogdando | the major wait raise to 35%+ can only be observed after a few minutes later after the nova_api container started | 09:41 |
| bogdando | tho LA was high enough for ... is it a v1 cpu system? | 09:43 |
| owalsh | bogdando: 8 cpus in var/log/extra/cpuinfo.txt.gz | 09:44 |
| bogdando | yay... so no, that issue has nothing to system load | 09:45 |
| owalsh | bogdando: I'll take a look at the service start_order in step_4 after lunch.... | 09:46 |
| * owalsh biab | 09:46 | |
| *** hberaud is now known as hberaud|lunch | 09:53 | |
| bogdando | owalsh: we don't have waiting loops for containers starting AFAIK | 09:56 |
| bogdando | we'd see amqp or db reconnects in the service logs then | 09:56 |
| bogdando | Tengu, owalsh: seems worth reporting a new bug | 09:58 |
| bogdando | >2 min delay after starting needs some extended debugging together with containers-team prolly | 09:59 |
| *** skramaja has quit IRC | 10:02 | |
| *** ratailor has quit IRC | 10:11 | |
| *** morazi has quit IRC | 10:13 | |
| Tengu | back | 10:15 |
| Tengu | bogdando: I agree with you. | 10:15 |
| *** aedc has quit IRC | 10:24 | |
| *** ansmith_ has joined #tripleo | 10:25 | |
| Tengu | grumpf. https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_fd7/684411/5/gate/tripleo-ci-centos-7-containers-multinode/fd7051e/logs/undercloud/home/zuul/undercloud_install.log.txt.gz also hit by that same nova-api thing. | 10:25 |
| Tengu | and that's in gate ;_; | 10:25 |
| *** iurygregory has quit IRC | 10:27 | |
| *** iurygregory has joined #tripleo | 10:27 | |
| *** xek has quit IRC | 10:31 | |
| *** ratailor has joined #tripleo | 10:31 | |
| *** xek has joined #tripleo | 10:31 | |
| openstackgerrit | yatin proposed openstack/tripleo-ci master: [DNM] Test queens with rhcloud provider settings https://review.opendev.org/685316 | 10:32 |
| *** florianf has joined #tripleo | 10:32 | |
| *** ansmith_ has quit IRC | 10:34 | |
| *** tesseract has quit IRC | 10:35 | |
| *** tesseract has joined #tripleo | 10:35 | |
| *** ayoung has quit IRC | 10:37 | |
| *** ayoung has joined #tripleo | 10:38 | |
| *** sshnaidm|off is now known as sshnaidm|pto | 10:44 | |
| *** ratailor_ has joined #tripleo | 10:45 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-common stable/rocky: Add profile_tasks to ansible runs https://review.opendev.org/684309 | 10:45 |
| *** ratailor has quit IRC | 10:47 | |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-common stable/queens: Set display_skipped_host to no https://review.opendev.org/684709 | 10:48 |
| *** rainleander has joined #tripleo | 10:49 | |
| *** leanderthal has quit IRC | 10:51 | |
| bogdando | Tengu: oh, that's why we can merge nothing it seems :) | 10:51 |
| Tengu | yeah... I suspect something changed in nova-api thing lately.... | 10:52 |
| Tengu | since it did work yesterday for instance. | 10:52 |
| Tengu | we were almost out of un-gate-ing things | 10:52 |
| openstackgerrit | Sorin Sbarnea proposed openstack/tripleo-common stable/queens: Set display_skipped_host to no https://review.opendev.org/684709 | 10:54 |
| Tengu | so we really need to make https://review.opendev.org/#/c/685063/3 happen. | 10:57 |
| Tengu | unfortunately, it just failed zuul on some weird thing with tox on py27 o_O | 10:57 |
| Tengu | si recheck..... and crossing fingers. | 10:57 |
| Tengu | once this one is merged, we should be able to, at last, merge mwhahaha patch for the speed, and get things green again. | 10:58 |
| *** dtantsur|lunch is now known as dtantsur | 11:03 | |
| *** iurygregory has quit IRC | 11:03 | |
| *** florianf has quit IRC | 11:09 | |
| *** bogdando has quit IRC | 11:16 | |
| *** jpena is now known as jpena|lunch | 11:17 | |
| *** bogdando has joined #tripleo | 11:17 | |
| *** jaosorior has joined #tripleo | 11:19 | |
| *** florianf has joined #tripleo | 11:19 | |
| *** lmiccini has quit IRC | 11:20 | |
| *** ansmith_ has joined #tripleo | 11:25 | |
| openstackgerrit | Merged openstack/tripleo-common master: Correct deprecated usage of ironicclient (broken with 3.0.0) https://review.opendev.org/684288 | 11:26 |
| *** matbu has quit IRC | 11:28 | |
| *** matbu has joined #tripleo | 11:29 | |
| *** hberaud|lunch is now known as hberaud | 11:29 | |
| *** fultonj has joined #tripleo | 11:33 | |
| *** jaosorior has quit IRC | 11:33 | |
| *** ansmith_ has quit IRC | 11:34 | |
| openstackgerrit | Chandan Kumar (raukadah) proposed openstack/tripleo-quickstart master: Added release file for train https://review.opendev.org/683932 | 11:36 |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-heat-templates master: Allow using registry authentication to pull ceph related containers https://review.opendev.org/682015 | 11:45 |
| *** ratailor__ has joined #tripleo | 11:49 | |
| *** ratailor_ has quit IRC | 11:51 | |
| *** weshay has joined #tripleo | 11:55 | |
| Tengu | o_O is there something specific regarding tox-py27 job? https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_005/685063/3/check/openstack-tox-py27/005fc4b/job-output.txt | 11:57 |
| Tengu | nothing about the failures seems to be linked to the patch itself (https://review.opendev.org/#/c/685063/) | 11:57 |
| Tengu | more over, `tox -epy27' is passing on my env, so I'm a bit lost right now. | 11:58 |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-heat-templates master: Add registry authentication parameters to pull ceph related containers https://review.opendev.org/677699 | 12:02 |
| *** morazi has joined #tripleo | 12:03 | |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-heat-templates master: Allow using registry authentication to pull ceph related containers https://review.opendev.org/682015 | 12:04 |
| *** zul has quit IRC | 12:06 | |
| *** abishop has quit IRC | 12:08 | |
| *** florianf has quit IRC | 12:08 | |
| *** abishop has joined #tripleo | 12:08 | |
| *** dsneddon has quit IRC | 12:09 | |
| *** dsneddon has joined #tripleo | 12:09 | |
| *** bogdando_ has joined #tripleo | 12:09 | |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Add DHCPv6 rules + set ipversion for dhcp rules https://review.opendev.org/684385 | 12:10 |
| *** bogdando has quit IRC | 12:10 | |
| *** bogdando_ is now known as bogdando | 12:10 | |
| *** jpena|lunch is now known as jpena | 12:10 | |
| *** bnemec has quit IRC | 12:10 | |
| *** mburned has quit IRC | 12:11 | |
| *** bnemec has joined #tripleo | 12:11 | |
| hjensas | mdbooth: did you open a downstream bugzilla for the mtu issues? If we open one for OSP-13 we can probably backport it? | 12:16 |
| *** ansmith has joined #tripleo | 12:18 | |
| *** ekultails has joined #tripleo | 12:19 | |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/python-tripleoclient master: DO NOT MERGE - testing tox-py27 https://review.opendev.org/685336 | 12:20 |
| *** mburned has joined #tripleo | 12:20 | |
| Tengu | grumpf. of course. | 12:21 |
| openstackgerrit | Cédric Jeanneret (Tengu) proposed openstack/python-tripleoclient master: DO NOT MERGE - testing tox-py27 https://review.opendev.org/685336 | 12:21 |
| *** rh-jelabarre has joined #tripleo | 12:21 | |
| *** rh-jelabarre has quit IRC | 12:23 | |
| *** rh-jelabarre has joined #tripleo | 12:23 | |
| *** jfrancoa has quit IRC | 12:26 | |
| openstackgerrit | yatin proposed openstack/tripleo-puppet-elements stable/queens: Add element for openssh and override UseDNS https://review.opendev.org/685342 | 12:27 |
| openstackgerrit | yatin proposed openstack/tripleo-common stable/queens: overcloud: include openssh element https://review.opendev.org/685343 | 12:28 |
| ykarel | amoralej, weshay ^^ | 12:28 |
| ykarel | checking one more patch if it's needed or not | 12:28 |
| *** ratailor__ has quit IRC | 12:30 | |
| *** iurygregory has joined #tripleo | 12:31 | |
| *** abishop has quit IRC | 12:32 | |
| ykarel | those two should be enough, let's see how job goes | 12:33 |
| owalsh | ykarel: I thought UseDNS was no by default? | 12:35 |
| owalsh | maybe the rpm defaults are different... | 12:36 |
| *** rpittau|bbl is now known as rpittau | 12:36 | |
| ykarel | owalsh, default is yes | 12:36 |
| ykarel | https://linux.die.net/man/5/sshd_config | 12:36 |
| ykarel | UseDNS:- Specifies whether sshd(8) should look up the remote host name and check that the resolved host name for the remote IP address maps back to the very same IP address. The default is ''yes''. | 12:37 |
| owalsh | ykarel: it changed at some point. Default is definitely no on F30 and RHEL8 | 12:38 |
| *** rfolco has joined #tripleo | 12:38 | |
| ykarel | owalsh, may be rpm doing it, let me check | 12:39 |
| *** rlandy has joined #tripleo | 12:40 | |
| *** rohit02 has quit IRC | 12:41 | |
| *** jfrancoa has joined #tripleo | 12:42 | |
| openstackgerrit | Takashi Kajinami proposed openstack/tripleo-heat-templates master: Add support to configure token caching https://review.opendev.org/683012 | 12:46 |
| openstackgerrit | Takashi Kajinami proposed openstack/tripleo-heat-templates master: DNM: Test token caching in keystone https://review.opendev.org/683172 | 12:46 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-quickstart master: update queens for bm testing https://review.opendev.org/685346 | 12:47 |
| *** dtrainor has quit IRC | 12:48 | |
| arxcruz|rover | Hey guys, i'm having this problem | 12:49 |
| arxcruz|rover | https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_15c/685302/1/check/kolla-build-centos-source/15c63ba/kolla/build/000_FAILED_monasca-grafana.txt.gz | 12:49 |
| *** bogdando has quit IRC | 12:50 | |
| arxcruz|rover | the monasca-grafana try to install a gem called rake, but it requires ruby 2.2 and the installed one is ruby 2.0 | 12:50 |
| arxcruz|rover | can someone help me figure out where to fix it? | 12:50 |
| openstackgerrit | Takashi Kajinami proposed openstack/tripleo-heat-templates master: Use OS::TripleO::Services::Logging::NovaLibvirt to configure libvirt logging https://review.opendev.org/683752 | 12:50 |
| openstackgerrit | Takashi Kajinami proposed openstack/tripleo-heat-templates master: Place readmet.txt for libvirt log https://review.opendev.org/684750 | 12:50 |
| openstackgerrit | Takashi Kajinami proposed openstack/tripleo-heat-templates master: Use OS::TripleO::Services::Logging::NovaLibvirt to configure libvirt logging https://review.opendev.org/683752 | 12:51 |
| *** soniya29 has quit IRC | 12:51 | |
| *** aedc has joined #tripleo | 12:53 | |
| *** dtrainor has joined #tripleo | 12:53 | |
| *** matbu has quit IRC | 12:54 | |
| ykarel | owalsh, https://git.centos.org/rpms/openssh/c/b7cc2db2dbb20a2ab098940cd802532c91d583b8?branch=c7 | 12:55 |
| ykarel | there it was changed to 1 | 12:55 |
| *** matbu has joined #tripleo | 12:56 | |
| ykarel | https://bugzilla.redhat.com/show_bug.cgi?id=1478175 | 12:56 |
| openstack | bugzilla.redhat.com bug 1478175 in openssh "The UseDNS configuration option changed default behaviour between 7.3 and 7.4, this breaks existing configurations until manually corrected" [Urgent,Closed: errata] - Assigned to jjelen | 12:56 |
| owalsh | ykarel: ack, explains why it's different in rhel8 | 12:56 |
| ykarel | owalsh, yes c8 don't have that patch https://git.centos.org/rpms/openssh/blob/c8/f/SPECS/openssh.spec | 12:57 |
| *** udesale has quit IRC | 12:58 | |
| owalsh | yea, it's more ok to change the behaviour when it's a different major release | 12:58 |
| *** udesale has joined #tripleo | 12:58 | |
| *** florianf_ has joined #tripleo | 13:00 | |
| mdbooth | hjensas: FWIW, just rebased your Ctrl-B patch on to my series so I could use it to check what MTU value is set by default on the introspection network, and it's 1450. Is that consistent with your expectations? | 13:00 |
| *** raildo has joined #tripleo | 13:01 | |
| *** lucasagomes has joined #tripleo | 13:02 | |
| *** tkajinam has joined #tripleo | 13:05 | |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-heat-templates master: Allow using registry authentication to pull ceph related containers https://review.opendev.org/682015 | 13:08 |
| *** pcaruana has quit IRC | 13:09 | |
| *** aedc has quit IRC | 13:09 | |
| *** mcornea has joined #tripleo | 13:10 | |
| hjensas | mdbooth: so 1450 is the mtu of the undercloud's br-ctlplane interface? | 13:10 |
| *** amoralej is now known as amoralej|lunch | 13:10 | |
| hjensas | mdbooth: and thanks for the rebase. :) | 13:14 |
| openstackgerrit | Kamil Sambor proposed openstack/tripleo-heat-templates master: Support connecting OVN DB over SSL https://review.opendev.org/680345 | 13:16 |
| *** rcernin has quit IRC | 13:18 | |
| mdbooth | hjensas: I shall have to check again in a while, as I just blew it away :) | 13:19 |
| openstackgerrit | Takashi Kajinami proposed openstack/puppet-tripleo master: Add support to configure virtlogd https://review.opendev.org/685349 | 13:20 |
| openstackgerrit | Takashi Kajinami proposed openstack/tripleo-heat-templates master: WIP: Configure virtlogd logging in addition to libvirt logging https://review.opendev.org/685350 | 13:25 |
| *** ricolin has joined #tripleo | 13:30 | |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: DNM: ensure we have at least 2 nova-api worker https://review.opendev.org/685351 | 13:30 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-quickstart master: Add support for running tempest against Octavia featuresets https://review.opendev.org/669666 | 13:31 |
| openstackgerrit | Brent Eagles proposed openstack/tripleo-quickstart master: Set the amphora image to stein branch https://review.opendev.org/678899 | 13:31 |
| owalsh | Tengu: ^^^ nova-api apache is configured with just a single worker/thread for standalone jobs, smells like a possible culprit | 13:34 |
| Tengu | owalsh: ah. did it change lately? | 13:35 |
| Tengu | EmilienM: -^^ maybe a solution for the nova-api issue, but still..... | 13:36 |
| owalsh | Tengu: not that I know of, but the healthcheck tasks did so it might be triggering it | 13:36 |
| owalsh | Tengu: it's from the low_memory_usage environment file | 13:37 |
| Tengu | owalsh: ah. damn. | 13:39 |
| *** artom has joined #tripleo | 13:46 | |
| owalsh | Tengu: seems to be reasonably stable in CI though, so maybe just my patch results in more requests hitting the api while the healthcheck runs | 13:47 |
| owalsh | hrmm nope - https://b6174f913a50d8c1a7b9-bfbfd8db497e7d54c8b9e3b8f437fb98.ssl.cf1.rackcdn.com/681042/16/check/tripleo-ci-centos-7-standalone/0c8b172/logs/undercloud/var/log/containers/httpd/nova-api/nova_api_wsgi_access.log.txt.gz | 13:48 |
| Tengu | :] | 13:48 |
| owalsh | hrmm, maybe it's searching for entry points that slows it down - https://b6174f913a50d8c1a7b9-bfbfd8db497e7d54c8b9e3b8f437fb98.ssl.cf1.rackcdn.com/681042/16/check/tripleo-ci-centos-7-standalone/0c8b172/logs/undercloud/var/log/containers/httpd/nova-api/nova_api_wsgi_error.log.txt.gz | 13:50 |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-heat-templates master: Allow using registry authentication to pull ceph related containers https://review.opendev.org/682015 | 13:50 |
| *** rainleander is now known as leanderthal | 13:51 | |
| *** surpatil has quit IRC | 13:55 | |
| cloudnull | mornings | 13:56 |
| *** udesale has quit IRC | 13:56 | |
| *** ratailor has joined #tripleo | 14:02 | |
| *** pcaruana has joined #tripleo | 14:04 | |
| *** Vorrtex has joined #tripleo | 14:07 | |
| *** mcornea has quit IRC | 14:08 | |
| *** amoralej|lunch is now known as amoralej | 14:12 | |
| openstackgerrit | Takashi Kajinami proposed openstack/puppet-tripleo master: Add support to configure virtlogd https://review.opendev.org/685349 | 14:12 |
| *** mcornea has joined #tripleo | 14:14 | |
| openstackgerrit | Carlos Goncalves proposed openstack/tripleo-heat-templates stable/stein: Add new parameter options to Octavia service https://review.opendev.org/685361 | 14:16 |
| *** gbarros has joined #tripleo | 14:19 | |
| openstackgerrit | Carlos Goncalves proposed openstack/tripleo-heat-templates stable/rocky: Add new parameter options to Octavia service https://review.opendev.org/685362 | 14:22 |
| openstackgerrit | Carlos Goncalves proposed openstack/tripleo-heat-templates stable/queens: Add new parameter options to Octavia service https://review.opendev.org/685363 | 14:23 |
| openstackgerrit | Tom Barron proposed openstack/tripleo-heat-templates master: Run Manila in scenario004 without pacemaker https://review.opendev.org/682853 | 14:24 |
| *** tkajinam has quit IRC | 14:24 | |
| *** hberaud is now known as hberaud|school-r | 14:25 | |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: WIP: ensure nova-api is running before starting nova-compute https://review.opendev.org/681042 | 14:26 |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: DNM: ensure we have at least 2 nova-api worker https://review.opendev.org/685351 | 14:27 |
| openstackgerrit | Jose Luis Franco proposed openstack/tripleo-quickstart-extras master: centos-release-openstack-ocata package not available. https://review.opendev.org/685364 | 14:31 |
| *** jaosorior has joined #tripleo | 14:34 | |
| openstackgerrit | Kamil Sambor proposed openstack/tripleo-heat-templates master: Support connecting OVN DB over SSL https://review.opendev.org/680345 | 14:34 |
| *** pcaruana has quit IRC | 14:35 | |
| *** avivgt has joined #tripleo | 14:36 | |
| *** hberaud|school-r is now known as hberaud | 14:37 | |
| *** aedc has joined #tripleo | 14:46 | |
| *** radeks has quit IRC | 14:49 | |
| *** _mmethot_ has quit IRC | 14:50 | |
| openstackgerrit | Slawek Kaplonski proposed openstack/tripleo-quickstart-extras master: Add playbook and role to run tobiko https://review.opendev.org/655423 | 14:51 |
| *** ekultails has quit IRC | 14:51 | |
| *** ratailor has quit IRC | 14:52 | |
| *** matbu has quit IRC | 14:58 | |
| *** ekultails has joined #tripleo | 15:00 | |
| openstackgerrit | yatin proposed openstack/tripleo-quickstart-extras master: Fix regex to detect overcloud image build https://review.opendev.org/685368 | 15:03 |
| ykarel | amoralej, weshay image image build didn't ran on usedns fixes due to ^^ | 15:03 |
| *** matbu has joined #tripleo | 15:04 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Implement threading locks around layers https://review.opendev.org/685175 | 15:05 |
| openstackgerrit | yatin proposed openstack/tripleo-quickstart-extras master: Fix regex to detect overcloud image build https://review.opendev.org/685368 | 15:05 |
| openstackgerrit | yatin proposed openstack/tripleo-common stable/queens: overcloud: include openssh element https://review.opendev.org/685343 | 15:06 |
| *** avivgt has quit IRC | 15:06 | |
| *** ykarel is now known as ykarel|afk | 15:08 | |
| mwhahaha | cloudnull -^ i think i addressed the possible conflicts with the thread locking | 15:09 |
| weshay | ykarel|afk, thanks.. looking | 15:10 |
| *** mmethot has joined #tripleo | 15:11 | |
| EmilienM | Tengu, mwhahaha : I'm about to send a patch to disable voting on py27 job | 15:13 |
| EmilienM | for tripleoclient only | 15:13 |
| *** morazi has quit IRC | 15:13 | |
| mwhahaha | k | 15:15 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Implement threading locks around layers https://review.opendev.org/685175 | 15:19 |
| *** avivgt has joined #tripleo | 15:20 | |
| *** dtantsur is now known as dtantsur|afk | 15:20 | |
| Tengu | EmilienM++ | 15:22 |
| *** arxcruz|rover is now known as arxcruz|zzz | 15:22 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: Disable voting on openstack-tox-py27 https://review.opendev.org/685378 | 15:24 |
| *** avivgt has quit IRC | 15:25 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Implement threading locks around layers https://review.opendev.org/685175 | 15:27 |
| *** avivgt has joined #tripleo | 15:27 | |
| *** yprokule has quit IRC | 15:28 | |
| *** artom has quit IRC | 15:28 | |
| openstackgerrit | Emilien Macchi proposed openstack/python-tripleoclient master: Disable voting on openstack-tox-py27 https://review.opendev.org/685378 | 15:29 |
| cloudnull | mwhahaha looks great! | 15:30 |
| *** avivgt has quit IRC | 15:32 | |
| *** marios has quit IRC | 15:32 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Improve ThreadPoolExecutor usage https://review.opendev.org/684411 | 15:35 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Improving debug logging https://review.opendev.org/684835 | 15:35 |
| *** pcaruana has joined #tripleo | 15:36 | |
| *** ykarel|afk is now known as ykarel | 15:39 | |
| ykarel | EmilienM, hi | 15:39 |
| *** alexmcleod has quit IRC | 15:39 | |
| ykarel | EmilienM, re. https://bugs.launchpad.net/puppet-nova/+bug/1845616 | 15:39 |
| openstack | Launchpad bug 1845616 in puppet-nova "puppet-openstack-beaker-centos-7 failing on puppet-nova changes" [Undecided,New] | 15:39 |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: DNM: increase keystone workers in low-memory-usage env https://review.opendev.org/685351 | 15:40 |
| ykarel | EmilienM, this can be related to osc-4.0.0, specifically https://github.com/openstack/python-openstackclient/commit/03a2accb2f13f82b1090e1ae6f154bf263e7e252 | 15:40 |
| ykarel | we have seen similar issue in puppet-cinder related to idempotency, https://review.opendev.org/#/q/topic:osc_train | 15:41 |
| ykarel | which was fixed like https://review.opendev.org/#/c/682936/2/lib/puppet/provider/cinder_type/openstack.rb | 15:42 |
| EmilienM | ykarel: ha | 15:42 |
| EmilienM | https://review.opendev.org/#/c/682936/2/lib/puppet/provider/cinder_type/openstack.rb | 15:42 |
| ykarel | jpena, fyi ^^ | 15:43 |
| EmilienM | I see | 15:43 |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Implement threading locks around layers https://review.opendev.org/685175 | 15:44 |
| *** avivgt has joined #tripleo | 15:45 | |
| EmilienM | ykarel: I'll fix it | 15:46 |
| ykarel | EmilienM, ack | 15:47 |
| *** rpittau is now known as rpittau|afk | 15:48 | |
| *** aedc has quit IRC | 15:49 | |
| jpena | ykarel: that metadata definition looks very much like the cinder type properties | 15:50 |
| ykarel | jpena, yes | 15:50 |
| *** jbadiapa has joined #tripleo | 15:56 | |
| *** artom has joined #tripleo | 15:57 | |
| *** suuuper has quit IRC | 16:00 | |
| *** dking_desktop has joined #tripleo | 16:06 | |
| *** jpich has quit IRC | 16:06 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Implement threading locks around layers https://review.opendev.org/685175 | 16:13 |
| *** abishop has joined #tripleo | 16:15 | |
| *** dking_desktop has quit IRC | 16:18 | |
| openstackgerrit | Jaganathan Palanisamy proposed openstack/tripleo-heat-templates master: Incorrect group name issue on non DPDK compute https://review.opendev.org/679227 | 16:20 |
| *** jaosorior has quit IRC | 16:21 | |
| *** chandankumar is now known as raukadah | 16:21 | |
| *** chem has quit IRC | 16:23 | |
| openstackgerrit | Merged openstack/tripleo-common stable/stein: [Stein] Optional flag file for tripleo-transfer to prevent overwriting https://review.opendev.org/684963 | 16:24 |
| *** holser has quit IRC | 16:26 | |
| *** bdodd has joined #tripleo | 16:29 | |
| *** florianf_ has quit IRC | 16:30 | |
| *** abishop has quit IRC | 16:31 | |
| *** jpena is now known as jpena|off | 16:31 | |
| *** tesseract has quit IRC | 16:32 | |
| *** dking_desktop has joined #tripleo | 16:32 | |
| *** ykarel has quit IRC | 16:34 | |
| *** abishop has joined #tripleo | 16:36 | |
| *** zbr|ruck is now known as zbr | 16:41 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates stable/stein: Resolve missing parenthesis causing variable exception https://review.opendev.org/685185 | 16:49 |
| openstackgerrit | Merged openstack/tripleo-common master: Add timestamps to log file for tripleo-container-image-prepare https://review.opendev.org/684810 | 16:49 |
| *** ykarel has joined #tripleo | 16:50 | |
| *** ramishra has quit IRC | 16:52 | |
| openstackgerrit | Daniel King proposed openstack/tripleo-docs master: Removed references to tripleo-centos-ceph-jewel.repo https://review.opendev.org/685193 | 16:53 |
| *** derekh has quit IRC | 17:01 | |
| *** ekultails has quit IRC | 17:03 | |
| *** ekultails has joined #tripleo | 17:06 | |
| *** weshay is now known as weshay|ruck | 17:08 | |
| *** amoralej is now known as amoralej|off | 17:10 | |
| openstackgerrit | yatin proposed openstack/ansible-role-tripleo-modify-image master: [DNM] Filter out repos not available or enabled https://review.opendev.org/685431 | 17:15 |
| *** radeks has joined #tripleo | 17:20 | |
| *** pcaruana has quit IRC | 17:23 | |
| cloudnull | mwhahaha on where are you seeing collisions https://review.opendev.org/#/c/685175 ? | 17:27 |
| cloudnull | do you have a good local way to test, something I can help with ? | 17:27 |
| *** abishop has quit IRC | 17:27 | |
| *** abishop has joined #tripleo | 17:28 | |
| *** itlinux has joined #tripleo | 17:39 | |
| dking_desktop | Is there a place I can find a list of tags to add to a bug report? | 17:46 |
| *** janki has quit IRC | 17:49 | |
| *** hberaud is now known as hberaud|gone | 17:50 | |
| *** bfournie has quit IRC | 17:56 | |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-heat-templates master: Allow using registry authentication to pull ceph related containers https://review.opendev.org/682015 | 17:58 |
| *** bfournie has joined #tripleo | 18:09 | |
| openstackgerrit | John Fulton proposed openstack/tripleo-quickstart master: Use CentOS Storage Sig Ceph for RHEL8 scenarios https://review.opendev.org/685449 | 18:11 |
| *** dciabrin_ has joined #tripleo | 18:19 | |
| *** dciabrin has quit IRC | 18:20 | |
| *** abishop has quit IRC | 18:21 | |
| ykarel | weshay|ruck, amoralej|off as suspected it moved forward(ssh was successful) but failed later at public connectivity(running ntpdate):- https://logs.rdoproject.org/43/685343/2/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/66f5b35/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz | 18:22 |
| ykarel | wrt. UseDNS patch:- https://review.opendev.org/#/c/685343/ | 18:22 |
| *** gbarros has quit IRC | 18:48 | |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-docs master: Add note on ceph registry authentication https://review.opendev.org/685456 | 18:49 |
| ykarel | EmilienM, cloudnull weshay|ruck, https://review.opendev.org/#/c/683100/ has broken atleast rocky https://review.rdoproject.org/zuul/builds?job_name=tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-rocky-branch | 18:56 |
| ykarel | seen rocky failure in one of my patch and reached to this patch | 18:56 |
| ykarel | yum_update.sh was used by docker too | 18:57 |
| ykarel | https://opendev.org/openstack/ansible-role-tripleo-modify-image/src/branch/master/tasks/yum_update_docker.yml#L47 | 18:58 |
| ykarel | copy vs template | 18:58 |
| *** itlinux has quit IRC | 19:01 | |
| *** lucasagomes has quit IRC | 19:01 | |
| mwhahaha | cloudnull: http://paste.openstack.org/show/779705/ | 19:02 |
| mwhahaha | cloudnull: that on an undercloud works (tm) | 19:03 |
| mwhahaha | cloudnull: then you can just run the following to reset the registry: rm -rf /var/lib/image-serve/v2/_catalog /var/lib/image-serve/v2/{tripleomaster,ceph,grafana,prom} | 19:03 |
| *** ricolin has quit IRC | 19:03 | |
| openstackgerrit | Harald Jensås proposed openstack/openstack-virtual-baremetal master: Add radvd helper instance for ipv6 provisioning https://review.opendev.org/685460 | 19:04 |
| openstackgerrit | Harald Jensås proposed openstack/openstack-virtual-baremetal master: [WiP] Add IPv6 radvd and dhcpv6 relay support https://review.opendev.org/685461 | 19:04 |
| mwhahaha | cloudnull: https://paste.centos.org/view/6c8a064d is the list of contention/collisions | 19:05 |
| mwhahaha | cloudnull: i'm not going to add the shuffle patch, but in my original testing teh conflicts were much lower when shuffled | 19:06 |
| mwhahaha | cloudnull: basically locking is still as slow as not locking (32 mins in my env) | 19:06 |
| mwhahaha | dking_desktop: https://bugs.launchpad.net/tripleo has some of the tags on the right side. we have a place in tripleo-specs i think, but we don't really use a bunch of tags other than the low-hanging-fruit, tech-debt, ci or the backport ones | 19:09 |
| *** Vorrtex has quit IRC | 19:10 | |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-heat-templates master: Add prometheus and alertmanager port params https://review.opendev.org/685465 | 19:11 |
| *** radeks has quit IRC | 19:12 | |
| openstackgerrit | yatin proposed openstack/ansible-role-tripleo-modify-image master: Switch to template for yum_update.sh https://review.opendev.org/685466 | 19:15 |
| *** kopecmartin is now known as kopecmartin|off | 19:15 | |
| ykarel | mwhahaha, EmilienM, weshay|ruck ^^ to fix rocky jobs | 19:16 |
| *** rh-jelabarre has quit IRC | 19:16 | |
| *** jamesdenton has joined #tripleo | 19:25 | |
| *** ayoung has quit IRC | 19:25 | |
| *** mcornea has quit IRC | 19:25 | |
| *** ayoung has joined #tripleo | 19:26 | |
| *** ykarel is now known as ykarel|away | 19:29 | |
| *** mcornea has joined #tripleo | 19:33 | |
| *** ayoung has quit IRC | 19:39 | |
| *** avivgt has quit IRC | 19:39 | |
| mwhahaha | cloudnull: so for comparison here is the same grep from the shuffle+locking https://paste.centos.org/view/6743b5e3 (about half the conflict messages 1029 vs 578) | 19:40 |
| *** fultonj has quit IRC | 19:42 | |
| *** ayoung has joined #tripleo | 19:42 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Remove containers before removing associated storage https://review.opendev.org/681647 | 19:43 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-heat-templates master: Add second fact to ensure type safty https://review.opendev.org/685469 | 19:45 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/tripleo-heat-templates master: Add second fact to ensure type safty https://review.opendev.org/685469 | 19:47 |
| weshay|ruck | rlandy++ | 19:49 |
| rlandy | let's see how this one goes | 19:50 |
| *** jbadiapa has quit IRC | 19:50 | |
| openstackgerrit | Alex Schultz proposed openstack/tripleo-common master: Implement threading locks around layers https://review.opendev.org/685175 | 19:51 |
| *** rh-jelabarre has joined #tripleo | 19:53 | |
| *** avivgt has joined #tripleo | 19:54 | |
| openstackgerrit | Merged openstack/tripleo-heat-templates master: Add no_log to include_vars https://review.opendev.org/682988 | 19:54 |
| openstackgerrit | wes hayutin proposed openstack/tripleo-heat-templates master: Run Manila in scenario004 without pacemaker https://review.opendev.org/682853 | 19:55 |
| *** rh-jelabarre has quit IRC | 20:06 | |
| *** rh-jelabarre has joined #tripleo | 20:06 | |
| openstackgerrit | Martin Mágr proposed openstack/puppet-tripleo master: Configuration changes to support Qdr-mesh topology. https://review.opendev.org/647908 | 20:07 |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-heat-templates master: Allow using registry authentication to pull ceph related containers https://review.opendev.org/682015 | 20:08 |
| openstackgerrit | Martin Mágr proposed openstack/puppet-tripleo master: Configuration changes to support Qdr-mesh topology. https://review.opendev.org/647908 | 20:13 |
| openstackgerrit | Merged openstack/tripleo-common master: Set display_skipped_host to no https://review.opendev.org/683428 | 20:16 |
| *** ykarel|away has quit IRC | 20:16 | |
| openstackgerrit | Francesco Pantano proposed openstack/tripleo-heat-templates master: Allow using registry authentication to pull ceph related containers https://review.opendev.org/682015 | 20:17 |
| *** mcornea has quit IRC | 20:23 | |
| *** dsneddon has quit IRC | 20:28 | |
| *** paramite has quit IRC | 20:43 | |
| *** paramite has joined #tripleo | 20:44 | |
| *** paramite has quit IRC | 20:52 | |
| *** jtomasek has quit IRC | 20:55 | |
| dking_desktop | Where would be a good place to learn about running tests locally for the python-tripleoclient repository? Should I just install tox and run it, or are there other steps that I should take? | 20:56 |
| mwhahaha | yea just tox | 20:56 |
| mwhahaha | usually in a virtual env | 20:56 |
| dking_desktop | Okay. I just tried to run it, just for -epy27 and got a bunch of failures. | 20:57 |
| *** dsneddon has joined #tripleo | 20:57 | |
| dking_desktop | - Passed: 566 / - Failed: 22 Shouldn't all tests pass on a clean pull from master? | 20:58 |
| *** avivgt has quit IRC | 21:02 | |
| mwhahaha | dking_desktop: py27 is broken on master because of some updates to the test framework | 21:05 |
| mwhahaha | dking_desktop: we generally will only officially support python3 with train, so i'd do py36 if possible | 21:05 |
| mwhahaha | i use pyenv + virtualenvs | 21:05 |
| *** aedc has joined #tripleo | 21:06 | |
| dking_desktop | Okay. I can try that. It's a shame, though about python2, as that's still the default for CentOS. | 21:09 |
| mwhahaha | yea but while we have packages for train it's not really tested outside of CI | 21:10 |
| mwhahaha | we just wish centos8 came out sooner :/ | 21:10 |
| weshay|ruck | like before rhel | 21:11 |
| * weshay|ruck runs | 21:11 | |
| dking_desktop | Well, centos has never been fast. | 21:11 |
| mwhahaha | a novel concept | 21:11 |
| *** avivgt has joined #tripleo | 21:15 | |
| *** jfrancoa has quit IRC | 21:17 | |
| mwhahaha | weshay|ruck: fyi i think https://review.opendev.org/#/c/685134/ shaves ~5 mins of container prepare for the undercloud | 21:19 |
| weshay|ruck | k /me looks | 21:27 |
| mwhahaha | undercloud took ~1100 seconds | 21:28 |
| mwhahaha | which is at lewast 300 seconds faster than anything i spotted in the job history | 21:28 |
| *** raildo has quit IRC | 21:32 | |
| weshay|ruck | mwhahaha++ | 21:35 |
| weshay|ruck | fastest I see w/ data in the last 48 hours is 22 min | 21:36 |
| weshay|ruck | http://dashboard-ci.tripleo.org/d/si1tipHZk/jobs-exploration?orgId=1&from=now-2d&to=now | 21:36 |
| mwhahaha | dat shuffle | 21:36 |
| mwhahaha | I shall call it "The Container Shuffle" | 21:36 |
| mwhahaha | and you & EmilienM can do a music video | 21:37 |
| weshay|ruck | mwhahaha, "the Container Truffle Shuffle" | 21:37 |
| weshay|ruck | mwhahaha, only if ur the guy in the back in dark glasses, arms folded and just slightly bobbing his head | 21:37 |
| weshay|ruck | and of course.. | 21:37 |
| weshay|ruck | FLIPS A TABLE at the end of the video | 21:37 |
| *** rlandy has quit IRC | 21:39 | |
| mwhahaha | HEH | 21:40 |
| *** rfolco has quit IRC | 21:42 | |
| *** mmethot has quit IRC | 22:04 | |
| *** lhinds has quit IRC | 22:07 | |
| *** jpena|off has quit IRC | 22:07 | |
| *** lhinds has joined #tripleo | 22:08 | |
| *** jpena|off has joined #tripleo | 22:09 | |
| *** rfolco has joined #tripleo | 22:10 | |
| *** rcernin has joined #tripleo | 22:25 | |
| *** dsneddon has quit IRC | 22:29 | |
| *** xek has quit IRC | 22:43 | |
| *** rcernin has quit IRC | 22:48 | |
| *** slaweq has quit IRC | 22:50 | |
| openstackgerrit | Oliver Walsh proposed openstack/tripleo-heat-templates master: Ensure nova-api is running before starting nova-compute containers https://review.opendev.org/681042 | 22:53 |
| *** ade_lee_ has quit IRC | 23:11 | |
| *** dsneddon has joined #tripleo | 23:18 | |
| *** dsneddon has quit IRC | 23:23 | |
| *** ekultails has quit IRC | 23:32 | |
| *** mmethot has joined #tripleo | 23:47 | |
| openstackgerrit | Harald Jensås proposed openstack/tripleo-heat-templates master: Fix Ironic configuration for IPv6 https://review.opendev.org/685507 | 23:57 |
| *** mmethot has quit IRC | 23:57 | |
| openstackgerrit | Harald Jensås proposed openstack/python-tripleoclient master: IronicIPXEIpVersion: '6' if local_ip is IPv6 https://review.opendev.org/685508 | 23:57 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!