| *** weifan has quit IRC | 00:08 | |
| *** gyee has quit IRC | 00:15 | |
| *** diablo_rojo has quit IRC | 00:17 | |
| *** dmsimard0 is now known as dmsimard | 00:25 | |
| *** rcernin has quit IRC | 00:25 | |
| *** rcernin has joined #openstack-infra | 00:27 | |
| *** liuyulong has joined #openstack-infra | 00:28 | |
| *** mattw4 has quit IRC | 00:37 | |
| *** michael-beaver has quit IRC | 00:43 | |
| openstackgerrit | Joshua Hesketh proposed zuul/zuul master: Expose ansible_date_time instead of date_time https://review.opendev.org/666268 | 00:44 |
|---|---|---|
| *** pkopec has joined #openstack-infra | 00:46 | |
| *** yamamoto has quit IRC | 00:48 | |
| *** ricolin has joined #openstack-infra | 00:48 | |
| *** goldyfruit has joined #openstack-infra | 00:56 | |
| *** markvoelker has joined #openstack-infra | 01:02 | |
| *** rcernin_ has joined #openstack-infra | 01:05 | |
| *** rcernin has quit IRC | 01:06 | |
| *** rajinir has quit IRC | 01:16 | |
| *** markvoelker has quit IRC | 01:22 | |
| ianw | i'm going to quickly reboot iad with the afs-next branch kernel. i'll connect this up to a netconsole. will add all details to https://etherpad.openstack.org/p/opendev-mirror-afs | 01:26 |
| *** pkopec has quit IRC | 01:32 | |
| *** _erlon_ has joined #openstack-infra | 01:32 | |
| *** spsurya has joined #openstack-infra | 01:39 | |
| *** rlandy|bbl is now known as rlandy | 01:42 | |
| *** goldyfruit has quit IRC | 01:42 | |
| *** apetrich has quit IRC | 01:57 | |
| openstackgerrit | David Pursehouse proposed opendev/system-config master: Remove dead link from 'paste' documentation https://review.opendev.org/666474 | 02:25 |
| *** yamamoto has joined #openstack-infra | 02:32 | |
| openstackgerrit | David Pursehouse proposed opendev/lodgeit master: Update URLs from openstack.org to opendev.org https://review.opendev.org/666475 | 02:33 |
| *** rlandy has quit IRC | 02:44 | |
| *** jamesmcarthur has joined #openstack-infra | 02:47 | |
| *** zhurong has joined #openstack-infra | 02:48 | |
| *** Goneri has quit IRC | 03:18 | |
| *** psachin has joined #openstack-infra | 03:22 | |
| *** bhavikdbavishi has joined #openstack-infra | 03:23 | |
| *** ykarel has joined #openstack-infra | 03:25 | |
| *** bhavikdbavishi1 has joined #openstack-infra | 03:26 | |
| *** bhavikdbavishi has quit IRC | 03:27 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 03:28 | |
| openstackgerrit | David Pursehouse proposed opendev/lodgeit master: Update URLs from openstack.org to opendev.org https://review.opendev.org/666475 | 03:30 |
| openstackgerrit | David Pursehouse proposed opendev/lodgeit master: Fix pep8 errors https://review.opendev.org/666479 | 03:30 |
| *** jamesmcarthur has quit IRC | 03:35 | |
| *** jamesmcarthur has joined #openstack-infra | 03:42 | |
| *** jamesmcarthur has quit IRC | 03:47 | |
| *** whoami-rajat has joined #openstack-infra | 03:47 | |
| *** jamesmcarthur has joined #openstack-infra | 03:55 | |
| *** udesale has joined #openstack-infra | 03:56 | |
| *** _erlon_ has quit IRC | 03:56 | |
| *** haleyb has quit IRC | 03:58 | |
| *** ramishra has joined #openstack-infra | 04:07 | |
| *** kjackal has joined #openstack-infra | 04:11 | |
| *** eernst has quit IRC | 04:22 | |
| *** dave-mccowan has quit IRC | 04:25 | |
| *** janki has joined #openstack-infra | 04:26 | |
| *** jamesmcarthur has quit IRC | 04:26 | |
| *** gregoryo has joined #openstack-infra | 04:34 | |
| *** jamesmcarthur has joined #openstack-infra | 04:34 | |
| *** ykarel has quit IRC | 04:40 | |
| *** bhavikdbavishi has quit IRC | 04:49 | |
| *** ykarel has joined #openstack-infra | 04:55 | |
| *** jamesmcarthur has quit IRC | 05:04 | |
| *** Lucas_Gray has joined #openstack-infra | 05:09 | |
| *** jamesmcarthur has joined #openstack-infra | 05:16 | |
| *** Lucas_Gray has quit IRC | 05:21 | |
| *** jamesmcarthur has quit IRC | 05:21 | |
| *** weifan has joined #openstack-infra | 05:24 | |
| *** jamesmcarthur has joined #openstack-infra | 05:27 | |
| *** weifan has quit IRC | 05:28 | |
| *** jamesmcarthur has quit IRC | 05:31 | |
| *** yamamoto has quit IRC | 05:32 | |
| *** raukadah is now known as chandankumar | 05:40 | |
| *** jamesmcarthur has joined #openstack-infra | 05:47 | |
| *** liuyulong has quit IRC | 05:47 | |
| *** ccamacho has quit IRC | 05:48 | |
| *** yamamoto has joined #openstack-infra | 05:52 | |
| *** jamesmcarthur has quit IRC | 05:52 | |
| *** kjackal has quit IRC | 06:02 | |
| *** kjackal has joined #openstack-infra | 06:02 | |
| *** lpetrut has joined #openstack-infra | 06:06 | |
| *** udesale has quit IRC | 06:07 | |
| *** udesale has joined #openstack-infra | 06:08 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/project-config master: Normalize projects.yaml https://review.opendev.org/665869 | 06:14 |
| *** pcaruana has joined #openstack-infra | 06:15 | |
| *** yamamoto has quit IRC | 06:19 | |
| *** jamesmcarthur has joined #openstack-infra | 06:20 | |
| *** yboaron_ has joined #openstack-infra | 06:21 | |
| *** jamesmcarthur has quit IRC | 06:30 | |
| *** yamamoto has joined #openstack-infra | 06:35 | |
| *** ccamacho has joined #openstack-infra | 06:42 | |
| *** udesale has quit IRC | 06:45 | |
| *** jamesmcarthur has joined #openstack-infra | 06:45 | |
| *** udesale has joined #openstack-infra | 06:46 | |
| *** jtomasek has joined #openstack-infra | 06:47 | |
| *** pgaxatte has joined #openstack-infra | 06:51 | |
| *** jamesmcarthur has quit IRC | 06:52 | |
| *** kopecmartin|off is now known as kopecmartin | 06:53 | |
| evrardjp | ianw: about https://review.opendev.org/#/c/666268/ ... you seem to refer to https://review.opendev.org/#/c/660576/ which is for me different: not in promote pipelines, and for which the date problem could be solved other ways (like you did with 'date', or by adding a setup task to refresh the facts). I am not sure to get your point, you would vote positively if there is a documentation pointing to the fact we expose | 06:54 |
| evrardjp | ansible_date_time on executors? | 06:54 |
| *** apetrich has joined #openstack-infra | 06:58 | |
| *** gregoryo has quit IRC | 07:00 | |
| *** dchen has quit IRC | 07:01 | |
| *** rcernin_ has quit IRC | 07:06 | |
| *** tesseract has joined #openstack-infra | 07:07 | |
| *** trident has quit IRC | 07:08 | |
| *** ccamacho has quit IRC | 07:09 | |
| *** trident has joined #openstack-infra | 07:10 | |
| *** slaweq has joined #openstack-infra | 07:11 | |
| *** slaweq has quit IRC | 07:15 | |
| *** rpittau|afk is now known as rpittau | 07:20 | |
| *** jamesmcarthur has joined #openstack-infra | 07:21 | |
| *** markvoelker has joined #openstack-infra | 07:22 | |
| *** tosky has joined #openstack-infra | 07:26 | |
| *** udesale has quit IRC | 07:27 | |
| *** udesale has joined #openstack-infra | 07:28 | |
| *** bhavikdbavishi has joined #openstack-infra | 07:32 | |
| *** udesale has quit IRC | 07:33 | |
| *** udesale has joined #openstack-infra | 07:33 | |
| *** udesale has quit IRC | 07:33 | |
| *** ginopc has joined #openstack-infra | 07:33 | |
| *** jamesmcarthur has quit IRC | 07:33 | |
| *** udesale has joined #openstack-infra | 07:34 | |
| *** bhavikdbavishi1 has joined #openstack-infra | 07:35 | |
| *** bhavikdbavishi has quit IRC | 07:36 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 07:36 | |
| *** ykarel is now known as ykarel|lunch | 07:38 | |
| *** yamamoto has quit IRC | 07:38 | |
| *** ralonsoh has joined #openstack-infra | 07:39 | |
| *** yamamoto has joined #openstack-infra | 07:39 | |
| *** slaweq has joined #openstack-infra | 07:42 | |
| *** yamamoto has quit IRC | 07:44 | |
| *** jamesmcarthur has joined #openstack-infra | 07:44 | |
| *** jpena|off is now known as jpena | 07:46 | |
| *** slaweq has quit IRC | 07:47 | |
| *** jamesmcarthur has quit IRC | 07:48 | |
| *** jamesmcarthur has joined #openstack-infra | 07:50 | |
| *** hamzy has quit IRC | 07:50 | |
| *** auristor has quit IRC | 07:50 | |
| *** markvoelker has quit IRC | 07:50 | |
| *** hamzy has joined #openstack-infra | 07:51 | |
| *** dklyle has quit IRC | 07:52 | |
| *** auristor has joined #openstack-infra | 07:53 | |
| yoctozepto | E: Failed to fetch http://mirror.iad.rax.opendev.org/ubuntu/dists/xenial-updates/main/binary-amd64/Packages.gz Hash Sum mismatch | 07:53 |
| yoctozepto | 'tis mirror a joke :P | 07:53 |
| yoctozepto | guys, is there a way to retrigger only gate jobs instead of check+gate? because it introduces a too high risk of random failure... | 07:54 |
| ianw | evrardjp: yes i think it needs to be documented. i guess my point is that "ansible_date_time" is actually *not* the time, it's something else. so we're now giving what is already a confusing variable another meaning too | 07:55 |
| ianw | yoctozepto: unfortunately no. can you link to that? i'd like to see the timestamp | 07:55 |
| ianw | there appear to be no errors on the iad mirror | 07:56 |
| *** jamesmcarthur has quit IRC | 07:57 | |
| *** yamamoto has joined #openstack-infra | 07:58 | |
| *** jamesmcarthur has joined #openstack-infra | 07:58 | |
| ianw | evrardjp: one other thought that just occured is i'm not sure it's 100% compatible with ansible_date_time ... no epoch value? | 07:59 |
| yoctozepto | ianw: http://logs.openstack.org/32/665432/2/gate/openstack-tox-pep8/70d1168/ | 07:59 |
| yoctozepto | it's from yesterday | 08:00 |
| *** iurygregory_ is now known as iurygregory | 08:00 | |
| yoctozepto | but there were so many failures yesterday due to m.iad.rax.o.o | 08:00 |
| yoctozepto | so afraid to issue a recheck :D | 08:00 |
| *** jpich has joined #openstack-infra | 08:01 | |
| *** bhavikdbavishi has quit IRC | 08:06 | |
| *** lucasagomes has joined #openstack-infra | 08:06 | |
| *** bhavikdbavishi has joined #openstack-infra | 08:06 | |
| evrardjp | ianw: yes, so it's not 100% the same meaning as ansible_date_time (because UTC vs server time to not leak info) | 08:08 |
| evrardjp | but an user can have an approximatively right value of time with this | 08:08 |
| evrardjp | (see also commit message on why I am exposing it again as ansible_date_time vs date_time) | 08:08 |
| evrardjp | I think wronging expectations for users is bad -- and require them to have ansible_date_time for some pipelines and date_time for others is terrible user experience | 08:09 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add command processor to zuul-web https://review.opendev.org/666307 | 08:10 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Add repl server for debug purposes https://review.opendev.org/579962 | 08:10 |
| evrardjp | for the "time it was cached" problem, I think it's not really a problem it's ansible behaviour, and the promote pipelines should be short lived | 08:10 |
| evrardjp | (keeping in mind it only applies in localhost) | 08:11 |
| *** slaweq has joined #openstack-infra | 08:11 | |
| *** yboaron_ has quit IRC | 08:13 | |
| *** bhavikdbavishi has quit IRC | 08:13 | |
| *** tkajinam has quit IRC | 08:16 | |
| *** slaweq has quit IRC | 08:16 | |
| *** zbr has joined #openstack-infra | 08:17 | |
| *** igordc has quit IRC | 08:20 | |
| ianw | yoctozepto: oh, ok ... well things are in progress. we're running directly from dhowells afs-next branch now, which has some fixes for us, including better trace points if something does go wrong | 08:21 |
| *** ociuhandu has joined #openstack-infra | 08:22 | |
| ianw | evrardjp: i'm just a bit sensitive because i wasted a lot of time debugging this silly variable :) do we agree it should be documented though? | 08:23 |
| *** udesale has quit IRC | 08:23 | |
| evrardjp | :) | 08:23 |
| *** slaweq has joined #openstack-infra | 08:23 | |
| evrardjp | Yeah I am fine with that, I just want to know my action points :) | 08:23 |
| evrardjp | And make sure things are well explained | 08:23 |
| *** udesale has joined #openstack-infra | 08:23 | |
| yoctozepto | ianw: thanks for info, I'll let you know if I catch something fresh | 08:27 |
| *** yboaron_ has joined #openstack-infra | 08:28 | |
| *** slaweq has quit IRC | 08:28 | |
| zbr | did anyone worked on building some starts regarding reviews, like the average number of rechecks a review needs before being merged? | 08:28 |
| zbr | i have a gut feeling that building such a graph, per months could paint a dark picture. | 08:29 |
| *** ociuhandu has quit IRC | 08:34 | |
| *** ykarel|lunch is now known as ykarel | 08:36 | |
| openstackgerrit | Arnaud Morin proposed openstack/project-config master: Reduce OVH BHS1 pool size to 120 https://review.opendev.org/666518 | 08:37 |
| *** yboaron_ has quit IRC | 08:37 | |
| amorin | hey all, I propose to reduce OVH pool size ^ | 08:38 |
| *** yboaron_ has joined #openstack-infra | 08:38 | |
| amorin | cc fungi mordred | 08:38 |
| *** imacdonn has quit IRC | 08:41 | |
| *** imacdonn has joined #openstack-infra | 08:41 | |
| *** ccamacho has joined #openstack-infra | 08:45 | |
| *** markvoelker has joined #openstack-infra | 08:47 | |
| *** ginux has joined #openstack-infra | 08:48 | |
| *** ginux is now known as Guest17979 | 08:49 | |
| *** Guest17979 is now known as ginux | 08:51 | |
| *** yboaron_ has quit IRC | 08:51 | |
| *** ginux is now known as Guest29102 | 08:51 | |
| *** ginopc has quit IRC | 08:52 | |
| *** Guest29102 is now known as ginopc | 08:52 | |
| *** markvoelker has quit IRC | 08:52 | |
| *** ociuhandu has joined #openstack-infra | 08:54 | |
| *** yolanda has quit IRC | 08:54 | |
| *** jamesmcarthur has quit IRC | 08:55 | |
| *** pcrews has quit IRC | 08:55 | |
| *** iurygregory_ has joined #openstack-infra | 08:58 | |
| *** jamesmcarthur has joined #openstack-infra | 08:59 | |
| *** iurygregory has quit IRC | 08:59 | |
| *** slaweq has joined #openstack-infra | 09:00 | |
| *** ociuhandu has quit IRC | 09:02 | |
| *** yboaron_ has joined #openstack-infra | 09:06 | |
| *** jamesmcarthur has quit IRC | 09:08 | |
| *** ginopc has quit IRC | 09:09 | |
| *** Lucas_Gray has joined #openstack-infra | 09:11 | |
| *** derekh has joined #openstack-infra | 09:12 | |
| *** gfidente has joined #openstack-infra | 09:13 | |
| *** psachin has quit IRC | 09:14 | |
| *** roman_g has joined #openstack-infra | 09:15 | |
| *** ginopc has joined #openstack-infra | 09:17 | |
| *** jamesmcarthur has joined #openstack-infra | 09:18 | |
| *** ginopc has quit IRC | 09:21 | |
| *** e0ne has joined #openstack-infra | 09:22 | |
| openstackgerrit | Javier Peña proposed opendev/puppet-openstackci master: Add AFS mirror support for RHEL/CentOS https://review.opendev.org/528739 | 09:22 |
| *** ykarel is now known as ykarel|afk | 09:28 | |
| openstackgerrit | Ian Wienand proposed zuul/nodepool master: Pin to openshift <= 0.8.9 https://review.opendev.org/666526 | 09:31 |
| ianw | amorin: ^ how urgent is it we turn it down? looks like you've unfortunately hit a gate breakage; i can force merge it if required | 09:33 |
| *** slaweq has quit IRC | 09:34 | |
| *** jamesmcarthur has quit IRC | 09:36 | |
| openstackgerrit | Jean-Philippe Evrard proposed zuul/zuul-jobs master: Dockerhub now returns 200 for DELETEs https://review.opendev.org/666529 | 09:37 |
| *** roman_g has quit IRC | 09:44 | |
| *** yamamoto has quit IRC | 09:45 | |
| *** jamesmcarthur has joined #openstack-infra | 09:45 | |
| *** psachin has joined #openstack-infra | 09:47 | |
| *** kjackal has quit IRC | 09:47 | |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Add Bitbucket Server source functionality https://review.opendev.org/657837 | 09:47 |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Create a basic Bitbucket build status reporter https://review.opendev.org/658335 | 09:47 |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Create a basic Bitbucket event source https://review.opendev.org/658835 | 09:47 |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Upgrade formatting of the patch series. https://review.opendev.org/660683 | 09:47 |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Extend event reporting https://review.opendev.org/662134 | 09:47 |
| *** markvoelker has joined #openstack-infra | 09:48 | |
| *** kjackal has joined #openstack-infra | 09:52 | |
| *** markvoelker has quit IRC | 09:53 | |
| *** bhavikdbavishi has joined #openstack-infra | 09:55 | |
| *** janki has quit IRC | 09:55 | |
| amorin | ianw: dont worry | 09:58 |
| amorin | could be nice to have it before the weekend | 09:59 |
| amorin | our duty guy would love it :p | 09:59 |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Extend event reporting https://review.opendev.org/662134 | 09:59 |
| *** roman_g has joined #openstack-infra | 09:59 | |
| *** jamesmcarthur has quit IRC | 10:00 | |
| *** iurygregory_ has quit IRC | 10:00 | |
| *** gfidente has quit IRC | 10:04 | |
| *** iurygregory has joined #openstack-infra | 10:04 | |
| stephenfin | Has anyone else noticed that accessing the nova project on opendev.org (https://opendev.org/openstack/nova) is very slow? | 10:08 |
| stephenfin | I mean, compared to something like https://opendev.org/openstack/os-vif/ | 10:08 |
| amorin | stephenfin: I confirm | 10:09 |
| *** gfidente has joined #openstack-infra | 10:14 | |
| evrardjp | I guess it depends on the size of the repo? :) | 10:17 |
| stephenfin | I'd say so too | 10:17 |
| stephenfin | It's probably regenerating all that metadata in the right column on each page load | 10:18 |
| stephenfin | gitea, that is | 10:18 |
| stephenfin | Worth noting GitHub doesn't suffer the same issue. Load times are consistent there | 10:18 |
| openstackgerrit | Jean-Philippe Evrard proposed zuul/zuul-jobs master: Dockerhub now returns 200 for DELETEs https://review.opendev.org/666529 | 10:20 |
| *** slaweq has joined #openstack-infra | 10:21 | |
| evrardjp | corvus: did you notice issues in latest promote pipelines? I ran something in no_log at home and found out DELETE on dockerhub returned 200. So I committed this: https://review.opendev.org/#/c/666529/ . I don't have enough feedback for what has triggered that change though, but I think you should have a look anyway :) | 10:21 |
| *** yboaron_ has quit IRC | 10:21 | |
| *** yboaron_ has joined #openstack-infra | 10:21 | |
| evrardjp | I should probably have said that in zuul's channel. Darned. | 10:21 |
| *** electrofelix has joined #openstack-infra | 10:23 | |
| *** yboaron_ has quit IRC | 10:24 | |
| *** yamamoto has joined #openstack-infra | 10:25 | |
| *** yboaron_ has joined #openstack-infra | 10:26 | |
| *** electrofelix has quit IRC | 10:27 | |
| *** electrofelix has joined #openstack-infra | 10:27 | |
| *** yboaron_ has quit IRC | 10:28 | |
| *** yboaron_ has joined #openstack-infra | 10:29 | |
| openstackgerrit | Marcin Juszkiewicz proposed opendev/system-config master: UCA: mirror Train packages https://review.opendev.org/666533 | 10:32 |
| *** yboaron_ has quit IRC | 10:33 | |
| lpetrut | fungi: Hi, thanks for the insight about zuul jobs and the fact that rebases won't be required. We had some post merge jobs that were not run and was trying to figure out why. | 10:34 |
| *** ociuhandu has joined #openstack-infra | 10:35 | |
| *** yamamoto has quit IRC | 10:35 | |
| lpetrut | fungi: we have two post merge jobs, one of which mirrors the repo to github | 10:35 |
| lpetrut | fungi: for example, the post merge jobs haven't run for 597324,16 as shown here: http://zuul.openstack.org/builds?project=x%2Fcloudbase-init | 10:36 |
| *** yamamoto has joined #openstack-infra | 10:39 | |
| *** jamesmcarthur has joined #openstack-infra | 10:45 | |
| *** trident has quit IRC | 10:47 | |
| *** trident has joined #openstack-infra | 10:48 | |
| *** markvoelker has joined #openstack-infra | 10:49 | |
| *** jamesmcarthur has quit IRC | 10:50 | |
| *** jamesmcarthur has joined #openstack-infra | 10:50 | |
| *** ykarel|afk is now known as ykarel | 10:52 | |
| *** markvoelker has quit IRC | 10:54 | |
| *** jpena is now known as jpena|lunch | 10:59 | |
| openstackgerrit | Alexander Braverman proposed zuul/nodepool master: Openshift client https://review.opendev.org/666541 | 10:59 |
| *** Lucas_Gray has quit IRC | 11:02 | |
| *** jamesmcarthur has quit IRC | 11:06 | |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Extend event reporting https://review.opendev.org/662134 | 11:08 |
| *** ykarel has quit IRC | 11:08 | |
| *** jamesmcarthur has joined #openstack-infra | 11:09 | |
| *** ykarel has joined #openstack-infra | 11:09 | |
| *** yamamoto has quit IRC | 11:10 | |
| *** ginopc has joined #openstack-infra | 11:13 | |
| *** yamamoto has joined #openstack-infra | 11:16 | |
| *** yamamoto has quit IRC | 11:16 | |
| *** guimaluf has quit IRC | 11:16 | |
| *** yamamoto has joined #openstack-infra | 11:16 | |
| *** janki has joined #openstack-infra | 11:18 | |
| *** rfolco_off has joined #openstack-infra | 11:20 | |
| *** yamamoto has quit IRC | 11:20 | |
| *** rfolco_off is now known as rfolco | 11:21 | |
| *** yamamoto has joined #openstack-infra | 11:27 | |
| *** ginopc has quit IRC | 11:27 | |
| *** ginopc has joined #openstack-infra | 11:28 | |
| *** rlandy has joined #openstack-infra | 11:30 | |
| *** yamamoto has quit IRC | 11:32 | |
| *** rlandy is now known as rlandy|afk | 11:33 | |
| *** lpetrut has quit IRC | 11:35 | |
| *** rfolco is now known as rfolco_pto | 11:36 | |
| *** udesale has quit IRC | 11:36 | |
| *** udesale has joined #openstack-infra | 11:37 | |
| *** goldyfruit has joined #openstack-infra | 11:37 | |
| *** yamamoto has joined #openstack-infra | 11:49 | |
| *** markvoelker has joined #openstack-infra | 11:50 | |
| *** rh-jelabarre has joined #openstack-infra | 11:50 | |
| *** markvoelker has quit IRC | 11:54 | |
| *** rh-jelabarre has quit IRC | 11:57 | |
| *** janki has quit IRC | 11:57 | |
| *** rh-jelabarre has joined #openstack-infra | 11:58 | |
| *** udesale has quit IRC | 11:58 | |
| amorin | hey ianw fungi and other, on OVH BHS1, you have an instance which is ACTIVE but does not have any IP address | 11:58 |
| amorin | instance ID is : | 11:58 |
| amorin | 4bbfd576-baa1-410f-8384-95c7fac8475b | 11:58 |
| *** udesale has joined #openstack-infra | 11:59 | |
| amorin | can you check if it's needed? or maybe I can delete it? | 11:59 |
| *** goldyfruit has quit IRC | 12:00 | |
| *** yboaron_ has joined #openstack-infra | 12:01 | |
| *** yboaron_ has quit IRC | 12:07 | |
| *** yboaron_ has joined #openstack-infra | 12:07 | |
| *** tosky__ has joined #openstack-infra | 12:07 | |
| *** tosky is now known as Guest73721 | 12:07 | |
| *** tosky__ is now known as tosky | 12:07 | |
| *** jistr is now known as jistr|lunch | 12:08 | |
| *** yboaron_ has quit IRC | 12:09 | |
| *** yboaron_ has joined #openstack-infra | 12:10 | |
| *** Guest73721 has quit IRC | 12:11 | |
| *** bhavikdbavishi has quit IRC | 12:12 | |
| *** ginopc has quit IRC | 12:12 | |
| *** yboaron_ has quit IRC | 12:13 | |
| *** yboaron_ has joined #openstack-infra | 12:13 | |
| *** ginopc has joined #openstack-infra | 12:15 | |
| *** yboaron_ has quit IRC | 12:16 | |
| *** psachin has quit IRC | 12:16 | |
| *** yboaron_ has joined #openstack-infra | 12:16 | |
| *** jpena|lunch is now known as jpena | 12:17 | |
| *** yboaron_ has quit IRC | 12:19 | |
| *** psachin has joined #openstack-infra | 12:20 | |
| *** tdasilva has joined #openstack-infra | 12:34 | |
| *** rfolco_pto has quit IRC | 12:35 | |
| *** goldyfruit has joined #openstack-infra | 12:37 | |
| *** aedc has joined #openstack-infra | 12:39 | |
| *** haleyb has joined #openstack-infra | 12:41 | |
| *** rfolco has joined #openstack-infra | 12:42 | |
| *** jamesmcarthur has quit IRC | 12:42 | |
| *** aedc has quit IRC | 12:44 | |
| *** ekultails has joined #openstack-infra | 12:47 | |
| *** Goneri has joined #openstack-infra | 12:48 | |
| *** markvoelker has joined #openstack-infra | 12:50 | |
| *** yamamoto has quit IRC | 12:53 | |
| *** aaronsheffield has joined #openstack-infra | 12:55 | |
| *** jistr|lunch is now known as jistr | 12:55 | |
| *** udesale has quit IRC | 12:58 | |
| *** yamamoto has joined #openstack-infra | 12:59 | |
| *** yamamoto has quit IRC | 12:59 | |
| *** yamamoto has joined #openstack-infra | 13:00 | |
| *** pcaruana has quit IRC | 13:00 | |
| *** pcaruana has joined #openstack-infra | 13:00 | |
| *** aedc has joined #openstack-infra | 13:00 | |
| *** sthussey has joined #openstack-infra | 13:01 | |
| *** kaisers has quit IRC | 13:02 | |
| *** kaisers has joined #openstack-infra | 13:03 | |
| *** yamamoto has quit IRC | 13:04 | |
| iurygregory | fungi, ping, about I've pushed a separate patch to the CI job, but zuul sill complains, its necessary to use Depends-On? CI - https://review.opendev.org/#/c/665910/ ironic-prometheus-exporter https://review.opendev.org/#/c/665321/ | 13:18 |
| openstackgerrit | Frode Nordahl proposed openstack/project-config master: Add ``octavia-diskimage-retrofit`` charm https://review.opendev.org/666568 | 13:18 |
| iurygregory | good morning =) | 13:18 |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Extend event reporting https://review.opendev.org/662134 | 13:19 |
| *** sreejithp has joined #openstack-infra | 13:19 | |
| *** markvoelker has quit IRC | 13:20 | |
| *** jcoufal has joined #openstack-infra | 13:21 | |
| *** mriedem has joined #openstack-infra | 13:23 | |
| *** ykarel is now known as ykarel|afk | 13:24 | |
| *** spsurya has quit IRC | 13:29 | |
| *** jistr is now known as jistr|call | 13:31 | |
| *** iurygregory is now known as iurygregory_ | 13:36 | |
| fungi | if lpetrut comes back (or ykarel|afk are you involved too?) we can point out that the post merge jobs for https://review.opendev.org/597324 almost certainly didn't get run because of a zuul scheduler restart. that change merged 2019-06-14 14:37 utc and was probably waiting in the post pipeline at the time we took the scheduler offline (status log says we completed a full zuul restart by 14:51 utc | 13:38 |
| fungi | that day) | 13:38 |
| *** iurygregory has joined #openstack-infra | 13:38 | |
| *** rlandy|afk is now known as rlandy | 13:40 | |
| *** iurygregory_ has quit IRC | 13:40 | |
| fungi | amorin: that is a very weird server indeed. it's been sitting there since 04:53 utc today. we almost certainly don't need it but i'm going to check our nodepool logs real quick to see if that was associated with any sort of boot failure | 13:41 |
| ykarel|afk | fungi, ack, i will let lpetrut know if he comes online in my timezone | 13:42 |
| fungi | strange, it's node 0008103245 and nodepool reports it originally had ip addresses 158.69.77.216 and 2607:5300:201:2000::8f1 and is locked, in-use for nearly 9 hours | 13:44 |
| *** tosky has quit IRC | 13:50 | |
| fungi | iurygregory: 665910 will continue reporting an error until 665321 is merged, which is why they have to be separate changes. appending "depends-on: https://review.opendev.org/665321" to the commit message for 665910 is probably still a good idea just so people know 665321 needs to merge first, but zuul doesn't speculatively test adding a project to a tenant (or any changes to trusted configuration | 13:50 |
| fungi | projects like openstack/project-config for that matter) | 13:50 |
| *** jistr|call is now known as jistr | 13:50 | |
| iurygregory | fungi, got it o/ I will add the Depends-On | 13:51 |
| fungi | yeah, it won't make the syntax error go away, but at least it will be more obvious what the dependency for making it stop is | 13:51 |
| fungi | okay, so zuul node request 200-0003967737 was fulfilled by node 0008103245 at 04:53:52z today according to the scheduler log | 13:54 |
| openstackgerrit | Iury Gregory Melo Ferreira proposed openstack/project-config master: Adding CI for ironic-prometheus-exporter https://review.opendev.org/665910 | 13:54 |
| fungi | for job swift-upload-image of queueitem 0x7fa6671cacc0 change 0x7fa6a08caf60 openstack/swift 665487,2 in the gate pipeline | 13:55 |
| *** yamamoto has joined #openstack-infra | 13:56 | |
| *** whoami-rajat has quit IRC | 13:56 | |
| *** rfolco has quit IRC | 13:58 | |
| openstackgerrit | Merged zuul/nodepool master: Pin to openshift <= 0.8.9 https://review.opendev.org/666526 | 13:59 |
| fungi | the job was accepted at build 395c781799ea452c8f639d3037f8de0f but then a few minutes later the scheduler logged this: | 13:59 |
| fungi | 2019-06-20 05:02:00,474 WARNING zuul.Scheduler: [e: 9b44cc146cd649e585ff229c0a8a296b] Build <Build 395c781799ea452c8f639d3037f8de0f of swift-upload-image voting:False on <Worker ze01.openstack.org>> is not in the current build set | 13:59 |
| fungi | i'm not sure if that's normal, seems to just be a warning | 13:59 |
| fungi | that's the last the scheduler mentions that build uuid, so i'll continue through the ze01 logs | 14:00 |
| fungi | aha! | 14:02 |
| fungi | 2019-06-20 05:02:00,472 INFO zuul.AnsibleJob: [e: 9b44cc146cd649e585ff229c0a8a296b] [build: 395c781799ea452c8f639d3037f8de0f] Pausing job swift-upload-image for ref refs/changes/87/665487/2 (change https://review.opendev.org/665487) | 14:02 |
| *** yamamoto has quit IRC | 14:03 | |
| fungi | so zuul paused the job and then... | 14:04 |
| fungi | no log entries about that build thereafter (9 hours later) | 14:04 |
| fungi | the instance is no longer reachable at the ip addresses zuul/nodepool originally knew for it, so perhaps it suffered some trauma while paused? | 14:05 |
| fungi | nothing logged at http://zuul.opendev.org/t/openstack/builds?uuid=395c781799ea452c8f639d3037f8de0f (yet) but i don't see any paused job for 665487,2 in the zuul status either | 14:07 |
| *** aedc has quit IRC | 14:08 | |
| *** jcoufal has quit IRC | 14:09 | |
| *** dave-mccowan has joined #openstack-infra | 14:09 | |
| fungi | https://review.opendev.org/665487 seems to have completed its gate testing at 07:31 and swift-upload-image reported success, logs at http://logs.openstack.org/87/665487/2/gate/swift-upload-image/09daedb/ | 14:11 |
| fungi | the build uuid reported in the inventory for it is 09daedbb33224f18ba53d374a2bbfa1b so not even the same... what am i missing? | 14:13 |
| fungi | oh! i bet there was a gate reset involved | 14:13 |
| fungi | and so the build which eventually completed was not the same as the build for which node 0008103245 was allocated | 14:14 |
| *** jcoufal has joined #openstack-infra | 14:15 | |
| openstackgerrit | Mark Meyer proposed zuul/zuul master: Extend event reporting https://review.opendev.org/662134 | 14:15 |
| fungi | yeah, logs bear that out. first timestamp in the console log is at 07:01:49, two hours later | 14:15 |
| fungi | `openstack console log show ...` for that instance is unremarkable. shows boot-time services starting and then "ubuntu-bionic-ovh-bhs1-0008103245 login:" | 14:17 |
| fungi | the interactive novnc console for it is responsive too, a carriage return at that login prompt gets me a fresh login prompt | 14:18 |
| fungi | i think i'm about at the end of my options for collecting anything useful since we don't have any passworded accounts for these nodes | 14:19 |
| fungi | i'm going to try a soft reboot through novnc | 14:19 |
| fungi | it rebooted cleanly, but still isn't reachable over the internet on its original addresses | 14:20 |
| fungi | Shrews: if you're around, is there anything we need to clean up for the lock in nodepool? | 14:21 |
| fungi | or do we need to somehow instruct zuul to unlock the node? | 14:22 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Add missing start-message in pipeline config schema https://review.opendev.org/665936 | 14:22 |
| *** jamesmcarthur has joined #openstack-infra | 14:22 | |
| fungi | ignoring the strange case of the missing ip addresses for a moment, i wonder if we have a hole in the scheduler where a build which is in a paused state when the change is requeued, for example in a "gate reset," can cause zuul to forget to release its lock | 14:24 |
| openstackgerrit | Merged openstack/project-config master: Reduce OVH BHS1 pool size to 120 https://review.opendev.org/666518 | 14:25 |
| fungi | also possible, i suppose, that the disappearing ip addresses are related, and caused zuul to spontaneously requeue the build because it lost contact with that node? | 14:26 |
| fungi | though in that case i'd have expected the executor to log something about that | 14:27 |
| fungi | and it doesn't seem to have done (or at least i'm not searching for the right needle in that haystack) | 14:27 |
| *** priteau has joined #openstack-infra | 14:28 | |
| *** ykarel|afk is now known as ykarel|away | 14:29 | |
| *** ykarel|away has quit IRC | 14:34 | |
| *** pgaxatte has quit IRC | 14:36 | |
| *** pgaxatte has joined #openstack-infra | 14:37 | |
| *** pgaxatte has quit IRC | 14:37 | |
| roman_g | Hello team. What would be a good name for a mailing list with CICD problems (e.g. failed Zuul post-merge, or read the docs docs build failure, etc.)? Should be mostly usefull for core reviewers who support CI for our projects. | 14:43 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Add missing doc for pipeline start-message https://review.opendev.org/665930 | 14:43 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Add support for item.change for pipeline start-message formater https://review.opendev.org/665968 | 14:43 |
| openstackgerrit | Fabien Boucher proposed zuul/zuul master: Add change replacement field in doc for start-message https://review.opendev.org/665974 | 14:43 |
| *** zbr is now known as zbr|ruck | 14:44 | |
| *** rajinir has joined #openstack-infra | 14:45 | |
| *** michael-beaver has joined #openstack-infra | 14:46 | |
| fungi | roman_g: i'm probably misunderstanding, but you want to create a mailing list to discuss ci/cd challenges? | 14:47 |
| Shrews | fungi: sorry, afk today and the next couple of weeks. I’m sure corvus can assist | 14:47 |
| fungi | Shrews: no worries, i couldn't remember when your thing was starting. good luck! | 14:47 |
| roman_g | fungi: no. To notify developers that there are problems with project's cicd. | 14:49 |
| fungi | roman_g: like automated reports or discussion? | 14:52 |
| fungi | or people sending announcements? | 14:52 |
| roman_g | fungi: only automated reports, no discussion | 14:52 |
| roman_g | *failure reports | 14:53 |
| *** tosky__ has joined #openstack-infra | 14:54 | |
| *** tosky__ is now known as tosky | 14:54 | |
| fungi | roman_g: got it. openstack does something similar (though for a subset of jobs related to release automation). we called that one release-job-failures@ | 14:55 |
| fungi | people subscribe to it and the release pipelines are configured to send failure reports to that address | 14:56 |
| fungi | and that has worked fairly well so far | 14:56 |
| roman_g | fungi: thank you! | 14:59 |
| openstackgerrit | Paul Belanger proposed zuul/nodepool master: Add release note about pinning openshift client https://review.opendev.org/666605 | 14:59 |
| fungi | #status log manually deleted instance 4bbfd576-baa1-410f-8384-95c7fac8475b in ovh bhs1; it has a stale node lock from zuul which will be released at the next scheduler restart (for some reason it has lost its ip addresses too, no idea if that's related) | 15:00 |
| *** lseki has joined #openstack-infra | 15:00 | |
| openstackstatus | fungi: finished logging | 15:00 |
| *** factor has joined #openstack-infra | 15:04 | |
| *** factor__ has quit IRC | 15:05 | |
| openstackgerrit | Roman Gorshunov proposed opendev/system-config master: Mailing list for Airship https://review.opendev.org/666606 | 15:07 |
| *** factor has quit IRC | 15:08 | |
| *** factor has joined #openstack-infra | 15:08 | |
| roman_g | ^^ - Quadruple change number | 15:09 |
| roman_g | Today there should be Sextuple change number | 15:09 |
| openstackgerrit | Merged zuul/zuul-jobs master: Dockerhub now returns 200 for DELETEs https://review.opendev.org/666529 | 15:09 |
| *** pcaruana has quit IRC | 15:14 | |
| *** dklyle has joined #openstack-infra | 15:16 | |
| *** priteau has quit IRC | 15:16 | |
| *** yboaron_ has joined #openstack-infra | 15:17 | |
| *** priteau has joined #openstack-infra | 15:18 | |
| dmsimard | corvus, pabelanger, clarkb, fungi: about that python version error when uploading logs in post-run from the other day -- I took the ansible_python_interpreter variable out of the job and it works now. | 15:23 |
| dmsimard | I'm confused a little bit because the fedora jobs aren't failing with ansible 2.7 which is supposed to fail when there's no /usr/bin/python | 15:23 |
| dmsimard | I don't have a good explanation as to why it suddenly started failing | 15:24 |
| pabelanger | dmsimard: I was thinking more about it, I wonder if it is a result of our changes to ansible_python_interpreter in zuul: https://review.opendev.org/637338/ | 15:26 |
| pabelanger | that's the only thing I can think of that recently changed | 15:26 |
| *** kopecmartin is now known as kopecmartin|off | 15:26 | |
| pabelanger | it would be good to figure out the last time the job worked, when it was defined | 15:26 |
| *** ginux has joined #openstack-infra | 15:32 | |
| *** ginux is now known as Guest67405 | 15:32 | |
| *** ginopc has quit IRC | 15:34 | |
| *** Guest67405 is now known as ginopc | 15:34 | |
| *** jamesmcarthur has quit IRC | 15:36 | |
| *** e0ne has quit IRC | 15:38 | |
| *** ginopc has quit IRC | 15:38 | |
| *** electrofelix has quit IRC | 15:44 | |
| *** gyee has joined #openstack-infra | 15:45 | |
| *** ykarel|away has joined #openstack-infra | 15:51 | |
| *** ykarel|away has quit IRC | 15:52 | |
| *** jamesmcarthur has joined #openstack-infra | 15:53 | |
| *** georgk9 has joined #openstack-infra | 15:55 | |
| *** MarkMaglana_ has joined #openstack-infra | 15:55 | |
| *** ginopc has joined #openstack-infra | 15:55 | |
| *** masayukig_ has joined #openstack-infra | 15:55 | |
| *** jcoufal has quit IRC | 15:55 | |
| *** jamesmcarthur has quit IRC | 15:56 | |
| *** jamesmcarthur has joined #openstack-infra | 15:56 | |
| *** persia_ has joined #openstack-infra | 15:57 | |
| *** rpittau is now known as rpittau|afk | 15:58 | |
| *** sean-k-mooney1 has joined #openstack-infra | 15:58 | |
| *** mattw4 has joined #openstack-infra | 15:58 | |
| *** pkopec has joined #openstack-infra | 16:00 | |
| *** lucasagomes has quit IRC | 16:01 | |
| *** jpich has quit IRC | 16:02 | |
| *** pcaruana has joined #openstack-infra | 16:02 | |
| *** masayukig has quit IRC | 16:03 | |
| *** MarkMaglana has quit IRC | 16:03 | |
| *** georgk has quit IRC | 16:03 | |
| *** szaher has quit IRC | 16:03 | |
| *** verdurin has quit IRC | 16:03 | |
| *** persia has quit IRC | 16:03 | |
| *** sean-k-mooney has quit IRC | 16:03 | |
| *** MarkMaglana_ is now known as MarkMaglana | 16:03 | |
| *** masayukig_ is now known as masayukig | 16:03 | |
| *** szaher has joined #openstack-infra | 16:09 | |
| *** verdurin has joined #openstack-infra | 16:10 | |
| *** iurygregory has quit IRC | 16:10 | |
| *** lyarwood has quit IRC | 16:18 | |
| *** smarcet has joined #openstack-infra | 16:22 | |
| *** ccamacho has quit IRC | 16:25 | |
| logan- | i am disabling the nodepool host aggregate on limestone for a few hours to do hypervisor reboots for kernel updates. will wait for nodes to drain off before rebooting anything. | 16:25 |
| *** factor has quit IRC | 16:26 | |
| fungi | thanks for the heads up, logan-! | 16:26 |
| *** tdasilva has quit IRC | 16:29 | |
| *** tdasilva has joined #openstack-infra | 16:29 | |
| *** aedc has joined #openstack-infra | 16:34 | |
| *** panda has quit IRC | 16:50 | |
| *** panda has joined #openstack-infra | 16:52 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Defer setting build result to event queue https://review.opendev.org/666643 | 16:55 |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: WIP: new devstack-based functional job https://review.opendev.org/665023 | 16:56 |
| *** derekh has quit IRC | 16:58 | |
| openstackgerrit | Roman Gorshunov proposed opendev/system-config master: Mailing list for Airship https://review.opendev.org/666606 | 16:59 |
| *** factor has joined #openstack-infra | 17:02 | |
| openstackgerrit | Merged zuul/nodepool master: Add release note about pinning openshift client https://review.opendev.org/666605 | 17:03 |
| *** ginopc has quit IRC | 17:03 | |
| *** aedc has quit IRC | 17:05 | |
| *** aedc has joined #openstack-infra | 17:05 | |
| *** pkopec has quit IRC | 17:06 | |
| *** weifan has joined #openstack-infra | 17:06 | |
| *** tdasilva has quit IRC | 17:08 | |
| *** tdasilva has joined #openstack-infra | 17:08 | |
| *** whoami-rajat has joined #openstack-infra | 17:09 | |
| *** ramishra has quit IRC | 17:13 | |
| *** tesseract has quit IRC | 17:14 | |
| *** tesseract has joined #openstack-infra | 17:15 | |
| *** jpena is now known as jpena|off | 17:15 | |
| openstackgerrit | Michael Johnson proposed openstack/diskimage-builder master: Add DIB_UBUNTU_KERNEL to ubuntu-minimal https://review.opendev.org/666063 | 17:18 |
| auristor | ianw: afs-next has been updated with additional bug fixes | 17:25 |
| *** smarcet has quit IRC | 17:26 | |
| *** ricolin has quit IRC | 17:32 | |
| *** ociuhandu has quit IRC | 17:34 | |
| *** ralonsoh has quit IRC | 17:41 | |
| *** tdasilva has quit IRC | 17:45 | |
| *** priteau has quit IRC | 17:46 | |
| *** lseki has quit IRC | 17:50 | |
| cloudnull | hey all, | 17:50 |
| cloudnull | what all do we need to do to get a project into storyboard? | 17:50 |
| cloudnull | specifically, the tripleo-ansible project is wanting to use storyboard | 17:51 |
| openstackgerrit | Kevin Carter (cloudnull) proposed openstack/project-config master: Set the tripleo-ansible project to use storyboard https://review.opendev.org/666655 | 17:51 |
| cloudnull | ^ is that it ? | 17:51 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Differentiate between queued and waiting jobs in zuul web UI https://review.opendev.org/660878 | 17:51 |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Differentiate between queued and waiting jobs in zuul web UI https://review.opendev.org/660878 | 17:52 |
| fungi | cloudnull: any import of existing bugs from launchpad is a manual process, but i'm happy to test it out and then import them in production on whatever schedule the project desires | 17:59 |
| *** tesseract has quit IRC | 17:59 | |
| *** tesseract has joined #openstack-infra | 18:01 | |
| *** jamesmcarthur has quit IRC | 18:06 | |
| *** cburgess has joined #openstack-infra | 18:07 | |
| fungi | gotta pop out for groceries real quick to avoid tourists, brb | 18:07 |
| *** jamesmcarthur has joined #openstack-infra | 18:07 | |
| *** cburgess has quit IRC | 18:07 | |
| *** smarcet has joined #openstack-infra | 18:10 | |
| *** jamesmcarthur has quit IRC | 18:12 | |
| *** jamesmcarthur has joined #openstack-infra | 18:12 | |
| *** spotz has quit IRC | 18:14 | |
| *** mudpuppy has quit IRC | 18:14 | |
| cloudnull | fungi we dont have anything we need to import | 18:14 |
| *** diablo_rojo has joined #openstack-infra | 18:15 | |
| *** jamesmcarthur_ has joined #openstack-infra | 18:17 | |
| *** jamesmcarthur has quit IRC | 18:17 | |
| *** jamesmcarthur_ has quit IRC | 18:22 | |
| *** jamesmcarthur has joined #openstack-infra | 18:23 | |
| *** igordc has joined #openstack-infra | 18:27 | |
| *** igordc has quit IRC | 18:36 | |
| *** spotz has joined #openstack-infra | 18:37 | |
| *** pkopec has joined #openstack-infra | 18:37 | |
| *** tesseract has quit IRC | 18:37 | |
| *** tesseract has joined #openstack-infra | 18:38 | |
| openstackgerrit | Merged zuul/zuul master: Update cached repo during job startup only if needed https://review.opendev.org/648229 | 18:39 |
| *** diablo_rojo has quit IRC | 18:40 | |
| *** jamesmcarthur has quit IRC | 18:42 | |
| *** diablo_rojo has joined #openstack-infra | 18:45 | |
| *** jamesmcarthur has joined #openstack-infra | 18:51 | |
| *** diablo_rojo has quit IRC | 18:55 | |
| *** armstrong has joined #openstack-infra | 18:55 | |
| fungi | cloudnull: in that case sure, forge right on ahead! | 19:00 |
| fungi | that change will autocreate the project and add it in the specified project group when it merges | 19:00 |
| *** jamesmcarthur has quit IRC | 19:01 | |
| *** diablo_rojo has joined #openstack-infra | 19:04 | |
| cloudnull | awesome | 19:08 |
| cloudnull | thanks! | 19:09 |
| *** psachin has quit IRC | 19:11 | |
| *** jamesmcarthur has joined #openstack-infra | 19:14 | |
| *** weifan has quit IRC | 19:17 | |
| *** weifan has joined #openstack-infra | 19:17 | |
| *** weifan has quit IRC | 19:18 | |
| *** weifan has joined #openstack-infra | 19:18 | |
| *** weifan has quit IRC | 19:19 | |
| *** weifan has joined #openstack-infra | 19:19 | |
| *** weifan has quit IRC | 19:19 | |
| *** weifan has joined #openstack-infra | 19:20 | |
| *** weifan has quit IRC | 19:20 | |
| *** weifan has joined #openstack-infra | 19:20 | |
| *** markvoelker has joined #openstack-infra | 19:20 | |
| *** weifan has quit IRC | 19:21 | |
| *** weifan has joined #openstack-infra | 19:21 | |
| *** jamesmcarthur has quit IRC | 19:22 | |
| *** weifan has quit IRC | 19:22 | |
| *** weifan has joined #openstack-infra | 19:22 | |
| *** weifan has quit IRC | 19:23 | |
| *** weifan has joined #openstack-infra | 19:23 | |
| *** weifan has quit IRC | 19:23 | |
| *** weifan has joined #openstack-infra | 19:24 | |
| *** weifan has quit IRC | 19:24 | |
| *** weifan has joined #openstack-infra | 19:24 | |
| *** weifan has quit IRC | 19:25 | |
| *** weifan has joined #openstack-infra | 19:25 | |
| *** weifan has quit IRC | 19:26 | |
| *** goldyfruit has quit IRC | 19:26 | |
| *** weifan has joined #openstack-infra | 19:26 | |
| *** weifan has quit IRC | 19:27 | |
| *** weifan has joined #openstack-infra | 19:27 | |
| *** weifan has quit IRC | 19:27 | |
| *** weifan has joined #openstack-infra | 19:28 | |
| *** weifan has quit IRC | 19:28 | |
| *** weifan has joined #openstack-infra | 19:28 | |
| *** weifan has quit IRC | 19:29 | |
| *** weifan has joined #openstack-infra | 19:29 | |
| *** weifan has quit IRC | 19:30 | |
| *** weifan has joined #openstack-infra | 19:30 | |
| *** weifan has quit IRC | 19:31 | |
| *** weifan has joined #openstack-infra | 19:31 | |
| *** weifan has quit IRC | 19:31 | |
| *** weifan has joined #openstack-infra | 19:32 | |
| *** weifan has quit IRC | 19:32 | |
| *** weifan has joined #openstack-infra | 19:32 | |
| *** weifan has quit IRC | 19:33 | |
| *** weifan has joined #openstack-infra | 19:33 | |
| openstackgerrit | Tobias Henkel proposed zuul/zuul master: Filter out unprotected branches from builds if excluded https://review.opendev.org/666664 | 19:34 |
| *** weifan has quit IRC | 19:38 | |
| *** whoami-rajat has quit IRC | 19:39 | |
| *** markvoelker has quit IRC | 19:40 | |
| *** goldyfruit has joined #openstack-infra | 19:44 | |
| *** bhavikdbavishi has joined #openstack-infra | 19:44 | |
| *** yamamoto has joined #openstack-infra | 19:47 | |
| *** yamamoto has quit IRC | 19:51 | |
| logan- | reenabling the nodepool host aggregate on limestone | 19:54 |
| logan- | mirror appears to be up and running fine after the hv reboot | 19:54 |
| fungi | thanks again logan-! | 19:59 |
| logan- | yep np | 19:59 |
| *** weifan has joined #openstack-infra | 20:02 | |
| cloudnull | fungi any change you, or pabelanger, have had time to think on https://review.opendev.org/#/c/666423 ? | 20:03 |
| cloudnull | o/ logan- | 20:03 |
| fungi | cloudnull: i bet we also need a firewall rule to open that new port, just a sec i'll get you a link | 20:05 |
| cloudnull | much appreciated :) | 20:05 |
| logan- | o/ cloudnull | 20:06 |
| fungi | oh, and i think apache needs a listen directive too | 20:06 |
| fungi | cloudnull: see review comment | 20:09 |
| cloudnull | thanks fungi | 20:09 |
| fungi | the configuration is unfortunately doubled temporarily as we're in the process of replacing our puppet+erb config management for the mirrors with ansible+j2 | 20:12 |
| *** bhavikdbavishi has quit IRC | 20:15 | |
| *** e0ne has joined #openstack-infra | 20:26 | |
| *** e0ne has quit IRC | 20:33 | |
| *** pcaruana has quit IRC | 20:45 | |
| *** eharney has quit IRC | 20:45 | |
| *** markmcclain has quit IRC | 20:50 | |
| *** markmcclain has joined #openstack-infra | 20:55 | |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: WIP: new devstack-based functional job https://review.opendev.org/665023 | 20:56 |
| *** owalsh has quit IRC | 21:00 | |
| *** yboaron_ has quit IRC | 21:04 | |
| *** kjackal has quit IRC | 21:04 | |
| *** kjackal has joined #openstack-infra | 21:05 | |
| *** jtomasek has quit IRC | 21:10 | |
| *** aedc has quit IRC | 21:11 | |
| openstackgerrit | James E. Blair proposed zuul/zuul master: WIP: Allow config projects to override allowed-projects https://review.opendev.org/666733 | 21:17 |
| *** pkopec has quit IRC | 21:24 | |
| *** ekultails has quit IRC | 21:30 | |
| openstackgerrit | Colleen Murphy proposed opendev/puppet-askbot master: Add rspec test https://review.opendev.org/558991 | 21:34 |
| openstackgerrit | Colleen Murphy proposed opendev/puppet-askbot master: Add idempotentcy rspec test https://review.opendev.org/572553 | 21:34 |
| openstackgerrit | Colleen Murphy proposed opendev/puppet-askbot master: Add curl test to beaker tests https://review.opendev.org/585196 | 21:34 |
| *** gyee has quit IRC | 21:34 | |
| *** mriedem is now known as mriedem_awau | 21:36 | |
| *** mriedem_awau is now known as mriedem_away | 21:36 | |
| *** weifan has quit IRC | 21:37 | |
| *** weifan has joined #openstack-infra | 21:37 | |
| *** markvoelker has joined #openstack-infra | 21:37 | |
| *** weifan has quit IRC | 21:38 | |
| *** weifan has joined #openstack-infra | 21:38 | |
| *** weifan has quit IRC | 21:39 | |
| openstackgerrit | Kendall Nelson proposed opendev/storyboard-webclient master: Converting a search query into a browse https://review.opendev.org/644424 | 21:39 |
| *** weifan has joined #openstack-infra | 21:39 | |
| *** weifan has quit IRC | 21:39 | |
| *** weifan has joined #openstack-infra | 21:39 | |
| *** weifan has quit IRC | 21:40 | |
| *** weifan has joined #openstack-infra | 21:40 | |
| *** weifan has quit IRC | 21:41 | |
| *** weifan has joined #openstack-infra | 21:41 | |
| *** weifan has quit IRC | 21:42 | |
| *** markvoelker has quit IRC | 21:42 | |
| *** weifan has joined #openstack-infra | 21:42 | |
| *** armstrong has quit IRC | 21:44 | |
| corvus | fungi, ianw: i'm watching a very slow centos7 dib build happening on a node in vexxhost | 21:45 |
| corvus | urlgrabbe 28313 root 4w REG 252,1 44773376 4731788 /opt/dib/tmp/dib_build.58lKoSAv/mnt/tmp/yum/x86_64/7/base/packages/linux-firmware-20180911-69.git85c5d90.el7.noarch.rpm | 21:45 |
| corvus | urlgrabbe 28313 root 5u IPv6 238696 0t0 TCP ubuntu-bionic-vexxhost-sjc1-0008158648:37918->[2606:4700:30::681c:577]:http (ESTABLISHED) | 21:45 |
| corvus | is that something that should be hitting our mirror? | 21:45 |
| ianw | probably ... is that using the mirror element to set things up in the chroot? | 21:46 |
| *** weifan has quit IRC | 21:47 | |
| corvus | ianw: i think so, but i could have missed something | 21:47 |
| corvus | it's at 38.108.68.247 if you want to log in | 21:47 |
| corvus | ianw: oh, i think i see it! | 21:48 |
| corvus | DIB_DISTRIBUTION_MIRROR_CENTOS: http://mirror.sjc1.vexxhost.openstack.org/centos | 21:48 |
| corvus | that should just be "DIB_DISTRIBUTION_MIRROR" i think | 21:48 |
| ianw | ohh, umm i'll have to context switch it back in, but maybe not. it's because of all the magic that needs to happen to get the mirror into the building chroot, but then not leave it behind in the final image | 21:49 |
| openstackgerrit | Kendall Nelson proposed opendev/storyboard master: Build containers using pbrx https://review.opendev.org/611191 | 21:49 |
| *** tosky has quit IRC | 21:49 | |
| corvus | ianw: it look like the centos7 element honors that env var, if i set it right | 21:50 |
| *** weifan has joined #openstack-infra | 21:51 | |
| ianw | centos7 or centos-minimal? | 21:51 |
| corvus | ianw: hrm, centos7 | 21:51 |
| corvus | which you're right, is not the one we use in the job | 21:52 |
| corvus | ianw: is it possible that we're ignoring the centos mirror in the current job? | 21:52 |
| ianw | i'm not defending any of this ... it has a long history, and the original code trying to have a single flag to do everything across every distro which never worked, which ended up with layers on layers ... organic growth as they say | 21:52 |
| ianw | for dib in the gate, we write out .repo files using the https://opendev.org/openstack/diskimage-builder/src/branch/master/roles/dib-setup-gate-mirrors role first | 21:53 |
| corvus | ianw: interesting, i don't see that in the nodepool job | 21:54 |
| *** sreejithp has quit IRC | 21:54 | |
| *** sreejithp has joined #openstack-infra | 21:55 | |
| ianw | then we use https://opendev.org/openstack/diskimage-builder/src/branch/master/diskimage_builder/elements/openstack-ci-mirrors/environment.d/11-dib-distribution-mirror.bash#L45 | 21:55 |
| *** sshnaidm is now known as sshnaidm|off | 21:55 | |
| openstackgerrit | Kendall Nelson proposed opendev/storyboard master: added field nickname into users table https://review.opendev.org/646036 | 21:55 |
| ianw | ok, starting to remember ... the role created .repo files and ^ are used *outside* the chroot; in the initial build parts | 21:56 |
| *** slaweq has quit IRC | 21:57 | |
| *** smarcet has quit IRC | 21:58 | |
| ianw | right, then setting DIB_DISTRIBUTION_MIRROR should work *inside* the chroot -- e.g. https://opendev.org/openstack/project-config/src/branch/master/nodepool/nodepool.yaml | 21:59 |
| ianw | if centos7 (not minimal) isn't working with that, it's probably a bug, but that element doesn't get as much attention as the minimal one | 22:00 |
| corvus | well, grepping for DIB_DISTRIBUTION_MIRROR in diskimage builder shows that the centos7 element uses it, but not centos-minimal or yum-minimal, so i'm not sure how it would get used. it looks like it would get used for the debuntu builds | 22:01 |
| *** david-lyle has joined #openstack-infra | 22:01 | |
| *** dklyle has quit IRC | 22:01 | |
| corvus | \o/ the job worked -- it just took 55m | 22:01 |
| *** david-lyle is now known as dklyle | 22:01 | |
| corvus | so i've got a version of the nodepool-devstack job that doesn't rely on being a devstack plugin or being parented to the devstack job | 22:02 |
| corvus | ianw: i think we can generalize this so that the dib project can use this as well -- i'm guessing dib is still relying on the nodepool jobs to do the "can we make a bootable image of each os" test? | 22:03 |
| openstackgerrit | Merged opendev/storyboard-webclient master: Fix project group view checkboxes https://review.opendev.org/640590 | 22:04 |
| openstackgerrit | Merged opendev/storyboard-webclient master: Add activity indicator to new worklist modal and detail https://review.opendev.org/640845 | 22:04 |
| *** owalsh has joined #openstack-infra | 22:05 | |
| *** diablo_rojo has quit IRC | 22:06 | |
| ianw | corvus: yes, that's the only booting test we use. i can see the point that it has too many things going on at once (testing nodepool, dib, openstacksdk, let alone the rest of the stack) | 22:06 |
| *** kjackal has quit IRC | 22:08 | |
| corvus | ianw: here's what i'm thinking -- parameterize the job in https://review.opendev.org/665023 so that we can pass in info for different distros. this way nodepool can define the job and only a single image that it uses, but then in the dib repo, we can inherit from that job and make new jobs for each distro | 22:08 |
| corvus | ianw: so it'd mostly work the way things are now, but we've moved all the distro-specific stuff to the dib repo. | 22:09 |
| corvus | we'll move the nodepool repo to the zuul tenant, but we can still have dib and nodepool in both tenants so we can do the job inheritance and depends-on cross tests | 22:09 |
| ianw | ok, back to the mirroring, i did leave breadcrumbs but in yum-minimal ... https://opendev.org/openstack/diskimage-builder/src/branch/master/diskimage_builder/elements/yum-minimal/README.rst | 22:09 |
| ianw | "If you wish to build from specific mirrors, set DIB_YUM_MINIMAL_BOOTSTRAP_REPOS to a directory with the .repo files to use during bootstrap and build. The repo files should be named with a prefix dib-mirror- and will be removed from the final image." | 22:10 |
| ianw | then this role sets that if they are there. in the dib jobs, the repo files are created by that role i mentioned -> https://opendev.org/openstack/diskimage-builder/src/branch/master/diskimage_builder/elements/openstack-ci-mirrors/environment.d/11-dib-distribution-mirror.bash#L45 | 22:12 |
| ianw | i mean this *element* ... the openstack-ci-mirrors element | 22:12 |
| corvus | but i don't think any of that happens in the nodepool jobs, the only thing i see is DIB_DISTRIBUTION_MIRROR env var | 22:13 |
| *** slaweq has joined #openstack-infra | 22:14 | |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: WIP: new devstack-based functional job https://review.opendev.org/665023 | 22:15 |
| ianw | yeah, it's very possible that bit got missed | 22:15 |
| ianw | corvus: oh, ok i see, when you said doesn't depend on being a devstack plugin i thought you were going for a static node or something. but i see you mean run ./stack.sh then do stuff after. | 22:16 |
| ianw | that sounds good, yes i agree there's too much boilerplate updating with the current job | 22:16 |
| ianw | because you can't use any interitance etc in the devstack plugin bit, so we're copying line after line of environment var setup | 22:17 |
| *** slaweq has quit IRC | 22:18 | |
| *** mriedem_away has quit IRC | 22:19 | |
| corvus | ianw: yeah, my immediate goal is to move the nodepool project to the zuul tenant, but if we inherit from the devstack job, we have to include that and a bunch of openstack repos too, which just seems excessive (doable, but it doesn't reflect reality). so this turns devstack into a bit of a black box. later on, we might be able to replace our use of devstack with some other way of getting "an | 22:19 |
| corvus | openstack" -- perhaps using some work that hogepodge is doing with containers. that way we might be able to get our openstack up and running faster. | 22:19 |
| ianw | yeah, that sounds fair. i've always thought it is a pretty valid nodepool test -- i mean it's a complete end-to-end "can it build, upload and boot an image". but extracting and modularising more like this is good | 22:22 |
| ianw | we keep that end-to-end test and have more flexibility going forward | 22:23 |
| corvus | yep, i don't want to lose any test coverage :) | 22:23 |
| openstackgerrit | Kendall Nelson proposed opendev/storyboard master: Add is_deprecated field for tags https://review.opendev.org/535586 | 22:32 |
| openstackgerrit | Kendall Nelson proposed opendev/storyboard master: added field nickname into users table https://review.opendev.org/646036 | 22:35 |
| *** tesseract has quit IRC | 22:35 | |
| *** markvoelker has joined #openstack-infra | 22:38 | |
| *** owalsh has quit IRC | 22:39 | |
| *** owalsh has joined #openstack-infra | 22:39 | |
| *** markvoelker has quit IRC | 22:43 | |
| *** mattw4 has quit IRC | 22:47 | |
| *** sean-k-mooney1 has quit IRC | 22:49 | |
| ianw | in typical heisenbug fashion, now i setup all the remote netconsole etc the iad mirror seems to be going ok | 22:49 |
| *** diablo_rojo has joined #openstack-infra | 22:50 | |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: WIP: new devstack-based functional job https://review.opendev.org/665023 | 22:53 |
| corvus | ianw: ^ this ps has a second job with a bit of parameterization -- i'm thinking the "-openstack" job would live in the nodepool repo, and the "-openstack-centos-7" and other jobs like it would inherit from it and live in the dib repo | 22:54 |
| corvus | oops it also has a syntax error | 22:54 |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: WIP: new devstack-based functional job https://review.opendev.org/665023 | 22:54 |
| ianw | auristor: ok, looks like a couple of things that certainly could stop a machine fixed in afs-next, spinlock init and the union thing in particular | 22:55 |
| openstackgerrit | James E. Blair proposed zuul/nodepool master: WIP: new devstack-based functional job https://review.opendev.org/665023 | 22:55 |
| *** auristor has quit IRC | 22:56 | |
| ianw | doesn't have to be immediate, but one nice thing atm is that we run all the rpms distros in one test, helps amortize the cost (and risk of failure :) of the devstack run | 22:57 |
| ianw | would just be some more magic in the templates to loop i guess | 22:57 |
| corvus | ah yeah -- using one image per job makes it simple, but we can add a bit more to handle 2 | 22:58 |
| *** dklyle has quit IRC | 23:02 | |
| *** mattw4 has joined #openstack-infra | 23:02 | |
| *** aaronsheffield has quit IRC | 23:05 | |
| *** rajinir has quit IRC | 23:09 | |
| *** rcernin has joined #openstack-infra | 23:11 | |
| *** dchen has joined #openstack-infra | 23:11 | |
| openstackgerrit | Kendall Nelson proposed opendev/storyboard master: added field nickname into users table https://review.opendev.org/646036 | 23:14 |
| openstackgerrit | Kendall Nelson proposed opendev/storyboard master: Add is_deprecated field for tags https://review.opendev.org/535586 | 23:16 |
| *** tkajinam has joined #openstack-infra | 23:21 | |
| *** auristor has joined #openstack-infra | 23:29 | |
| *** goldyfruit has quit IRC | 23:31 | |
| *** owalsh has quit IRC | 23:32 | |
| *** dchen has quit IRC | 23:35 | |
| *** dchen has joined #openstack-infra | 23:36 | |
| *** diablo_rojo has quit IRC | 23:37 | |
| *** markvoelker has joined #openstack-infra | 23:39 | |
| *** sthussey has quit IRC | 23:40 | |
| *** markvoelker has quit IRC | 23:43 | |
| *** rlandy has quit IRC | 23:44 | |
| *** goldyfruit has joined #openstack-infra | 23:44 | |
| *** yamamoto has joined #openstack-infra | 23:46 | |
| *** owalsh has joined #openstack-infra | 23:49 | |
| *** auristor has quit IRC | 23:55 | |
| *** weifan has quit IRC | 23:56 | |
| *** weifan has joined #openstack-infra | 23:56 | |
| *** auristor has joined #openstack-infra | 23:57 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!