| *** slaweq has joined #openstack-infra | 00:13 | |
| *** bobh has joined #openstack-infra | 00:16 | |
| *** dklyle has joined #openstack-infra | 00:16 | |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder master: fix systemd import-tar for gentoo https://review.openstack.org/633425 | 00:19 |
|---|---|---|
| *** slaweq has quit IRC | 00:23 | |
| *** dklyle has quit IRC | 00:31 | |
| *** e0ne has quit IRC | 00:32 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: upgrade react-scripts to ^2.0.0 https://review.openstack.org/631902 | 00:35 |
| *** mgoddard has quit IRC | 00:42 | |
| *** mgoddard has joined #openstack-infra | 00:48 | |
| *** whoami-rajat has joined #openstack-infra | 00:49 | |
| *** hwoarang has quit IRC | 00:49 | |
| *** hwoarang has joined #openstack-infra | 00:50 | |
| *** wolverineav has joined #openstack-infra | 00:54 | |
| *** wolverineav has quit IRC | 00:58 | |
| *** wolverineav has joined #openstack-infra | 00:58 | |
| *** jistr has quit IRC | 01:00 | |
| *** jistr has joined #openstack-infra | 01:01 | |
| *** hongbin has joined #openstack-infra | 01:03 | |
| *** wolverineav has quit IRC | 01:11 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: upgrade react-scripts to ^2.0.0 https://review.openstack.org/631902 | 01:15 |
| *** slaweq has joined #openstack-infra | 01:16 | |
| *** wolverineav has joined #openstack-infra | 01:19 | |
| *** auristor has quit IRC | 01:22 | |
| *** slaweq has quit IRC | 01:24 | |
| *** auristor has joined #openstack-infra | 01:28 | |
| *** hongbin has quit IRC | 01:37 | |
| *** auristor has quit IRC | 01:38 | |
| *** wolverineav has quit IRC | 01:38 | |
| *** markvoelker has joined #openstack-infra | 01:39 | |
| *** auristor has joined #openstack-infra | 01:41 | |
| *** auristor has quit IRC | 01:45 | |
| *** auristor has joined #openstack-infra | 01:47 | |
| *** hongbin has joined #openstack-infra | 01:56 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: upgrade react and react-scripts to ^2.0.0 https://review.openstack.org/631902 | 01:57 |
| *** jamesmcarthur has joined #openstack-infra | 01:57 | |
| *** slaweq has joined #openstack-infra | 02:11 | |
| *** markvoelker has quit IRC | 02:12 | |
| *** tkajinam_ has joined #openstack-infra | 02:19 | |
| *** jamesmcarthur has quit IRC | 02:21 | |
| *** tkajinam has quit IRC | 02:21 | |
| *** slaweq has quit IRC | 02:24 | |
| *** hongbin has quit IRC | 02:25 | |
| *** wolverineav has joined #openstack-infra | 02:46 | |
| *** jtomasek has joined #openstack-infra | 02:48 | |
| *** wolverineav has quit IRC | 02:51 | |
| *** wolverineav has joined #openstack-infra | 02:52 | |
| *** jtomasek has quit IRC | 02:53 | |
| *** wolverineav has quit IRC | 02:53 | |
| *** wolverineav has joined #openstack-infra | 02:55 | |
| *** yamamoto has joined #openstack-infra | 02:56 | |
| *** wolverineav has quit IRC | 02:59 | |
| *** psachin has joined #openstack-infra | 03:01 | |
| *** bhavikdbavishi has joined #openstack-infra | 03:02 | |
| *** jamesmcarthur has joined #openstack-infra | 03:08 | |
| *** jamesmcarthur has quit IRC | 03:08 | |
| *** whoami-rajat has quit IRC | 03:09 | |
| *** markvoelker has joined #openstack-infra | 03:09 | |
| *** slaweq has joined #openstack-infra | 03:13 | |
| *** jamesmcarthur has joined #openstack-infra | 03:17 | |
| *** slaweq has quit IRC | 03:24 | |
| *** markvoelker has quit IRC | 03:42 | |
| *** jamesmcarthur has quit IRC | 03:44 | |
| *** hongbin has joined #openstack-infra | 03:46 | |
| *** ramishra has joined #openstack-infra | 04:02 | |
| *** jamesmcarthur has joined #openstack-infra | 04:05 | |
| *** jamesmcarthur has quit IRC | 04:09 | |
| *** slaweq has joined #openstack-infra | 04:12 | |
| *** jamesmcarthur has joined #openstack-infra | 04:18 | |
| *** ykarel|away has joined #openstack-infra | 04:21 | |
| *** slaweq has quit IRC | 04:24 | |
| *** Bhujay has joined #openstack-infra | 04:24 | |
| *** Bhujay has quit IRC | 04:25 | |
| *** Bhujay has joined #openstack-infra | 04:26 | |
| *** Bhujay has quit IRC | 04:27 | |
| *** spsurya has joined #openstack-infra | 04:28 | |
| *** Bhujay has joined #openstack-infra | 04:28 | |
| *** lifeless has quit IRC | 04:31 | |
| *** ykarel|away is now known as ykarel | 04:35 | |
| *** markvoelker has joined #openstack-infra | 04:39 | |
| *** jamesmcarthur has quit IRC | 04:40 | |
| *** ykarel has quit IRC | 04:42 | |
| *** lifeless has joined #openstack-infra | 04:51 | |
| *** hongbin has quit IRC | 04:55 | |
| *** ykarel has joined #openstack-infra | 04:58 | |
| *** mgoddard has quit IRC | 04:59 | |
| *** jamesmcarthur has joined #openstack-infra | 05:01 | |
| *** tkajinam_ is now known as tkajinam | 05:03 | |
| *** jamesmcarthur has quit IRC | 05:05 | |
| *** markvoelker has quit IRC | 05:11 | |
| *** bobh has quit IRC | 05:11 | |
| *** slaweq has joined #openstack-infra | 05:14 | |
| *** jamesmcarthur has joined #openstack-infra | 05:22 | |
| *** slaweq has quit IRC | 05:25 | |
| *** jamesmcarthur has quit IRC | 05:26 | |
| *** tkajinam has quit IRC | 05:31 | |
| *** bobh has joined #openstack-infra | 05:38 | |
| *** bobh has quit IRC | 05:42 | |
| *** Bhujay has quit IRC | 05:48 | |
| *** wolverineav has joined #openstack-infra | 05:55 | |
| *** wolverineav has quit IRC | 05:56 | |
| *** wolverineav has joined #openstack-infra | 05:56 | |
| *** wolverineav has quit IRC | 05:59 | |
| *** wolverin_ has joined #openstack-infra | 05:59 | |
| *** Bhujay has joined #openstack-infra | 06:01 | |
| *** jamesmcarthur has joined #openstack-infra | 06:01 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: switch jobs list to a tree view https://review.openstack.org/633437 | 06:04 |
| *** efried has quit IRC | 06:05 | |
| *** jamesmcarthur has quit IRC | 06:07 | |
| *** slaweq has joined #openstack-infra | 06:11 | |
| *** tkajinam has joined #openstack-infra | 06:13 | |
| *** ddurst has quit IRC | 06:14 | |
| *** jamesmcarthur has joined #openstack-infra | 06:18 | |
| *** jamesmcarthur has quit IRC | 06:18 | |
| *** quiquell|off is now known as quiquell | 06:20 | |
| *** slaweq has quit IRC | 06:24 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: scheduler: add job's parent name to the rpc job_list method https://review.openstack.org/573473 | 06:25 |
| *** sshnaidm|off is now known as sshnaidm | 06:30 | |
| *** jamesmcarthur has joined #openstack-infra | 06:39 | |
| *** jamesmcarthur has quit IRC | 06:43 | |
| *** efried has joined #openstack-infra | 06:44 | |
| *** jbadiapa has joined #openstack-infra | 06:48 | |
| *** slaweq has joined #openstack-infra | 06:51 | |
| *** bobh has joined #openstack-infra | 06:55 | |
| *** wolverin_ has quit IRC | 06:56 | |
| *** dpawlik has joined #openstack-infra | 06:57 | |
| *** wolverineav has joined #openstack-infra | 06:58 | |
| *** slaweq has quit IRC | 06:58 | |
| *** bobh has quit IRC | 07:00 | |
| *** jamesmcarthur has joined #openstack-infra | 07:00 | |
| openstackgerrit | Simon Westphahl proposed openstack-infra/zuul master: List changed files for all commits between refs https://review.openstack.org/631797 | 07:01 |
| *** jtomasek has joined #openstack-infra | 07:03 | |
| *** jamesmcarthur has quit IRC | 07:04 | |
| *** chandankumar has quit IRC | 07:07 | |
| *** ramishra has quit IRC | 07:08 | |
| *** markvoelker has joined #openstack-infra | 07:09 | |
| prometheanfire | cent mirrors seem sad (both osa and dib changes for things unrelated to cent)? | 07:11 |
| *** slaweq has joined #openstack-infra | 07:11 | |
| *** slaweq has quit IRC | 07:15 | |
| *** fnordahl has joined #openstack-infra | 07:19 | |
| *** panda|off is now known as panda | 07:20 | |
| *** jamesmcarthur has joined #openstack-infra | 07:21 | |
| *** chandan_kumar has joined #openstack-infra | 07:21 | |
| *** jamesmcarthur has quit IRC | 07:25 | |
| *** kopecmartin|off is now known as kopecmartin | 07:27 | |
| *** wolverineav has quit IRC | 07:30 | |
| *** rpittau has joined #openstack-infra | 07:32 | |
| *** wolverineav has joined #openstack-infra | 07:32 | |
| *** wolverineav has quit IRC | 07:37 | |
| *** wolverineav has joined #openstack-infra | 07:38 | |
| *** wolverineav has quit IRC | 07:38 | |
| *** wolverineav has joined #openstack-infra | 07:38 | |
| *** dayou has quit IRC | 07:39 | |
| *** bnemec has quit IRC | 07:40 | |
| *** ramishra has joined #openstack-infra | 07:41 | |
| *** jamesmcarthur has joined #openstack-infra | 07:41 | |
| *** jtomasek has quit IRC | 07:42 | |
| *** markvoelker has quit IRC | 07:42 | |
| *** dayou has joined #openstack-infra | 07:42 | |
| *** Bhujay has quit IRC | 07:44 | |
| openstackgerrit | Rui Chen proposed openstack-infra/zuul master: Avoid using list branches with protected=1 in github driver https://review.openstack.org/630038 | 07:45 |
| *** hwoarang has quit IRC | 07:45 | |
| *** jamesmcarthur has quit IRC | 07:45 | |
| *** hwoarang has joined #openstack-infra | 07:46 | |
| *** ijw has joined #openstack-infra | 07:47 | |
| *** ijw_ has quit IRC | 07:49 | |
| *** rascasoft has joined #openstack-infra | 07:49 | |
| *** lpetrut has joined #openstack-infra | 07:50 | |
| *** pgaxatte has joined #openstack-infra | 07:52 | |
| openstackgerrit | Simon Westphahl proposed openstack-infra/zuul master: List changed files for all commits between refs https://review.openstack.org/631797 | 07:54 |
| *** ginopc has joined #openstack-infra | 07:58 | |
| *** pcaruana has joined #openstack-infra | 08:01 | |
| *** jtomasek has joined #openstack-infra | 08:02 | |
| *** chandan_kumar is now known as chandankumar | 08:02 | |
| *** slaweq has joined #openstack-infra | 08:03 | |
| *** tkajinam has quit IRC | 08:15 | |
| *** ykarel is now known as ykarel|lunch | 08:20 | |
| *** jamesmcarthur has joined #openstack-infra | 08:22 | |
| *** ccamacho has joined #openstack-infra | 08:25 | |
| *** apetrich has joined #openstack-infra | 08:27 | |
| *** jamesmcarthur has quit IRC | 08:27 | |
| *** bnemec has joined #openstack-infra | 08:35 | |
| *** markvoelker has joined #openstack-infra | 08:39 | |
| *** tosky has joined #openstack-infra | 08:41 | |
| *** ykarel|lunch is now known as ykarel | 08:51 | |
| *** kjackal has joined #openstack-infra | 08:52 | |
| *** electrofelix has joined #openstack-infra | 08:53 | |
| *** jpena|off is now known as jpena | 08:54 | |
| *** jpich has joined #openstack-infra | 08:56 | |
| *** lpetrut has quit IRC | 08:57 | |
| *** mgoddard has joined #openstack-infra | 09:02 | |
| *** jamesmcarthur has joined #openstack-infra | 09:04 | |
| *** yamamoto has quit IRC | 09:07 | |
| *** jamesmcarthur has quit IRC | 09:09 | |
| *** jaosorior has joined #openstack-infra | 09:10 | |
| *** yamamoto has joined #openstack-infra | 09:11 | |
| *** yamamoto has quit IRC | 09:11 | |
| *** yamamoto has joined #openstack-infra | 09:12 | |
| *** kjackal has quit IRC | 09:12 | |
| *** markvoelker has quit IRC | 09:12 | |
| *** bobh has joined #openstack-infra | 09:13 | |
| *** aojea has joined #openstack-infra | 09:13 | |
| *** ijw has quit IRC | 09:13 | |
| *** xek has joined #openstack-infra | 09:15 | |
| *** yamamoto has quit IRC | 09:16 | |
| *** bobh has quit IRC | 09:17 | |
| *** whoami-rajat has joined #openstack-infra | 09:18 | |
| *** wolverineav has quit IRC | 09:22 | |
| *** wolverineav has joined #openstack-infra | 09:23 | |
| *** shardy has joined #openstack-infra | 09:29 | |
| *** dtantsur|afk is now known as dtantsur | 09:32 | |
| *** derekh has joined #openstack-infra | 09:37 | |
| *** e0ne has joined #openstack-infra | 09:38 | |
| *** gfidente has joined #openstack-infra | 09:38 | |
| *** jamesmcarthur has joined #openstack-infra | 09:39 | |
| *** luizbag has joined #openstack-infra | 09:41 | |
| *** jamesmcarthur has quit IRC | 09:44 | |
| *** jesusaur has quit IRC | 09:45 | |
| *** jesusaur has joined #openstack-infra | 09:51 | |
| *** bhavikdbavishi has quit IRC | 09:53 | |
| mnasiadka | are there some network problems in rax-iad? getting connection reset by peer when trying to download from apache mirrors... | 09:55 |
| *** e0ne has quit IRC | 09:58 | |
| *** wolverineav has quit IRC | 09:59 | |
| *** yamamoto has joined #openstack-infra | 10:00 | |
| *** jamesmcarthur has joined #openstack-infra | 10:01 | |
| *** e0ne has joined #openstack-infra | 10:02 | |
| *** AJaeger has quit IRC | 10:03 | |
| *** jamesmcarthur has quit IRC | 10:06 | |
| *** yamamoto has quit IRC | 10:06 | |
| *** AJaeger has joined #openstack-infra | 10:07 | |
| openstackgerrit | Tobias Rydberg proposed openstack-infra/irc-meetings master: Change of agenda url for weekly meetings https://review.openstack.org/633466 | 10:10 |
| *** markvoelker has joined #openstack-infra | 10:10 | |
| *** kjackal has joined #openstack-infra | 10:16 | |
| *** jamesmcarthur has joined #openstack-infra | 10:23 | |
| *** wolverineav has joined #openstack-infra | 10:26 | |
| *** jamesmcarthur has quit IRC | 10:28 | |
| *** hrubi has joined #openstack-infra | 10:35 | |
| *** SotK is now known as SotK__ | 10:39 | |
| *** SotK_ is now known as SotK | 10:39 | |
| *** markvoelker has quit IRC | 10:43 | |
| *** jamesmcarthur has joined #openstack-infra | 10:44 | |
| *** psachin has quit IRC | 10:45 | |
| *** jamesmcarthur has quit IRC | 10:49 | |
| *** rfolco has joined #openstack-infra | 10:56 | |
| *** ldnunes has joined #openstack-infra | 10:57 | |
| *** wolverineav has quit IRC | 10:59 | |
| *** e0ne has quit IRC | 11:03 | |
| *** jpich has quit IRC | 11:04 | |
| *** bobh has joined #openstack-infra | 11:05 | |
| *** jamesmcarthur has joined #openstack-infra | 11:05 | |
| *** jpich has joined #openstack-infra | 11:06 | |
| *** ginopc has quit IRC | 11:08 | |
| *** e0ne has joined #openstack-infra | 11:10 | |
| *** bobh has quit IRC | 11:10 | |
| *** jamesmcarthur has quit IRC | 11:10 | |
| *** szaher has quit IRC | 11:26 | |
| *** jamesmcarthur has joined #openstack-infra | 11:27 | |
| *** bobh has joined #openstack-infra | 11:29 | |
| *** szaher has joined #openstack-infra | 11:29 | |
| *** jamesmcarthur has quit IRC | 11:31 | |
| *** bobh has quit IRC | 11:33 | |
| *** ginopc has joined #openstack-infra | 11:38 | |
| *** markvoelker has joined #openstack-infra | 11:40 | |
| *** bnemec has quit IRC | 11:40 | |
| *** ginopc has quit IRC | 11:40 | |
| *** ginopc has joined #openstack-infra | 11:41 | |
| *** bnemec has joined #openstack-infra | 11:42 | |
| *** psachin has joined #openstack-infra | 11:42 | |
| fungi | infra-root: just a reminder, i'm away from the computer pretty much all week (travelling on vacation) but do expect to be around again starting monday 2019-02-04 | 11:44 |
| *** whoami-rajat has quit IRC | 11:45 | |
| *** bobh has joined #openstack-infra | 11:45 | |
| mordred | infra-root: although I'm not travelling to the same place as fungi (at least I think I'm not) I'm also AFK for most of the week (Tue-Thur for sure, will be spotty today) | 11:46 |
| *** jamesmcarthur has joined #openstack-infra | 11:48 | |
| AJaeger | fungi: have a great vacation! | 11:48 |
| fungi | thanks! | 11:49 |
| *** bobh has quit IRC | 11:50 | |
| *** kjackal has quit IRC | 11:50 | |
| *** bhavikdbavishi has joined #openstack-infra | 11:53 | |
| *** jamesmcarthur has quit IRC | 11:53 | |
| *** wolverineav has joined #openstack-infra | 11:57 | |
| *** yamamoto has joined #openstack-infra | 12:00 | |
| *** panda is now known as panda|lunch | 12:07 | |
| *** jesusaur has quit IRC | 12:07 | |
| *** jamesmcarthur has joined #openstack-infra | 12:10 | |
| *** bobh has joined #openstack-infra | 12:10 | |
| *** markvoelker has quit IRC | 12:13 | |
| *** jamesmcarthur has quit IRC | 12:14 | |
| *** bobh has quit IRC | 12:15 | |
| *** jesusaur has joined #openstack-infra | 12:15 | |
| *** nmimi has joined #openstack-infra | 12:16 | |
| *** e0ne has quit IRC | 12:20 | |
| *** ramishra has quit IRC | 12:20 | |
| *** nmimi has quit IRC | 12:23 | |
| *** kjackal has joined #openstack-infra | 12:25 | |
| *** wolverineav has quit IRC | 12:29 | |
| *** jamesmcarthur has joined #openstack-infra | 12:31 | |
| *** jpena is now known as jpena|lunch | 12:31 | |
| *** panda|lunch is now known as panda | 12:33 | |
| *** e0ne has joined #openstack-infra | 12:35 | |
| *** jamesmcarthur has quit IRC | 12:36 | |
| *** pcaruana has quit IRC | 12:36 | |
| *** pcaruana has joined #openstack-infra | 12:37 | |
| *** yamamoto has quit IRC | 12:37 | |
| *** jesusaur has quit IRC | 12:46 | |
| *** rh-jelabarre has joined #openstack-infra | 12:48 | |
| *** jesusaur has joined #openstack-infra | 12:51 | |
| *** jamesmcarthur has joined #openstack-infra | 12:52 | |
| *** rlandy has joined #openstack-infra | 12:54 | |
| *** jamesmcarthur has quit IRC | 12:57 | |
| *** yamamoto has joined #openstack-infra | 13:05 | |
| *** ramishra has joined #openstack-infra | 13:09 | |
| *** bhavikdbavishi has quit IRC | 13:10 | |
| *** jamesmcarthur has joined #openstack-infra | 13:13 | |
| *** wolverineav has joined #openstack-infra | 13:17 | |
| *** e0ne has quit IRC | 13:17 | |
| *** jamesmcarthur has quit IRC | 13:17 | |
| *** bhavikdbavishi has joined #openstack-infra | 13:18 | |
| *** yamamoto has quit IRC | 13:19 | |
| *** e0ne has joined #openstack-infra | 13:20 | |
| *** derekh has quit IRC | 13:24 | |
| *** whoami-rajat has joined #openstack-infra | 13:26 | |
| *** noama has joined #openstack-infra | 13:27 | |
| *** jpena|lunch is now known as jpena | 13:29 | |
| *** agopi has quit IRC | 13:31 | |
| *** pcaruana has quit IRC | 13:32 | |
| *** jamesmcarthur has joined #openstack-infra | 13:35 | |
| *** bhavikdbavishi has quit IRC | 13:37 | |
| *** jamesmcarthur has quit IRC | 13:39 | |
| *** yamamoto has joined #openstack-infra | 13:40 | |
| *** yamamoto has quit IRC | 13:40 | |
| openstackgerrit | Benedikt Löffler proposed openstack-infra/zuul master: Report retried builds via sql reporter. https://review.openstack.org/633501 | 13:40 |
| *** yamamoto has joined #openstack-infra | 13:40 | |
| *** psachin has quit IRC | 13:41 | |
| *** mriedem has joined #openstack-infra | 13:43 | |
| *** yamamoto has quit IRC | 13:45 | |
| *** jcoufal has joined #openstack-infra | 13:48 | |
| *** pcaruana has joined #openstack-infra | 13:50 | |
| *** jento has quit IRC | 13:50 | |
| *** wolverineav has quit IRC | 13:50 | |
| *** kgiusti has joined #openstack-infra | 13:57 | |
| *** bhavikdbavishi has joined #openstack-infra | 13:58 | |
| *** bobh has joined #openstack-infra | 14:00 | |
| *** jento has joined #openstack-infra | 14:02 | |
| *** agopi has joined #openstack-infra | 14:03 | |
| *** trown|outtypewww is now known as trown | 14:03 | |
| *** bhavikdbavishi has quit IRC | 14:07 | |
| *** jpich has quit IRC | 14:07 | |
| *** jpich has joined #openstack-infra | 14:08 | |
| *** dave-mccowan has joined #openstack-infra | 14:09 | |
| *** smarcet has joined #openstack-infra | 14:12 | |
| *** derekh has joined #openstack-infra | 14:13 | |
| *** agopi has quit IRC | 14:13 | |
| *** agopi has joined #openstack-infra | 14:13 | |
| *** agopi_ has joined #openstack-infra | 14:15 | |
| *** agopi_ has quit IRC | 14:16 | |
| *** agopi_ has joined #openstack-infra | 14:16 | |
| *** agopi has quit IRC | 14:18 | |
| *** agopi_ is now known as agopi | 14:18 | |
| pabelanger | I am seeing a large amount of failues in limestone to clone from github.com or package update: http://logs.openstack.org/70/633370/1/check/windmill-src-fedora-latest/5a4904c/job-output.txt.gz#_2019-01-28_04_28_16_028454 | 14:19 |
| *** sthussey has joined #openstack-infra | 14:33 | |
| *** bhavikdbavishi has joined #openstack-infra | 14:34 | |
| *** jamesmcarthur has joined #openstack-infra | 14:35 | |
| *** eharney has joined #openstack-infra | 14:35 | |
| *** ykarel is now known as ykarel|away | 14:43 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Fix wedged scheduler on force-merge of non-existing template https://review.openstack.org/633514 | 14:44 |
| *** quiquell is now known as quiquell|off | 14:47 | |
| *** wolverineav has joined #openstack-infra | 14:48 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/project-config master: Add stable branches to rpm-packaging gerritbot https://review.openstack.org/633517 | 14:49 |
| *** ekultails has joined #openstack-infra | 14:58 | |
| *** kaiokmo has quit IRC | 15:03 | |
| dulek | frickler: Hi, can you please take a look on https://review.openstack.org/#/c/626885/ ? I've addressed your comments threre. | 15:05 |
| pabelanger | infra-root: any objections on tagging new gear release? Would like to pick up tobiash keepalive support? there is only that commit and author-email change. | 15:06 |
| *** ykarel|away has quit IRC | 15:07 | |
| *** chandankumar is now known as chkumar|out | 15:08 | |
| *** dkehn has joined #openstack-infra | 15:19 | |
| *** shardy has quit IRC | 15:19 | |
| *** wolverineav has quit IRC | 15:21 | |
| *** shardy has joined #openstack-infra | 15:21 | |
| *** redrobot has joined #openstack-infra | 15:24 | |
| *** ykarel|away has joined #openstack-infra | 15:36 | |
| pabelanger | clarkb: weshay: EmilienM: there seems to be non-voting jobs in gate, 630212,6 is an example. Seems to be puppet jobs? | 15:37 |
| *** anteaya has joined #openstack-infra | 15:37 | |
| pabelanger | happen to notice it when looking for something else | 15:38 |
| openstackgerrit | Benedikt Löffler proposed openstack-infra/zuul master: Report retried builds via sql reporter. https://review.openstack.org/633501 | 15:42 |
| openstackgerrit | Merged openstack-infra/project-config master: Add openstack/ansible-role-borgbackup https://review.openstack.org/633095 | 15:46 |
| openstackgerrit | Merged openstack-infra/project-config master: Add openstack/ansible-role-borgmatic https://review.openstack.org/633413 | 15:47 |
| *** cristoph has joined #openstack-infra | 15:49 | |
| *** openstackgerrit has quit IRC | 15:51 | |
| *** openstackgerrit has joined #openstack-infra | 15:55 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config master: Create project for ansible-role-ara-server https://review.openstack.org/633538 | 15:55 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config master: Create project for ansible-role-ara-web https://review.openstack.org/633539 | 15:55 |
| corvus | pabelanger, tobiash: should i tag 7a6ef8ac60283425107e192085125c732aa1eb7c as gear 0.13.0 ? | 15:59 |
| pabelanger | corvus: +1 | 16:00 |
| clarkb | prometheanfire: mnasiadka links to logs would be useful in both cases | 16:00 |
| clarkb | re gear reviews on https://review.openstack.org/#/c/611920/ are appreciated to address some ipv6 vs ipv4 listening issues | 16:01 |
| prometheanfire | sure | 16:03 |
| prometheanfire | clarkb: https://review.openstack.org/633425 is probably a good test, as it not touching anything cent | 16:04 |
| corvus | clarkb: what input values would cause that problem? | 16:04 |
| prometheanfire | https://review.openstack.org/633424 too | 16:04 |
| clarkb | corvus: the defaults, the AF_UNSPEC results in ipv4 being returned before ipv6 so the for loop breaks after finding valid ipv4 addr and doesn't listen on ipv6 | 16:05 |
| *** dklyle has joined #openstack-infra | 16:06 | |
| *** shardy has quit IRC | 16:06 | |
| *** shardy has joined #openstack-infra | 16:07 | |
| clarkb | prometheanfire: which job? | 16:07 |
| clarkb | http://logs.openstack.org/25/633425/1/check/nodepool-functional-py35-redhat-src/7de7c5f/ ? | 16:07 |
| prometheanfire | ja | 16:08 |
| clarkb | prometheanfire: http://logs.openstack.org/25/633425/1/check/nodepool-functional-py35-redhat-src/7de7c5f/controller/nodepool/nodepool-list.txt shows that the centos 7 image built and booted fine. The fedora image built but hasn't booted | 16:08 |
| prometheanfire | previously I've seen repo related issues, not sure about this run though | 16:08 |
| corvus | clarkb: ack. maybe the default host should have been "::". would that be better, or should we stick with your change? | 16:09 |
| clarkb | corvus: :: won't work if you don't have an ipv6 stack (unlikely these days), but my change should work if ipv4 or ipv6 only networking is configured | 16:10 |
| corvus | ok. +3, i'll wait for that to land before release (cc: tobiash, pabelanger) | 16:11 |
| pabelanger | wfm | 16:12 |
| clarkb | prometheanfire: http://logs.openstack.org/25/633425/1/check/nodepool-functional-py35-redhat-src/7de7c5f/controller/logs/screen-nodepool-launcher.txt.gz#_Jan_28_08_37_16_796672 looks like networking issue? was there something you saw that made centos mirrors look suspect? | 16:12 |
| clarkb | prometheanfire: looking at that console log I Think glean is starting too late in the boot process to have the config written for dhcp get picked up by the network manager startup? | 16:13 |
| clarkb | ianw may want to look at that | 16:14 |
| prometheanfire | racy? | 16:14 |
| prometheanfire | racey sp? | 16:14 |
| clarkb | prometheanfire: ya or we've not accomodated the dep graph properly | 16:14 |
| prometheanfire | true, systemd should handle it (or openrc :P) | 16:14 |
| clarkb | http://logs.openstack.org/25/633425/1/check/nodepool-functional-py35-redhat-src/7de7c5f/controller/logs/screen-nodepool-launcher.txt.gz#_Jan_28_08_37_16_784479 is where network manager has a sad | 16:15 |
| clarkb | then later glean configures the interfaces | 16:15 |
| prometheanfire | yep | 16:15 |
| clarkb | and it seems to be dbus that is failing | 16:15 |
| clarkb | wait for udev? | 16:16 |
| prometheanfire | https://github.com/openstack-infra/glean/blob/master/glean/init/glean-nm%40.service should be modified to have before=nm | 16:16 |
| prometheanfire | I think | 16:16 |
| prometheanfire | or wantedby | 16:16 |
| *** wolverineav has joined #openstack-infra | 16:17 | |
| clarkb | possibly though in this case nm is failing entirely so we probably need to fix that too | 16:18 |
| prometheanfire | [[0;1;31mFAILED[0m] Failed to listen on D-Bus System Message Bus Socket. ya, that seems bad | 16:19 |
| openstackgerrit | Merged openstack-infra/gear master: Prefer ipv6 for listen addrs if available https://review.openstack.org/611920 | 16:21 |
| pabelanger | clarkb: prometheanfire: https://bugzilla.redhat.com/show_bug.cgi?id=1623781 seems to say dbus-broker might help | 16:22 |
| openstack | bugzilla.redhat.com bug 1623781 in dbus "dbus-1:1.12.10-2.fc29 breaks system boot" [High,New] - Assigned to walters | 16:22 |
| prometheanfire | :D | 16:22 |
| corvus | gear 0.13.0 pushed | 16:23 |
| clarkb | pabelanger: looks like that was fixed then broken then fixed again between october and december | 16:25 |
| clarkb | pabelanger: I guess it could be a new regression? | 16:25 |
| pabelanger | yah, possible. job looks to have been working a few days ago. Would be interesting to audit package differences | 16:26 |
| tobiash | corvus: thanks :) | 16:27 |
| *** efried has quit IRC | 16:28 | |
| *** bhavikdbavishi has quit IRC | 16:34 | |
| *** aojea has quit IRC | 16:35 | |
| *** bhavikdbavishi has joined #openstack-infra | 16:37 | |
| corvus | tobiash, pabelanger, clarkb: gear 0.13.0 is on pypi | 16:41 |
| tobiash | \o/ | 16:41 |
| pabelanger | tyty | 16:42 |
| pabelanger | going to upgrade POC now :) | 16:42 |
| *** dmsimard has quit IRC | 16:43 | |
| *** pcaruana has quit IRC | 16:47 | |
| *** dmsimard has joined #openstack-infra | 16:47 | |
| *** dmsimard has quit IRC | 16:49 | |
| *** wolverineav has quit IRC | 16:50 | |
| *** bhavikdbavishi has quit IRC | 16:50 | |
| *** bhavikdbavishi has joined #openstack-infra | 16:51 | |
| *** dmsimard has joined #openstack-infra | 16:52 | |
| *** ginopc has quit IRC | 16:55 | |
| *** agopi is now known as agopi|lunch | 16:58 | |
| *** wolverineav has joined #openstack-infra | 17:00 | |
| *** pgaxatte has quit IRC | 17:02 | |
| *** noama has quit IRC | 17:03 | |
| *** dims has quit IRC | 17:08 | |
| *** gyee has joined #openstack-infra | 17:09 | |
| *** ijw has joined #openstack-infra | 17:09 | |
| *** rpittau has quit IRC | 17:12 | |
| *** bhavikdbavishi has quit IRC | 17:12 | |
| *** bhavikdbavishi has joined #openstack-infra | 17:13 | |
| *** dkehn has quit IRC | 17:15 | |
| clarkb | speaking of all the travel. I get on a plane wednesday morning to go to fosdem and will be largely afk until mid next week. I will be here today and tomorrow though | 17:16 |
| prometheanfire | clarkb: have fun, it's always a good time :D | 17:18 |
| *** kopecmartin is now known as kopecmartin|off | 17:18 | |
| *** wolverineav has quit IRC | 17:26 | |
| *** eernst has joined #openstack-infra | 17:26 | |
| *** jpich has quit IRC | 17:29 | |
| corvus | the existence of ~/git/opendev/ is really messing with my tab-completion. | 17:33 |
| corvus | i've got 8 years of habit to undo | 17:33 |
| clarkb | corvus: I've got openstack openstack-infra and openstack-dev too, so I have two layers of tab pain :) | 17:34 |
| *** sdake has quit IRC | 17:34 | |
| *** whoami-rajat has quit IRC | 17:35 | |
| corvus | yeah i "cd open^T-in^T" to get to openstack-infra, but that first ^T doesn't expand 'openstack' anymore. | 17:35 |
| corvus | except! | 17:35 |
| corvus | emacs is smart enough to still work the same way :) | 17:36 |
| prometheanfire | heh, I do the same | 17:36 |
| prometheanfire | need to set up some symlinks, dev, infra, base | 17:36 |
| corvus | "open^T-in^T" totally works in emacs, because even though the first ^T doesn't do anything, it can figure it out from the second one | 17:36 |
| prometheanfire | zfs could too probably, but I use bash for portability | 17:37 |
| *** sdake has joined #openstack-infra | 17:37 | |
| clarkb | corvus: speaking of is https://review.openstack.org/#/c/632785/4 basically ready to go at this point? It isn't tested because it isn't part of any trusted config yet. We may also need to rename roles if we want to add that into the existing openstack tenant? | 17:39 |
| *** bnemec has quit IRC | 17:39 | |
| clarkb | so I guess my open questions are should I rename roles so that we don't conflict on the openstack side and then next step is to merge then add to trusted config for openstack tenant? | 17:40 |
| tobiash | pabelanger, corvus: the gearman keepalive in zuul is green now :) | 17:40 |
| openstackgerrit | Paul Vinciguerra proposed openstack-infra/zuul master: Identify the gerrit source when logging stream-events errors. https://review.openstack.org/633567 | 17:40 |
| tobiash | https://review.openstack.org/599573 | 17:40 |
| pabelanger | looking | 17:41 |
| *** trown is now known as trown|lunch | 17:42 | |
| corvus | clarkb: i'm refreshing my memory :) | 17:42 |
| *** e0ne has quit IRC | 17:42 | |
| clarkb | an alternative would be to get the reorg of base jobs in first then update that change | 17:42 |
| corvus | clarkb: duplicating the role names is fine; we should simply remove them from ozj after the complete | 17:43 |
| corvus | er, after the move is complete | 17:43 |
| clarkb | corvus: ok | 17:43 |
| corvus | clarkb: re base jobs reorg -- that depends on your timetable. most important i think is that we don't do both at the same time. | 17:43 |
| pabelanger | tobiash: left comment, but looks good in general | 17:44 |
| clarkb | corvus: I think ttx would like to get the dco checking running for kata in a tenant prior to their existing service being turned off (soon/nowish) I probably slightly prefer switching to opendev base jobs first. Also I think the converted base jobs would require a second opendev/ repo which is another step we'd have to take | 17:45 |
| corvus | clarkb: it would be really helpful to have a non-whitelabel zuul web ui before we start making any tenant changes | 17:45 |
| tobiash | pabelanger: do we need a note in the upgrades section for the dependency bump? It's listed in the requirements so I don't know what a user could need to do when upgrading. | 17:45 |
| clarkb | corvus: ya that is the other piece | 17:45 |
| clarkb | I think shrews was going to look at ^ | 17:45 |
| clarkb | or was at least interested in it iirc | 17:45 |
| pabelanger | tobiash: thinking more about people who package, but yah pip install should pull in new dependency | 17:46 |
| corvus | i think we can merge 785, i just don't want to do a change to main.yaml before the web stuff is in place (i'd like to ask the zuul api what the config errors are for the new tenant) | 17:46 |
| pabelanger | tobiash: also, not a -1, wanted to raise the topic | 17:46 |
| corvus | please don't add a reno note for the gear upgrade | 17:47 |
| pabelanger | tobiash: also, possible for people to run external gear server, so that wouldn't upgrade | 17:47 |
| corvus | what would it say? | 17:47 |
| Shrews | clarkb: look at what? | 17:48 |
| pabelanger | hard depencency on 0.13.0 mostly | 17:48 |
| clarkb | looking at puppet-zuul the current vhost setup switches on whether or not a specific tenant is configured to add in the rewrite rules. I think we can probably update that to switch on whether or not there is a whitelabel name to write a generic vhost and if there is a tenant to write the tenant specific vhost | 17:48 |
| clarkb | Shrews: the zuul whitelabel hosting | 17:48 |
| *** electrofelix has quit IRC | 17:48 | |
| tobiash | pabelanger: that's a pure client side thing, so for people that run an external gear server there is no change | 17:48 |
| Shrews | clarkb: hrm, not i | 17:49 |
| clarkb | I need to get through a couple of items on my backlog (like meeting agenda prep) but I can take a stab at reorging the zuul vhost config | 17:49 |
| corvus | pabelanger: there's already a file that tells packagers and installation systems that. release upgrade notes are for communicating actions that zuul users or operators need to perform. no change is required here. | 17:49 |
| pabelanger | tobiash: ah, okay. then I have not understood the fix properly :) Feel free to ignore then | 17:49 |
| clarkb | ya we can do a list of servername + tenant_name (or nil) + ssl cert file details then iterate over that in the erb | 17:50 |
| clarkb | the diff between what we have now and the end result might be a bit ugly but the end result should hopefully be readable | 17:51 |
| *** sdake has quit IRC | 17:51 | |
| pabelanger | corvus: tobiash: +3 | 17:52 |
| tobiash | \o/ | 17:52 |
| corvus | clarkb: 785 lgtm; needs secrets. | 17:52 |
| pabelanger | grabbing some coffee, sorry for noise again | 17:52 |
| *** sdake_ has joined #openstack-infra | 17:52 | |
| *** sdake_ has quit IRC | 17:53 | |
| *** jamesmcarthur has quit IRC | 17:54 | |
| *** jamesmcarthur has joined #openstack-infra | 17:55 | |
| *** bmeier has joined #openstack-infra | 17:55 | |
| *** agopi|lunch is now known as agopi | 17:55 | |
| *** bmeier has quit IRC | 17:55 | |
| clarkb | corvus: oh right. We need to wait on adding the project to zuul before it will have encryption keys right? | 17:55 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Update DNS documentation https://review.openstack.org/633569 | 17:56 |
| corvus | clarkb: oh, yes that is true | 17:56 |
| corvus | let's get that going then :) | 17:56 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config master: Update DNS documentation https://review.openstack.org/633569 | 17:58 |
| *** derekh has quit IRC | 18:00 | |
| *** whoami-rajat has joined #openstack-infra | 18:02 | |
| *** bnemec has joined #openstack-infra | 18:03 | |
| *** gfidente has quit IRC | 18:03 | |
| *** bhavikdbavishi has quit IRC | 18:04 | |
| *** wolverineav has joined #openstack-infra | 18:11 | |
| *** jpena is now known as jpena|off | 18:13 | |
| *** ykarel|away has quit IRC | 18:13 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/nodepool master: Amazon EC2 driver https://review.openstack.org/535558 | 18:13 |
| *** sdake_ has joined #openstack-infra | 18:13 | |
| *** wolverineav has quit IRC | 18:14 | |
| *** wolverineav has joined #openstack-infra | 18:14 | |
| *** yamamoto has joined #openstack-infra | 18:14 | |
| *** sdake_ is now known as sdkae | 18:14 | |
| *** wolverineav has quit IRC | 18:16 | |
| *** wolverineav has joined #openstack-infra | 18:17 | |
| openstackgerrit | Merged openstack-infra/zuul master: Incoming connections over 19885/TCP are needed on nodes https://review.openstack.org/633242 | 18:18 |
| *** wolverineav has quit IRC | 18:18 | |
| *** wolverineav has joined #openstack-infra | 18:19 | |
| tosky | infra team, who could help us (sahara) to remove the really old versions of sahara published on pip? They have an higher version number: https://pypi.org/project/sahara/#history | 18:32 |
| *** shardy has quit IRC | 18:33 | |
| *** trown|lunch is now known as trown | 18:36 | |
| openstackgerrit | Paul Vinciguerra proposed openstack-infra/zuul master: Identify the gerrit source when logging stream-events errors. https://review.openstack.org/633567 | 18:37 |
| clarkb | tosky: 2014.* versions basically? | 18:37 |
| clarkb | corvus: is the zuul web /api/status url always tenant specific or is there a global json too? | 18:39 |
| tosky | clarkb: yes, correct | 18:40 |
| tosky | I see also a weird version "0", but it should not matter much | 18:40 |
| corvus | clarkb: always tenant -- most of the api calls are tenant specific | 18:40 |
| corvus | clarkb: http://git.zuul-ci.org/cgit/zuul/tree/zuul/web/__init__.py#n622 | 18:41 |
| corvus | clarkb: all but the first 2 there | 18:41 |
| clarkb | ok so the status backups stuff we have will need to be aware of whitelabeling too | 18:41 |
| corvus | clarkb: yeah. would probably be best for that to be a script which hit /tenants then iterated over every tenant/status | 18:42 |
| clarkb | what I've currently got can infer whether or not to set up cron jobs based on puppet config for tenant specifics. I agree though a good followon would be a proper script | 18:43 |
| corvus | that'll work too :) | 18:44 |
| corvus | clarkb: it's just that i'm not sure that we're going to want more whitelables | 18:44 |
| *** jamesmcarthur has quit IRC | 18:45 | |
| clarkb | oh good point | 18:45 |
| corvus | but i don't think anything other than the openstack one matters now, so i think what you have is fine | 18:45 |
| clarkb | ya I'm currently melting my brain on pupept so don't want to context switch to proper script just yet | 18:46 |
| clarkb | but definitely that would be a good followon | 18:46 |
| *** jamesmcarthur has joined #openstack-infra | 18:47 | |
| clarkb | if anyone is wondering create_resources function is teh for each loop of puppet | 18:47 |
| *** jamesmcarthur has quit IRC | 18:52 | |
| *** ramishra has quit IRC | 18:53 | |
| *** e0ne has joined #openstack-infra | 18:59 | |
| *** dave-mccowan has quit IRC | 19:00 | |
| *** rfolco has quit IRC | 19:00 | |
| *** wolverineav has quit IRC | 19:02 | |
| *** wolverineav has joined #openstack-infra | 19:03 | |
| *** jamesmcarthur has joined #openstack-infra | 19:05 | |
| *** yamamoto has quit IRC | 19:06 | |
| *** jamesmcarthur_ has joined #openstack-infra | 19:06 | |
| *** yamamoto has joined #openstack-infra | 19:08 | |
| *** dave-mccowan has joined #openstack-infra | 19:09 | |
| *** yamamoto has quit IRC | 19:09 | |
| *** yamamoto has joined #openstack-infra | 19:09 | |
| *** yamamoto has quit IRC | 19:09 | |
| *** jamesmcarthur has quit IRC | 19:09 | |
| *** yamamoto has joined #openstack-infra | 19:10 | |
| *** raissa has joined #openstack-infra | 19:14 | |
| *** spsurya has quit IRC | 19:14 | |
| *** yamamoto has quit IRC | 19:14 | |
| *** dims has joined #openstack-infra | 19:15 | |
| *** luizbag has quit IRC | 19:15 | |
| *** raissa has quit IRC | 19:16 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-zuul master: Allow for both global zuul web and whitelabeled tenant https://review.openstack.org/633586 | 19:18 |
| clarkb | corvus: ttx ^ that is likely to fail testing, but I think I've got most of the big things sorted out there | 19:18 |
| clarkb | Then from the consumption point we would provide details for those three new config hashes that correspond to the various vhosts that we want | 19:19 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/nodepool master: Amazon EC2 driver https://review.openstack.org/535558 | 19:19 |
| *** diablo_rojo has joined #openstack-infra | 19:20 | |
| *** jamesmcarthur_ has quit IRC | 19:21 | |
| clarkb | that also sort of presumes our existing ssl setup of feeding in a cert that doesn't change often | 19:22 |
| corvus | clarkb: is there a change to add opendev/base-jobs to zuul? (we can just add it to openstack as an untrusted project to get it bootstrapped) | 19:22 |
| clarkb | corvus: I think ttx has one up to make it part of trusted config for a new kata tenant, but not one liek you describe /me finds a link | 19:23 |
| clarkb | corvus: https://review.openstack.org/#/c/630994/5/zuul/main.yaml | 19:23 |
| pabelanger | clarkb: prometheanfire: ianw: I am guessing, based on how long fedora-29 nodes are taking to launch, they are also not working properly on all clouds. | 19:23 |
| pabelanger | wonder if we should roll back, if possible fedora-29 | 19:24 |
| clarkb | pabelanger: if the distro is broken... | 19:24 |
| corvus | clarkb: let's make a new change that just adds it to the config so we can get a key, then update/merge your change, then get the web stuff in, then make a new tenant (can be ttx's change) | 19:24 |
| clarkb | what would we roll back to? | 19:24 |
| clarkb | corvus: ++ | 19:24 |
| *** jamesmcarthur has joined #openstack-infra | 19:24 | |
| clarkb | corvus: I'm actually going to step away now because I never had that breakfast I told myself I would find | 19:24 |
| clarkb | but I can help with that after my brunch | 19:24 |
| corvus | clarkb: ok, i'll write the p-c change | 19:25 |
| corvus | then you can review when you're back from breakfast and i'm off to lunch :) | 19:25 |
| pabelanger | clarkb: I need to confirm, but they were booting this morning / yesterday. possible we uploaded latest DIBs in the last few hours | 19:25 |
| *** jamesmcarthur_ has joined #openstack-infra | 19:25 | |
| pabelanger | https://nb01.openstack.org/fedora-29-0000013924.log | 19:26 |
| pabelanger | think nb01 is out of space | 19:26 |
| pabelanger | and nb02 | 19:26 |
| pabelanger | and nb03 | 19:27 |
| pabelanger | I think something is wrong with the builders | 19:27 |
| *** jamesmcarthur_ has quit IRC | 19:27 | |
| pabelanger | /dev/mapper/main-nodepool 1008G 1008G 0 100% /opt/dib_cache/gentoo/portage/distfiles | 19:28 |
| pabelanger | clarkb: portdirect: both nb01 and nb02 have that^ | 19:28 |
| pabelanger | recent changes for gentoo? | 19:28 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config master: Temporarily add opendev/base-jobs as an untrusted-project https://review.openstack.org/633590 | 19:28 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config master: Remove opendev/base-jobs https://review.openstack.org/633591 | 19:28 |
| corvus | clarkb: ^ | 19:29 |
| *** jamesmcarthur has quit IRC | 19:29 | |
| *** hwoarang has quit IRC | 19:30 | |
| *** hwoarang has joined #openstack-infra | 19:30 | |
| clarkb | corvus: existing base jobs win because they come first in the project list right? | 19:30 |
| *** wolverineav has quit IRC | 19:30 | |
| corvus | clarkb: yes, but i'm not proposing we merge any changes to that repo before removing it | 19:31 |
| clarkb | ah | 19:31 |
| clarkb | purely to generate the keys then | 19:31 |
| *** wolverineav has joined #openstack-infra | 19:31 | |
| *** wolverineav has quit IRC | 19:31 | |
| corvus | clarkb: (in fact, because we haven't set 'shadow', it will be a config error) | 19:31 |
| *** wolverineav has joined #openstack-infra | 19:31 | |
| corvus | (so we won't be able to merge it) | 19:31 |
| *** dtantsur is now known as dtantsur|afk | 19:32 | |
| *** jamesmcarthur has joined #openstack-infra | 19:33 | |
| clarkb | pabelanger: fun | 19:33 |
| *** jamesmcarthur_ has joined #openstack-infra | 19:34 | |
| *** dave-mccowan has quit IRC | 19:34 | |
| prometheanfire | pabelanger: ya, not sure, it seems like clarkb found issues with fedora udev? | 19:34 |
| *** rfolco has joined #openstack-infra | 19:34 | |
| clarkb | prometheanfire: I think pabelanger is pointing out that portage is using 1TB of disk | 19:34 |
| clarkb | prometheanfire: and that is preventing us from building new images currently | 19:34 |
| prometheanfire | pabelanger: shouldn't have, distfiles should be shared for all builds (only systemd is used) | 19:34 |
| prometheanfire | 1T is crazy for that | 19:35 |
| clarkb | does it not prune itself overtime? | 19:35 |
| clarkb | could have been a slow leak | 19:35 |
| pabelanger | clarkb: sadly, I won't be able to debug much today, but figured I'd notify. Also, it seems inap might not be working for image uploads, it has been 2 weeks since we've successfully uploaded anything there | 19:35 |
| prometheanfire | slaanesh-zp00/GENTOO/portage/distfiles 36.8G 396G 36.8G /usr/portage/distfiles | 19:35 |
| prometheanfire | it doesn't normally, but even over a year on a desktop is just 100G | 19:36 |
| clarkb | prometheanfire: can we safely delete it and have it rebuild? I assume we can | 19:36 |
| prometheanfire | clarkb: iirc we only build one image type for gentoo (systemd), we could run eclean-dist | 19:37 |
| prometheanfire | and yes, cleaning is fine | 19:37 |
| prometheanfire | it's a local cache | 19:37 |
| *** smarcet has quit IRC | 19:37 | |
| prometheanfire | packages is similiar, just a binpkg instead of compiling every run | 19:37 |
| *** xek has quit IRC | 19:37 | |
| prometheanfire | slaanesh-zp00/GENTOO/portage/distfiles 36.8G 396G 36.8G /usr/portage/distfiles | 19:37 |
| *** jamesmcarthur has quit IRC | 19:37 | |
| prometheanfire | oops, repaste :| | 19:37 |
| *** xek has joined #openstack-infra | 19:37 | |
| clarkb | can we just rm ? | 19:37 |
| prometheanfire | I'm still interested as to why it's so large | 19:38 |
| prometheanfire | can you get some file structure info? | 19:38 |
| prometheanfire | and ya, just rm | 19:38 |
| prometheanfire | it seems odd that it'd use so much space | 19:40 |
| prometheanfire | my distfiles cache (when building my images and infra images both) is only 289MB | 19:41 |
| prometheanfire | and I have not cleaned | 19:41 |
| clarkb | I think pabelanger misread du output | 19:42 |
| clarkb | portage is doing lvm magic | 19:42 |
| *** sthussey has quit IRC | 19:42 | |
| clarkb | 843GB or so is just in the images on the builder and we are keeping far more than just 2 | 19:42 |
| prometheanfire | portage itself is? that's news to me :P | 19:42 |
| clarkb | prometheanfire: ya those paths show up in df as mounts | 19:42 |
| clarkb | but it is sharing the underlying storage I think | 19:43 |
| prometheanfire | ah, bind mounts? | 19:43 |
| pabelanger | possible, I only looked quickly | 19:43 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Add a table for attachments https://review.openstack.org/633411 | 19:43 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Add a WSME model for attachments https://review.openstack.org/633417 | 19:43 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Add support for retrieving attachment metadata https://review.openstack.org/633418 | 19:43 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard master: Add support for creating attachments https://review.openstack.org/633421 | 19:43 |
| clarkb | prometheanfire: ah yup those may be bind mounts from aborted builds | 19:44 |
| clarkb | we can start by cleaning those out | 19:44 |
| tobiash | evrardjp: commented on https://review.openstack.org/632979 | 19:46 |
| prometheanfire | one thing I think will be needed is a gentoo binhost (binary package mirror), the main issue with it is package signing (I don't think it exists for binpkgs yet) | 19:47 |
| prometheanfire | atm, I think having an OSA job publishing binpkgs for mirroring would be a good start, possibly integrate them into the image (so there's better trust), or set up something like https://wiki.gentoo.org/wiki/Binary_package_guide#SSH_binary_package_host | 19:48 |
| clarkb | other than the intermediate files which I have cleaned up we seem to have leaked files for at least ubuntu-bionic-0000000076 since january 3 | 19:51 |
| clarkb | Shrews: ^ any idea how that might happen? we can clean them up easily enough but may be worth understanding better | 19:51 |
| prometheanfire | right now I'm bind mounting between containers (in OSA) (end of file https://review.openstack.org/#/c/633092/3/vars/gentoo.yml ) | 19:52 |
| clarkb | debian-stretch-0000000070 gentoo-17-0-systemd-0000000058 opensuse-423-0000000060 also appear to be leaks on nb01 | 19:55 |
| prometheanfire | clarkb: let me know what you find | 20:00 |
| clarkb | centos-7-0000000078 fedora-29-0000000019 fedora-29-0000000030 (maybe fedora-29-0000000248?) gentoo-17-0-systemd-0000000073 opensuse-423-0000000082 ubuntu-bionic-0000000093 on nb02 | 20:00 |
| clarkb | Shrews: ^ I can clean those up after lunch if I don't hear a reason not to do so | 20:01 |
| clarkb | that should get builds running happily again | 20:01 |
| clarkb | actually why don't I go ahead and do the first couple on the list from each host to give us breathing room for the builds that are trying to run now | 20:02 |
| clarkb | that should give us plenty of remaining data if we want to debug after | 20:02 |
| prometheanfire | clarkb: inflating the gentoo images by 600-1000M, if binpkgs/distfiles are shipped in the image (to save on download/compiling) | 20:03 |
| prometheanfire | given that the nodepool gentoo image is 6G when I build it, may or may not be an issue | 20:03 |
| *** wolverineav has quit IRC | 20:05 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/nodepool master: Amazon EC2 driver https://review.openstack.org/535558 | 20:08 |
| *** yboaron has quit IRC | 20:09 | |
| openstackgerrit | Merged openstack-infra/zuul master: Use gearman client keepalive https://review.openstack.org/599573 | 20:09 |
| *** yboaron has joined #openstack-infra | 20:09 | |
| *** yboaron has quit IRC | 20:15 | |
| *** ldnunes has quit IRC | 20:15 | |
| *** efried has joined #openstack-infra | 20:24 | |
| mgagne | pabelanger: working on it (image upload at inap) | 20:26 |
| *** whoami-rajat has quit IRC | 20:35 | |
| Shrews | clarkb: not sure how that would happen. i don't have deep knowledge of dib | 20:39 |
| *** eernst has quit IRC | 20:41 | |
| *** wolverineav has joined #openstack-infra | 20:41 | |
| *** wolverineav has quit IRC | 20:41 | |
| *** wolverineav has joined #openstack-infra | 20:41 | |
| clarkb | Shrews: it is nodepool leaking the image files though | 20:42 |
| Shrews | clarkb: interesting. nothing off the top of my head | 20:42 |
| *** eernst has joined #openstack-infra | 20:43 | |
| *** eernst has quit IRC | 20:44 | |
| clarkb | ok I'll finish cleaning up the leaks | 20:49 |
| Shrews | clarkb: if you want, file a bug with what you found. busy with other things atm, so can't look at it myself right now | 20:51 |
| clarkb | #status log Cleaned up leaked images on nb01, nb02, and nb03 to free up disk space for dib. | 20:58 |
| openstackstatus | clarkb: finished logging | 20:58 |
| clarkb | On nb03 ubuntu-bionic-arm64-0000000055 was the lone leaked image | 20:58 |
| clarkb | I think builds should work now and once inap imgae uploads are working we should automatically clean up the older images that are in use there | 20:58 |
| *** eernst has joined #openstack-infra | 20:59 | |
| clarkb | infra-root anyone able to quickly review https://review.openstack.org/#/c/633590/ ? that will generate encryption keys for that project so that I can add secrets to my change to move the base jobs into that repo | 20:59 |
| *** eernst_ has joined #openstack-infra | 21:02 | |
| *** eernst has quit IRC | 21:02 | |
| *** eernst_ has quit IRC | 21:02 | |
| *** eernst has joined #openstack-infra | 21:02 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-zuul master: Allow for both global zuul web and whitelabeled tenant https://review.openstack.org/633586 | 21:04 |
| AJaeger | clarkb: done | 21:04 |
| clarkb | AJaeger: thank you | 21:05 |
| clarkb | tosky: still around? I think I've mostly dug out of my morning and am ready to figure out these release deletions | 21:08 |
| tosky | clarkb: hi! | 21:09 |
| clarkb | tosky: my first question was going to be is there any reason to fetch and save the old release artifacts? | 21:09 |
| clarkb | I think they are on tarballs.o.o already if you want to compare hashes /me looks | 21:09 |
| clarkb | https://tarballs.openstack.org/sahara/ seems to have files that line up | 21:10 |
| *** smarcet has joined #openstack-infra | 21:11 | |
| clarkb | tosky: but I'm ready to delete things when you are | 21:12 |
| tosky | clarkb: uhm, I'd say that it would be better to save that for historical reasons | 21:13 |
| clarkb | tosky: ya, I think the real question is if the files at https://tarballs.openstack.org/sahara/ match the contents of pypi | 21:13 |
| clarkb | if they do then we are already done | 21:14 |
| clarkb | if they don't then we should fetch and store somewhere | 21:14 |
| tosky | uhm, good question, I would need some time to download them and compare | 21:14 |
| clarkb | no worries, let me know if they check out | 21:14 |
| openstackgerrit | Merged openstack-infra/project-config master: Temporarily add opendev/base-jobs as an untrusted-project https://review.openstack.org/633590 | 21:18 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: WIP add provides/requires support https://review.openstack.org/633605 | 21:19 |
| *** smarcet has quit IRC | 21:23 | |
| clarkb | encrypt_secret.py is 404'ing on opendev/base-jobs pubkey url. Opening in the browser seems to do something funny too. Testing with project-config I get a 404 via curl as well | 21:23 |
| clarkb | https://zuul.openstack.org/api/key/opestack-infra/project-config.pub is that the right url? | 21:23 |
| corvus | clarkb: puppet hasn't run yet | 21:23 |
| clarkb | corvus: ya, but it should work with project-config now though | 21:24 |
| clarkb | after finding opendev didn't work I'm trying to make it work with project-config to make sure it all works | 21:24 |
| clarkb | aha | 21:25 |
| clarkb | I typoed openstack | 21:25 |
| openstackgerrit | Paul Vinciguerra proposed openstack-infra/zuul master: configloader.py: Not all jobs have an .updated attribute. https://review.openstack.org/633259 | 21:25 |
| clarkb | ok now to find the source materal for this secret | 21:25 |
| *** jamesmcarthur has joined #openstack-infra | 21:26 | |
| *** wolverineav has quit IRC | 21:26 | |
| *** wolverineav has joined #openstack-infra | 21:27 | |
| *** wolverineav has quit IRC | 21:28 | |
| *** wolverineav has joined #openstack-infra | 21:28 | |
| *** jtomasek has quit IRC | 21:29 | |
| clarkb | (it is in hiera if anyone is wondering) | 21:29 |
| *** jamesmcarthur_ has quit IRC | 21:29 | |
| *** panda is now known as panda|off | 21:35 | |
| openstackgerrit | Paul Vinciguerra proposed openstack-infra/zuul master: configloader.py: Not all jobs have an .updated attribute. https://review.openstack.org/633259 | 21:36 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-zuul master: Allow for both global zuul web and whitelabeled tenant https://review.openstack.org/633586 | 21:40 |
| pabelanger | clarkb: yes, but you need to use curl for it to render, at least under chrome. | 21:40 |
| pabelanger | I just get Fetching info... | 21:40 |
| clarkb | pabelanger: ya I ran into that too. Got it working with curl | 21:41 |
| pabelanger | clarkb: I think tristanC / ianw know more what is going on with chrome support. | 21:41 |
| pabelanger | we discussed it on friday in #zuul I believe | 21:41 |
| *** kgiusti has left #openstack-infra | 21:42 | |
| *** jamesmcarthur has quit IRC | 21:51 | |
| *** ijw has quit IRC | 21:53 | |
| *** jamesmcarthur has joined #openstack-infra | 21:53 | |
| *** panda|off has quit IRC | 21:54 | |
| *** jamesmcarthur_ has joined #openstack-infra | 21:54 | |
| *** panda has joined #openstack-infra | 21:56 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-zuul master: Allow for both global zuul web and whitelabeled tenant https://review.openstack.org/633586 | 21:56 |
| corvus | clarkb: key's there | 21:57 |
| tosky | clarkb: I checked and all the 2014.* sahara files (tar and whl) on pypi are available on tarballs.openstack.org | 21:58 |
| tosky | the opposite is not true | 21:58 |
| tosky | so I'd say that they can be removed from pypi | 21:58 |
| *** jamesmcarthur has quit IRC | 21:58 | |
| clarkb | corvus: remote: https://review.openstack.org/632785 Initial pass at global opendev base job set updated | 21:59 |
| clarkb | tosky: ok I will remove them from pypi shortly | 22:00 |
| *** eharney has quit IRC | 22:00 | |
| corvus | clarkb: we should +3 https://review.openstack.org/633591 | 22:00 |
| clarkb | corvus: done | 22:01 |
| corvus | oh, heh, it didn't match our ^openstack.* system-required regex, so it's in no pipelines, so zuul didn't review it. | 22:02 |
| *** jamesmcarthur_ has quit IRC | 22:02 | |
| corvus | er, wow that sentence requires too many footnotes | 22:02 |
| corvus | base-jobs didn't match our ^openstack.* system-required regex, so it's in no pipelines, so zuul didn't review 632785. | 22:03 |
| *** sdkae has quit IRC | 22:03 | |
| tosky | clarkb: thank you! | 22:04 |
| *** smarcet has joined #openstack-infra | 22:04 | |
| *** sdake has joined #openstack-infra | 22:06 | |
| clarkb | tosky: done | 22:07 |
| clarkb | tosky: please confirm it looks like you expect now | 22:07 |
| tosky | clarkb: it looks fine, thank you! Should that bogus entry named 0 be removed, or do you think that it should not cause any issue? | 22:09 |
| clarkb | tosky: I don't expect it to cause any issues so probably just leave it for now | 22:11 |
| tosky | oki | 22:11 |
| *** wolverineav has quit IRC | 22:12 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove opendev/base-jobs https://review.openstack.org/633591 | 22:12 |
| *** rcernin has joined #openstack-infra | 22:13 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-zuul master: Allow for both global zuul web and whitelabeled tenant https://review.openstack.org/633586 | 22:13 |
| *** wolverineav has joined #openstack-infra | 22:13 | |
| *** rcernin has quit IRC | 22:15 | |
| openstackgerrit | Paul Vinciguerra proposed openstack-infra/zuul master: configloader.py: Not all jobs have an .updated attribute. https://review.openstack.org/633259 | 22:15 |
| *** rcernin has joined #openstack-infra | 22:15 | |
| *** wolverineav has quit IRC | 22:17 | |
| clarkb | #status log Deleted old 2014.* Sahara releases from pypi so that modern releases are sorted properly. Old sahara releases are available on the tarballs server. | 22:18 |
| openstackstatus | clarkb: finished logging | 22:18 |
| clarkb | corvus: what is next for https://review.openstack.org/#/c/632785/ ? | 22:19 |
| corvus | clarkb: force-merge? | 22:20 |
| clarkb | ok wasn't sure based on your +1 instead of +2 if you thought there were other things weshould do | 22:20 |
| clarkb | I guess get another infra-root to look it over then force merge | 22:20 |
| corvus | clarkb: i'll just go set up the groups for that so i can +2 it :) | 22:20 |
| clarkb | corvus: oh its that problem :) | 22:21 |
| clarkb | pabelanger: mgagne fyi | 0000000091 | 0000000080 | inap-mtl01 | ubuntu-xenial | ubuntu-xenial-1548711121 | edccf954-e8bb-4f74-a5bc-77b0027754d4 | ready | 00:00:00:54 | that image just successfully uploaded | 22:21 |
| mgagne | good news then =) | 22:21 |
| corvus | clarkb: i added project-config-core | 22:21 |
| clarkb | corvus: ++ | 22:21 |
| *** jamesmcarthur has joined #openstack-infra | 22:27 | |
| *** jamesmcarthur has quit IRC | 22:31 | |
| *** ekultails has quit IRC | 22:32 | |
| *** efried1 has joined #openstack-infra | 22:32 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: WIP add provides/requires support https://review.openstack.org/633605 | 22:34 |
| *** efried has quit IRC | 22:34 | |
| *** efried1 is now known as efried | 22:34 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient master: Add a resource subcontroller for attachments https://review.openstack.org/633611 | 22:36 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient master: Display download links for files attached to stories https://review.openstack.org/633612 | 22:36 |
| *** e0ne has quit IRC | 22:36 | |
| *** wolverineav has joined #openstack-infra | 22:37 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient master: Make tags section heading consistent with others https://review.openstack.org/633613 | 22:37 |
| *** sdake has quit IRC | 22:45 | |
| *** jamesmcarthur has joined #openstack-infra | 22:45 | |
| *** xek has quit IRC | 22:46 | |
| *** sdake has joined #openstack-infra | 22:48 | |
| *** jamesmcarthur has quit IRC | 22:50 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-zuul master: Allow for both global zuul web and whitelabeled tenant https://review.openstack.org/633586 | 22:53 |
| clarkb | I think ^ is very close now | 22:54 |
| *** sdake has quit IRC | 22:57 | |
| pabelanger | clarkb: great, thanks | 22:58 |
| *** kjackal has quit IRC | 22:58 | |
| *** sdake has joined #openstack-infra | 22:59 | |
| *** sdake has quit IRC | 23:01 | |
| *** sdake has joined #openstack-infra | 23:02 | |
| *** cristoph has quit IRC | 23:07 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-zuul master: Allow for both global zuul web and whitelabeled tenant https://review.openstack.org/633586 | 23:09 |
| clarkb | It will get there eventually :) the testing we have is actualyl really good for this type of refactor which is nice | 23:09 |
| *** tkajinam has joined #openstack-infra | 23:09 | |
| *** mriedem has quit IRC | 23:15 | |
| *** smarcet has quit IRC | 23:27 | |
| *** armstrong has joined #openstack-infra | 23:28 | |
| *** slaweq has quit IRC | 23:28 | |
| clarkb | I think I've found more leaked images that my first pass missed | 23:28 |
| clarkb | I'm going to make another pass at cleaning those up. Basically I'm checking what we have on disk against dib-image-list. If not in dib-image-list but on disk then rm | 23:29 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: WIP add provides/requires support https://review.openstack.org/633605 | 23:35 |
| *** smarcet has joined #openstack-infra | 23:36 | |
| *** bnemec has quit IRC | 23:37 | |
| *** wolverineav has quit IRC | 23:38 | |
| *** wolverineav has joined #openstack-infra | 23:39 | |
| clarkb | thats odd looking at the logs for one of these images it did build successfully | 23:40 |
| clarkb | but then it seems to just disappear in the log like nodepool forgets about it | 23:41 |
| *** ianw is now known as ianw_pto | 23:42 | |
| clarkb | I'm going to clean them all up to get to a known point with disk available for future builds then we can monitor going forward I guess | 23:43 |
| *** yamamoto has joined #openstack-infra | 23:49 | |
| *** agopi is now known as agopi|brb | 23:52 | |
| *** agopi|brb is now known as agopi|off | 23:52 | |
| *** agopi|off has quit IRC | 23:53 | |
| *** eernst has quit IRC | 23:53 | |
| *** yamamoto has quit IRC | 23:54 | |
| *** wolverineav has quit IRC | 23:55 | |
| *** eernst has joined #openstack-infra | 23:58 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!