| *** wolverineav has quit IRC | 00:03 | |
| *** aeng has joined #openstack-infra | 00:06 | |
| *** thorst_ has joined #openstack-infra | 00:10 | |
| *** thorst_ has quit IRC | 00:12 | |
| *** Julien-zte has joined #openstack-infra | 00:14 | |
| *** sflanigan_afk has joined #openstack-infra | 00:18 | |
| *** wolverineav has joined #openstack-infra | 00:18 | |
| *** baoli has joined #openstack-infra | 00:22 | |
| *** baoli has quit IRC | 00:23 | |
| *** baoli has joined #openstack-infra | 00:23 | |
| *** wolverineav has quit IRC | 00:23 | |
| *** jamielennox is now known as jamielennox|away | 00:24 | |
| *** jlwhite has quit IRC | 00:28 | |
| *** jlwhite has joined #openstack-infra | 00:28 | |
| *** rvba has quit IRC | 00:28 | |
| *** rvba has joined #openstack-infra | 00:28 | |
| *** rvba has quit IRC | 00:29 | |
| *** rvba has joined #openstack-infra | 00:29 | |
| *** mdbooth has quit IRC | 00:29 | |
| *** ansiwen has quit IRC | 00:29 | |
| *** wolverineav has joined #openstack-infra | 00:29 | |
| *** nunchuck has quit IRC | 00:29 | |
| *** mdbooth has joined #openstack-infra | 00:30 | |
| *** ansiwen has joined #openstack-infra | 00:30 | |
| *** EricGonczer_ has quit IRC | 00:31 | |
| *** stakeda has joined #openstack-infra | 00:32 | |
| *** wolverineav has quit IRC | 00:33 | |
| *** aeng has quit IRC | 00:34 | |
| *** baoli has quit IRC | 00:38 | |
| *** wolverineav has joined #openstack-infra | 00:39 | |
| *** esikachev has joined #openstack-infra | 00:43 | |
| *** wolverineav has quit IRC | 00:44 | |
| *** jkilpatr has quit IRC | 00:45 | |
| *** esikachev has quit IRC | 00:47 | |
| *** wolverineav has joined #openstack-infra | 00:49 | |
| *** jamielennox|away is now known as jamielennox | 00:50 | |
| *** bmjen has quit IRC | 00:52 | |
| *** bmjen has joined #openstack-infra | 00:52 | |
| *** wolverineav has quit IRC | 00:54 | |
| *** wolverineav has joined #openstack-infra | 01:00 | |
| *** dimtruck is now known as zz_dimtruck | 01:02 | |
| *** cody-somerville has quit IRC | 01:02 | |
| *** aeng has joined #openstack-infra | 01:03 | |
| *** wolverineav has quit IRC | 01:04 | |
| *** mattmceuen has quit IRC | 01:05 | |
| *** mattmceuen has joined #openstack-infra | 01:06 | |
| *** zz_dimtruck is now known as dimtruck | 01:13 | |
| *** cody-somerville has joined #openstack-infra | 01:14 | |
| *** sileht has quit IRC | 01:19 | |
| *** zz_ja has quit IRC | 01:19 | |
| *** thorst_ has joined #openstack-infra | 01:19 | |
| *** thorst_ has quit IRC | 01:19 | |
| *** liujiong has joined #openstack-infra | 01:19 | |
| *** wolverineav has joined #openstack-infra | 01:20 | |
| *** cody-somerville has quit IRC | 01:21 | |
| *** zz_ja has joined #openstack-infra | 01:22 | |
| *** sileht has joined #openstack-infra | 01:22 | |
| *** eranrom has joined #openstack-infra | 01:22 | |
| openstackgerrit | tonytan4ever proposed openstack/os-testr master: Allow using ostestr with --ipdb option https://review.openstack.org/425354 | 01:22 |
|---|---|---|
| *** salv-orlando has joined #openstack-infra | 01:23 | |
| *** wolverineav has quit IRC | 01:24 | |
| *** eranrom has quit IRC | 01:27 | |
| *** salv-orlando has quit IRC | 01:27 | |
| *** esikachev has joined #openstack-infra | 01:31 | |
| *** yanyanhu has joined #openstack-infra | 01:35 | |
| *** esikachev has quit IRC | 01:36 | |
| *** wolverineav has joined #openstack-infra | 01:40 | |
| *** tuanluong has joined #openstack-infra | 01:41 | |
| *** gongysh has joined #openstack-infra | 01:41 | |
| *** mattmceuen has quit IRC | 01:42 | |
| *** wolverineav has quit IRC | 01:44 | |
| *** tphummel has quit IRC | 01:45 | |
| *** thorst_ has joined #openstack-infra | 01:45 | |
| *** thorst_ has quit IRC | 01:45 | |
| *** wolverineav has joined #openstack-infra | 01:50 | |
| *** tphummel has joined #openstack-infra | 01:51 | |
| *** wolverineav has quit IRC | 01:54 | |
| *** hongbin_ has joined #openstack-infra | 01:57 | |
| *** dargains has quit IRC | 01:59 | |
| *** aeng has quit IRC | 01:59 | |
| *** dargains has joined #openstack-infra | 02:00 | |
| *** dargains has quit IRC | 02:04 | |
| *** baoli has joined #openstack-infra | 02:04 | |
| openstackgerrit | Zhenguo Niu proposed openstack-infra/project-config master: Create mogan-ui plugin https://review.openstack.org/431352 | 02:04 |
| *** jamesdenton has quit IRC | 02:07 | |
| *** esikachev has joined #openstack-infra | 02:10 | |
| *** wolverineav has joined #openstack-infra | 02:10 | |
| *** dargains has joined #openstack-infra | 02:12 | |
| openstackgerrit | Zhenguo Niu proposed openstack-infra/project-config master: Add py35 job to senlin-dashboard https://review.openstack.org/432810 | 02:14 |
| *** rbuzatu has joined #openstack-infra | 02:14 | |
| *** aeng has joined #openstack-infra | 02:14 | |
| *** esikachev has quit IRC | 02:14 | |
| *** wolverineav has quit IRC | 02:15 | |
| *** esikachev has joined #openstack-infra | 02:15 | |
| *** rbuzatu has quit IRC | 02:18 | |
| *** thorst_ has joined #openstack-infra | 02:19 | |
| *** thorst_ has quit IRC | 02:19 | |
| *** esikachev has quit IRC | 02:20 | |
| *** wolverineav has joined #openstack-infra | 02:20 | |
| *** tphummel has quit IRC | 02:22 | |
| *** eranrom has joined #openstack-infra | 02:23 | |
| *** esikachev has joined #openstack-infra | 02:24 | |
| *** wolverineav has quit IRC | 02:25 | |
| *** eranrom has quit IRC | 02:27 | |
| *** nunchuck has joined #openstack-infra | 02:28 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add missing periodic ocata jobs https://review.openstack.org/432454 | 02:29 |
| openstackgerrit | Merged openstack-infra/project-config master: Sahara: Use normal python jobs https://review.openstack.org/429265 | 02:30 |
| *** deduped has joined #openstack-infra | 02:30 | |
| *** esikachev has quit IRC | 02:32 | |
| *** mixos has quit IRC | 02:33 | |
| *** nunchuck has quit IRC | 02:33 | |
| *** baoli has quit IRC | 02:33 | |
| *** sdake has joined #openstack-infra | 02:34 | |
| *** armax has joined #openstack-infra | 02:35 | |
| openstackgerrit | Merged openstack-infra/project-config master: Move OpenStack-Ansible os_tempest Centos to non-voting https://review.openstack.org/432783 | 02:39 |
| *** nunchuck has joined #openstack-infra | 02:39 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add experimental jobs for TripleO OOOQ Multinode scenario 2-5 https://review.openstack.org/431786 | 02:40 |
| openstackgerrit | Merged openstack-infra/project-config master: cleanup: Remove builders of old chef-jobs tests https://review.openstack.org/432760 | 02:40 |
| *** sdake_ has joined #openstack-infra | 02:40 | |
| *** wolverineav has joined #openstack-infra | 02:41 | |
| openstackgerrit | Merged openstack-infra/project-config master: Run magnum migration on non-doc changes https://review.openstack.org/431193 | 02:42 |
| *** sdake has quit IRC | 02:43 | |
| *** wolverineav has quit IRC | 02:45 | |
| openstackgerrit | Merged openstack-infra/project-config master: cleanup: remove publisher upload-swagger-draft https://review.openstack.org/432746 | 02:49 |
| *** wolverineav has joined #openstack-infra | 02:51 | |
| openstackgerrit | Merged openstack-infra/project-config master: tripleo: improve tacker service coverage https://review.openstack.org/432327 | 02:51 |
| openstackgerrit | Merged openstack-infra/project-config master: Add ceph tempest dsvm job to devstack experimental https://review.openstack.org/432340 | 02:52 |
| *** anilvenkata has joined #openstack-infra | 02:55 | |
| *** esikachev has joined #openstack-infra | 02:55 | |
| *** wolverineav has quit IRC | 02:55 | |
| openstackgerrit | Merged openstack-infra/project-config master: OSA: Gerritbot follows stable/ocata https://review.openstack.org/431843 | 02:57 |
| openstackgerrit | Merged openstack-infra/project-config master: Publish nova-dpm to pypi https://review.openstack.org/431467 | 02:58 |
| *** esikachev has quit IRC | 03:00 | |
| *** esikachev has joined #openstack-infra | 03:01 | |
| *** cody-somerville has joined #openstack-infra | 03:02 | |
| *** gouthamr has quit IRC | 03:03 | |
| *** esikachev has quit IRC | 03:06 | |
| *** tphummel has joined #openstack-infra | 03:10 | |
| *** EricGonczer_ has joined #openstack-infra | 03:15 | |
| *** cody-somerville has quit IRC | 03:16 | |
| *** sdake has joined #openstack-infra | 03:17 | |
| *** sdake_ has quit IRC | 03:18 | |
| *** thorst_ has joined #openstack-infra | 03:20 | |
| *** wolverineav has joined #openstack-infra | 03:20 | |
| *** rhallisey has quit IRC | 03:21 | |
| *** armax has quit IRC | 03:21 | |
| *** salv-orlando has joined #openstack-infra | 03:24 | |
| *** eranrom has joined #openstack-infra | 03:24 | |
| *** thorst_ has quit IRC | 03:25 | |
| *** wolverineav has quit IRC | 03:25 | |
| *** eranrom has quit IRC | 03:28 | |
| *** salv-orlando has quit IRC | 03:28 | |
| *** cody-somerville has joined #openstack-infra | 03:29 | |
| *** gongysh has quit IRC | 03:39 | |
| *** wolverineav has joined #openstack-infra | 03:40 | |
| *** sree has joined #openstack-infra | 03:41 | |
| *** wolverineav has quit IRC | 03:45 | |
| *** sdake has quit IRC | 03:49 | |
| *** wolverineav has joined #openstack-infra | 03:51 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Add diskimage-builder coverage jobs https://review.openstack.org/421658 | 03:55 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Add -nv coverage jobs, convert existing non-voting tests https://review.openstack.org/431783 | 03:55 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Move coverage jobs to non-voting gate jobs https://review.openstack.org/432836 | 03:55 |
| *** wolverineav has quit IRC | 03:55 | |
| *** gildub has quit IRC | 03:55 | |
| *** wolverineav has joined #openstack-infra | 04:01 | |
| *** wolverineav has quit IRC | 04:06 | |
| *** EricGonczer_ has quit IRC | 04:08 | |
| *** Julien-zte has quit IRC | 04:09 | |
| *** Julien-z_ has joined #openstack-infra | 04:10 | |
| *** psachin has joined #openstack-infra | 04:10 | |
| *** wolverineav has joined #openstack-infra | 04:11 | |
| *** dargains has quit IRC | 04:13 | |
| *** dargains has joined #openstack-infra | 04:14 | |
| *** Julien-z_ has quit IRC | 04:15 | |
| *** Julien-zte has joined #openstack-infra | 04:15 | |
| *** wolverineav has quit IRC | 04:16 | |
| *** cody-somerville has quit IRC | 04:16 | |
| *** dargains has quit IRC | 04:18 | |
| *** thorst_ has joined #openstack-infra | 04:21 | |
| *** wolverineav has joined #openstack-infra | 04:22 | |
| *** eranrom has joined #openstack-infra | 04:25 | |
| *** adriant has quit IRC | 04:25 | |
| *** thorst_ has quit IRC | 04:26 | |
| *** wolverineav has quit IRC | 04:26 | |
| *** eranrom has quit IRC | 04:29 | |
| *** csomerville has joined #openstack-infra | 04:32 | |
| *** asselin_ has joined #openstack-infra | 04:32 | |
| *** openstackgerrit has quit IRC | 04:32 | |
| *** esikachev has joined #openstack-infra | 04:33 | |
| *** asselin has quit IRC | 04:34 | |
| *** esikachev has quit IRC | 04:38 | |
| *** wolverineav has joined #openstack-infra | 04:42 | |
| *** esikachev has joined #openstack-infra | 04:43 | |
| *** faizy has joined #openstack-infra | 04:46 | |
| *** wolverineav has quit IRC | 04:46 | |
| *** esikachev has quit IRC | 04:47 | |
| *** salv-orlando has joined #openstack-infra | 04:54 | |
| *** salv-orlando has quit IRC | 04:59 | |
| *** stakeda has quit IRC | 05:03 | |
| *** wolverineav has joined #openstack-infra | 05:11 | |
| *** wolverineav has quit IRC | 05:16 | |
| *** openstackgerrit has joined #openstack-infra | 05:18 | |
| openstackgerrit | Hongbin Lu proposed openstack-infra/project-config master: Zun: Move gate_hook.sh to the right location https://review.openstack.org/432844 | 05:18 |
| openstackgerrit | Tomasz Trębski proposed openstack-infra/project-config master: Enable coverage jobs for monasca projects https://review.openstack.org/432845 | 05:18 |
| *** mtreinish has quit IRC | 05:20 | |
| *** hongbin_ has quit IRC | 05:21 | |
| *** eranrom has joined #openstack-infra | 05:25 | |
| *** jaosorior has joined #openstack-infra | 05:28 | |
| *** eranrom has quit IRC | 05:30 | |
| *** mtreinish has joined #openstack-infra | 05:31 | |
| *** wolverineav has joined #openstack-infra | 05:32 | |
| *** Rockyg has quit IRC | 05:32 | |
| *** amotoki has joined #openstack-infra | 05:33 | |
| openstackgerrit | Hongbin Lu proposed openstack-infra/project-config master: Zun: Move gate_hook.sh to the right location https://review.openstack.org/432844 | 05:33 |
| *** amotoki has quit IRC | 05:36 | |
| *** wolverineav has quit IRC | 05:37 | |
| openstackgerrit | Merged openstack-infra/project-config master: Rack/cerberus: Remove python-db jobs https://review.openstack.org/432175 | 05:43 |
| *** sree_ has joined #openstack-infra | 05:46 | |
| *** sree_ is now known as Guest55678 | 05:47 | |
| *** amotoki has joined #openstack-infra | 05:48 | |
| *** sree has quit IRC | 05:50 | |
| *** rbuzatu has joined #openstack-infra | 05:51 | |
| *** wolverineav has joined #openstack-infra | 05:52 | |
| *** asselin has joined #openstack-infra | 05:53 | |
| *** asselin_ has quit IRC | 05:55 | |
| *** salv-orlando has joined #openstack-infra | 05:55 | |
| *** rbuzatu has quit IRC | 05:56 | |
| *** wolverineav has quit IRC | 05:57 | |
| *** saibarspeis has joined #openstack-infra | 05:58 | |
| *** saibarspeis has quit IRC | 05:59 | |
| *** salv-orlando has quit IRC | 06:00 | |
| *** Jeffrey4l__ has joined #openstack-infra | 06:01 | |
| *** wolverineav has joined #openstack-infra | 06:02 | |
| *** Jeffrey4l_ has quit IRC | 06:04 | |
| *** jrist has joined #openstack-infra | 06:04 | |
| *** pgadiya has joined #openstack-infra | 06:07 | |
| *** wolverineav has quit IRC | 06:07 | |
| *** tphummel has quit IRC | 06:07 | |
| *** amotoki has quit IRC | 06:10 | |
| *** wolverineav has joined #openstack-infra | 06:12 | |
| *** wolverineav has quit IRC | 06:17 | |
| *** sandanar has joined #openstack-infra | 06:18 | |
| openstackgerrit | Jaivish Kothari(janonymous) proposed openstack-infra/project-config master: [wip]Adding non-voting job for fullstack tests on ssl https://review.openstack.org/432862 | 06:20 |
| *** nunchuck has quit IRC | 06:20 | |
| *** deduped has quit IRC | 06:20 | |
| *** thorst_ has joined #openstack-infra | 06:22 | |
| *** amotoki has joined #openstack-infra | 06:23 | |
| *** thorst_ has quit IRC | 06:26 | |
| *** rbuzatu has joined #openstack-infra | 06:29 | |
| openstackgerrit | wangxiyuan proposed openstack-infra/project-config master: Specify {backend} for zaqar grenade CI https://review.openstack.org/432868 | 06:30 |
| *** faizy has quit IRC | 06:31 | |
| *** faizy has joined #openstack-infra | 06:31 | |
| *** wolverineav has joined #openstack-infra | 06:32 | |
| *** psachin has quit IRC | 06:33 | |
| *** faizy has quit IRC | 06:35 | |
| *** faizy has joined #openstack-infra | 06:36 | |
| *** faizy has quit IRC | 06:37 | |
| *** wolverineav has quit IRC | 06:37 | |
| *** faizy has joined #openstack-infra | 06:38 | |
| *** sree has joined #openstack-infra | 06:40 | |
| *** faizy has quit IRC | 06:41 | |
| *** faizy has joined #openstack-infra | 06:41 | |
| *** itisha has quit IRC | 06:42 | |
| *** Guest55678 has quit IRC | 06:43 | |
| *** salv-orlando has joined #openstack-infra | 06:43 | |
| *** tnovacik has joined #openstack-infra | 06:50 | |
| *** psachin has joined #openstack-infra | 06:50 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE - testing TLS everywhere job with RabbitMQ https://review.openstack.org/426307 | 06:50 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE - testing TLS everywhere job https://review.openstack.org/426863 | 06:51 |
| *** wolverineav has joined #openstack-infra | 06:52 | |
| *** psachin has quit IRC | 06:54 | |
| *** psachin has joined #openstack-infra | 06:54 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE - testing TLS everywhere job with RabbitMQ https://review.openstack.org/426307 | 06:55 |
| *** wolverineav has quit IRC | 06:57 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE - testing TLS everywhere job with RabbitMQ https://review.openstack.org/426307 | 06:59 |
| *** salv-orl_ has joined #openstack-infra | 07:00 | |
| *** eranrom has joined #openstack-infra | 07:02 | |
| *** nmathew has joined #openstack-infra | 07:02 | |
| openstackgerrit | Jaivish Kothari(janonymous) proposed openstack-infra/project-config master: [wip]Adding non-voting job for fullstack tests on ssl https://review.openstack.org/432862 | 07:02 |
| *** wolverineav has joined #openstack-infra | 07:03 | |
| *** salv-orlando has quit IRC | 07:03 | |
| AJaeger | ianw: thanks for the reviews! did you see my comment on your coverage change? Let's inform openstack-dev about the change - and if nobody speaks up, we can merge. Also, see https://review.openstack.org/432845 for another coverage job. | 07:04 |
| *** wolverineav has quit IRC | 07:08 | |
| *** kjackal_ has joined #openstack-infra | 07:10 | |
| openstackgerrit | Jaivish Kothari(janonymous) proposed openstack-infra/project-config master: [wip]Adding non-voting job for fullstack tests on ssl https://review.openstack.org/432862 | 07:13 |
| *** markus_z has joined #openstack-infra | 07:14 | |
| *** rcernin has joined #openstack-infra | 07:15 | |
| *** kornicameister has quit IRC | 07:16 | |
| *** claudiub has joined #openstack-infra | 07:16 | |
| *** rbuzatu has quit IRC | 07:21 | |
| *** rbuzatu has joined #openstack-infra | 07:21 | |
| *** wolverineav has joined #openstack-infra | 07:23 | |
| *** sree has quit IRC | 07:24 | |
| *** seongsoocho has joined #openstack-infra | 07:24 | |
| *** sree has joined #openstack-infra | 07:24 | |
| *** wolverineav has quit IRC | 07:28 | |
| *** rbuzatu has quit IRC | 07:28 | |
| *** tesseract has joined #openstack-infra | 07:28 | |
| *** tesseract is now known as tesseract-RH | 07:29 | |
| *** rbuzatu has joined #openstack-infra | 07:30 | |
| *** slaweq_ has joined #openstack-infra | 07:31 | |
| *** tnovacik has quit IRC | 07:32 | |
| *** kornicameister has joined #openstack-infra | 07:32 | |
| *** wolverineav has joined #openstack-infra | 07:33 | |
| *** yamamoto has quit IRC | 07:34 | |
| *** wolverineav has quit IRC | 07:38 | |
| *** ijw has joined #openstack-infra | 07:38 | |
| *** cody-somerville has joined #openstack-infra | 07:39 | |
| *** cody-somerville has joined #openstack-infra | 07:39 | |
| *** esikachev has joined #openstack-infra | 07:39 | |
| *** nadya has joined #openstack-infra | 07:41 | |
| *** csomerville has quit IRC | 07:41 | |
| *** nadya has quit IRC | 07:42 | |
| *** nadya has joined #openstack-infra | 07:42 | |
| *** wolverineav has joined #openstack-infra | 07:43 | |
| *** ralonsoh has joined #openstack-infra | 07:46 | |
| *** wolverineav has quit IRC | 07:48 | |
| *** pcaruana has joined #openstack-infra | 07:49 | |
| *** sree has quit IRC | 08:02 | |
| *** wolverineav has joined #openstack-infra | 08:03 | |
| *** kjackal_ has quit IRC | 08:03 | |
| *** rbuzatu has quit IRC | 08:04 | |
| *** rbuzatu has joined #openstack-infra | 08:04 | |
| *** gildub has joined #openstack-infra | 08:06 | |
| *** sree has joined #openstack-infra | 08:08 | |
| *** wolverineav has quit IRC | 08:08 | |
| *** rbuzatu has quit IRC | 08:09 | |
| *** rbuzatu has joined #openstack-infra | 08:11 | |
| *** wolverineav has joined #openstack-infra | 08:14 | |
| *** matrohon has joined #openstack-infra | 08:14 | |
| *** hwoarang has quit IRC | 08:16 | |
| *** yamamoto has joined #openstack-infra | 08:17 | |
| *** yamamoto has quit IRC | 08:17 | |
| *** yamamoto has joined #openstack-infra | 08:18 | |
| *** wolverineav has quit IRC | 08:18 | |
| openstackgerrit | Zhenguo Niu proposed openstack-infra/project-config master: Add coverage job to Mogan project https://review.openstack.org/432909 | 08:18 |
| *** faizy has quit IRC | 08:19 | |
| *** faizy has joined #openstack-infra | 08:20 | |
| *** aviau has quit IRC | 08:20 | |
| *** aviau has joined #openstack-infra | 08:20 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config master: Normalize projects.yaml https://review.openstack.org/432911 | 08:22 |
| *** thorst_ has joined #openstack-infra | 08:22 | |
| *** yuval has joined #openstack-infra | 08:23 | |
| *** wolverineav has joined #openstack-infra | 08:24 | |
| *** claudiub has quit IRC | 08:24 | |
| *** thorst_ has quit IRC | 08:27 | |
| *** wolverineav has quit IRC | 08:28 | |
| ianw | AJaeger: sounds good, i'll do that | 08:29 |
| *** dingyichen has quit IRC | 08:29 | |
| *** furlongm has quit IRC | 08:30 | |
| *** askb has quit IRC | 08:30 | |
| *** furlongm has joined #openstack-infra | 08:31 | |
| *** abregman has joined #openstack-infra | 08:34 | |
| *** wolverineav has joined #openstack-infra | 08:44 | |
| *** prazumovsky has joined #openstack-infra | 08:44 | |
| prazumovsky | hello! Can someone help me with core-group for recently added repo (repo added by me) | 08:45 |
| *** jpich has joined #openstack-infra | 08:45 | |
| prazumovsky | We have no core reviewers in fuel-ccp-designate repo, we need someone in core-group for this repo | 08:46 |
| *** hughhalf has quit IRC | 08:46 | |
| *** hughhalf has joined #openstack-infra | 08:47 | |
| *** ijw has quit IRC | 08:48 | |
| kevinbenton | masayukig: hey, can you take a look at https://review.openstack.org/#/c/432825/ again? | 08:49 |
| *** wolverineav has quit IRC | 08:49 | |
| kevinbenton | masayukig: i made an adjustment that will protect it in a multiple IP case as well | 08:49 |
| masayukig | kevinbenton : sure, just a minute :) | 08:50 |
| masayukig | kevinbenton : but you are in the wrong channel :) | 08:50 |
| *** hashar has joined #openstack-infra | 08:50 | |
| *** kashyap has left #openstack-infra | 08:52 | |
| *** claudiub has joined #openstack-infra | 08:52 | |
| *** abregman has quit IRC | 08:56 | |
| *** wolverineav has joined #openstack-infra | 08:56 | |
| prazumovsky | hey?:) | 08:57 |
| *** zzzeek has quit IRC | 09:00 | |
| *** zzzeek has joined #openstack-infra | 09:00 | |
| *** shardy has joined #openstack-infra | 09:00 | |
| *** wolverineav has quit IRC | 09:01 | |
| *** rbuzatu has quit IRC | 09:01 | |
| *** faizy has quit IRC | 09:01 | |
| *** faizy has joined #openstack-infra | 09:02 | |
| *** rbuzatu has joined #openstack-infra | 09:02 | |
| *** sshnaidm|off is now known as sshnaidm | 09:02 | |
| *** john-davidge has joined #openstack-infra | 09:03 | |
| *** rossella_s has joined #openstack-infra | 09:03 | |
| *** rossella_s has quit IRC | 09:04 | |
| *** rossella_s has joined #openstack-infra | 09:05 | |
| *** tqtran has joined #openstack-infra | 09:05 | |
| *** tqtran has quit IRC | 09:09 | |
| *** rbuzatu has quit IRC | 09:10 | |
| *** gongysh has joined #openstack-infra | 09:10 | |
| *** faizy_ has joined #openstack-infra | 09:11 | |
| *** dtantsur|afk is now known as dtantsur | 09:15 | |
| *** faizy has quit IRC | 09:15 | |
| AJaeger | yolanda: could you please review these project-config changes: https://review.openstack.org/432784 (enables pypy/compare-coverage for test-setup), https://review.openstack.org/#/c/431120/ (python-db for vmware-nsx change) ? | 09:15 |
| *** acabot has joined #openstack-infra | 09:16 | |
| *** markmcd has quit IRC | 09:17 | |
| *** rbuzatu has joined #openstack-infra | 09:18 | |
| *** shu-mutou is now known as shu-mutou-AWAY | 09:23 | |
| *** faizy_ has quit IRC | 09:24 | |
| *** shardy has quit IRC | 09:24 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE - testing TLS everywhere job (placement fix) https://review.openstack.org/432939 | 09:27 |
| *** rfolco has joined #openstack-infra | 09:27 | |
| *** derekh has joined #openstack-infra | 09:32 | |
| *** lucas-afk is now known as lucasagomes | 09:36 | |
| yolanda | hi, good morning | 09:36 |
| yolanda | arrived a bit later today, going to start now with the review queue | 09:36 |
| *** eliqiao has quit IRC | 09:37 | |
| *** eliqiao has joined #openstack-infra | 09:40 | |
| *** eliqiao has quit IRC | 09:40 | |
| *** monester has joined #openstack-infra | 09:40 | |
| *** faizy has joined #openstack-infra | 09:42 | |
| *** monester has quit IRC | 09:43 | |
| *** monester has joined #openstack-infra | 09:43 | |
| *** panda|off is now known as panda | 09:44 | |
| *** Julien-zte has quit IRC | 09:45 | |
| *** wolverineav has joined #openstack-infra | 09:46 | |
| *** faizy has quit IRC | 09:47 | |
| *** hwoarang has joined #openstack-infra | 09:47 | |
| *** jamielennox is now known as jamielennox|away | 09:48 | |
| *** tosky has joined #openstack-infra | 09:49 | |
| *** e0ne has joined #openstack-infra | 09:52 | |
| *** e0ne has quit IRC | 09:54 | |
| openstackgerrit | Sergey Kulanov proposed openstack-infra/jeepyb master: Allow to customize values in .gitreview file https://review.openstack.org/410137 | 09:54 |
| *** wolverineav has quit IRC | 09:56 | |
| *** e0ne has joined #openstack-infra | 09:58 | |
| *** amotoki has quit IRC | 09:59 | |
| *** denisra has joined #openstack-infra | 10:00 | |
| *** amotoki has joined #openstack-infra | 10:01 | |
| *** e0ne has quit IRC | 10:02 | |
| *** nadya has quit IRC | 10:03 | |
| *** askb has joined #openstack-infra | 10:05 | |
| *** e0ne has joined #openstack-infra | 10:07 | |
| *** wolverineav has joined #openstack-infra | 10:07 | |
| *** e0ne has quit IRC | 10:09 | |
| *** ldnunes has joined #openstack-infra | 10:10 | |
| openstackgerrit | Merged openstack-infra/project-config master: Update pypy and compare-coverage for test-setup https://review.openstack.org/432784 | 10:10 |
| *** wolverineav has quit IRC | 10:11 | |
| *** e0ne has joined #openstack-infra | 10:13 | |
| *** yanyanhu has quit IRC | 10:14 | |
| *** wolverineav has joined #openstack-infra | 10:14 | |
| *** e0ne_ has joined #openstack-infra | 10:15 | |
| *** tqtran has joined #openstack-infra | 10:15 | |
| *** e0ne has quit IRC | 10:15 | |
| *** wolverineav has quit IRC | 10:15 | |
| *** dims_ has quit IRC | 10:16 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add py35 job to senlin-dashboard https://review.openstack.org/432810 | 10:17 |
| *** rossella_s has quit IRC | 10:17 | |
| *** sdake has joined #openstack-infra | 10:18 | |
| openstackgerrit | Merged openstack-infra/project-config master: Specify {backend} for zaqar grenade CI https://review.openstack.org/432868 | 10:18 |
| *** askb has quit IRC | 10:19 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add -nv coverage jobs, convert existing non-voting tests https://review.openstack.org/431783 | 10:19 |
| openstackgerrit | Merged openstack-infra/project-config master: Add diskimage-builder coverage jobs https://review.openstack.org/421658 | 10:19 |
| *** rossella_s has joined #openstack-infra | 10:20 | |
| *** tqtran has quit IRC | 10:20 | |
| *** e0ne_ has quit IRC | 10:21 | |
| *** chem has joined #openstack-infra | 10:23 | |
| *** dims has joined #openstack-infra | 10:23 | |
| *** thorst_ has joined #openstack-infra | 10:23 | |
| *** liujiong has quit IRC | 10:24 | |
| *** jamielennox|away is now known as jamielennox | 10:24 | |
| *** salv-orl_ has quit IRC | 10:24 | |
| *** salv-orlando has joined #openstack-infra | 10:25 | |
| *** e0ne has joined #openstack-infra | 10:25 | |
| *** wolverineav has joined #openstack-infra | 10:26 | |
| *** e0ne has quit IRC | 10:27 | |
| *** thorst_ has quit IRC | 10:28 | |
| frickler | If someone could review this devstack-gate patch of mine, that would be nice, has been pending for 5 weeks now: https://review.openstack.org/416985 | 10:29 |
| *** ralonsoh_ has joined #openstack-infra | 10:30 | |
| *** wolverineav has quit IRC | 10:30 | |
| *** chem has quit IRC | 10:30 | |
| *** chem has joined #openstack-infra | 10:30 | |
| *** chem has left #openstack-infra | 10:31 | |
| *** wolverineav has joined #openstack-infra | 10:31 | |
| *** ralonsoh_ has quit IRC | 10:32 | |
| *** e0ne has joined #openstack-infra | 10:32 | |
| *** ralonsoh_ has joined #openstack-infra | 10:32 | |
| *** ralonsoh has quit IRC | 10:33 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add Jenkins smoke test for networking-vpp https://review.openstack.org/427031 | 10:34 |
| openstackgerrit | Merged openstack-infra/project-config master: Add python-tricircleclient repository https://review.openstack.org/426419 | 10:34 |
| openstackgerrit | Merged openstack-infra/project-config master: Add Ceph MDS charm interface https://review.openstack.org/429834 | 10:34 |
| openstackgerrit | Merged openstack-infra/project-config master: Move mistral dashboard py35 to voting https://review.openstack.org/431385 | 10:35 |
| *** wolverineav has quit IRC | 10:35 | |
| *** hashar has quit IRC | 10:37 | |
| *** hashar has joined #openstack-infra | 10:38 | |
| *** e0ne has quit IRC | 10:41 | |
| *** dimtruck is now known as zz_dimtruck | 10:42 | |
| *** e0ne has joined #openstack-infra | 10:45 | |
| *** tuanluong has quit IRC | 10:49 | |
| *** claudiub|2 has joined #openstack-infra | 10:49 | |
| *** claudiub has quit IRC | 10:50 | |
| *** e0ne has quit IRC | 10:51 | |
| *** rossella_s has quit IRC | 10:53 | |
| *** electrofelix has joined #openstack-infra | 10:53 | |
| *** rossella_s has joined #openstack-infra | 10:54 | |
| *** nadya has joined #openstack-infra | 10:54 | |
| *** sambetts|afk is now known as sambetts | 10:58 | |
| *** portdirect is now known as portdirect_ | 10:58 | |
| *** portdirect_ is now known as portdirec | 10:59 | |
| *** ccamacho has quit IRC | 10:59 | |
| *** portdirec is now known as portdirect_ | 10:59 | |
| *** portdirect_ is now known as portdirect | 10:59 | |
| *** portdirect has joined #openstack-infra | 11:00 | |
| *** ccamacho has joined #openstack-infra | 11:02 | |
| openstackgerrit | dongwenjuan proposed openstack-infra/project-config master: reorganize the vitrage job definition https://review.openstack.org/432990 | 11:06 |
| openstackgerrit | Zhenguo Niu proposed openstack-infra/project-config master: Add coverage job to Mogan project https://review.openstack.org/432909 | 11:07 |
| *** psachin has quit IRC | 11:08 | |
| panda | pabelanger: looks like we have problems with the mirrors here http://logs.openstack.org/38/432938/1/check/gate-puppet-tripleo-puppet-syntax-3-centos-7/9a8426b/console.html#_2017-02-13_09_25_57_339724 | 11:11 |
| *** shardy has joined #openstack-infra | 11:16 | |
| *** ldnunes_ has joined #openstack-infra | 11:16 | |
| *** hughhalf has quit IRC | 11:16 | |
| *** openstackgerrit has quit IRC | 11:18 | |
| *** hughhalf has joined #openstack-infra | 11:18 | |
| *** sdague has joined #openstack-infra | 11:19 | |
| *** ldnunes has quit IRC | 11:20 | |
| *** wolverineav has joined #openstack-infra | 11:22 | |
| *** psachin has joined #openstack-infra | 11:25 | |
| *** Julien-zte has joined #openstack-infra | 11:25 | |
| *** ldnunes_ has quit IRC | 11:25 | |
| *** wolverineav has quit IRC | 11:27 | |
| *** pgadiya has quit IRC | 11:31 | |
| *** nicolasbock has joined #openstack-infra | 11:31 | |
| *** e0ne has joined #openstack-infra | 11:34 | |
| *** wolverineav has joined #openstack-infra | 11:34 | |
| *** psachin has quit IRC | 11:39 | |
| *** wolverineav has quit IRC | 11:39 | |
| *** nmathew has quit IRC | 11:40 | |
| *** e0ne has quit IRC | 11:45 | |
| *** e0ne has joined #openstack-infra | 11:47 | |
| *** e0ne has quit IRC | 11:48 | |
| *** e0ne has joined #openstack-infra | 11:48 | |
| *** e0ne has quit IRC | 11:48 | |
| *** e0ne has joined #openstack-infra | 11:49 | |
| *** e0ne has quit IRC | 11:49 | |
| *** e0ne has joined #openstack-infra | 11:50 | |
| *** e0ne has quit IRC | 11:50 | |
| *** e0ne has joined #openstack-infra | 11:50 | |
| *** e0ne has quit IRC | 11:51 | |
| *** e0ne has joined #openstack-infra | 11:51 | |
| *** jkilpatr has joined #openstack-infra | 11:51 | |
| *** e0ne has quit IRC | 11:52 | |
| *** gongysh has quit IRC | 11:52 | |
| *** ralonsoh__ has joined #openstack-infra | 11:52 | |
| *** claudiub|2 has quit IRC | 11:53 | |
| *** openstackgerrit has joined #openstack-infra | 11:54 | |
| openstackgerrit | Tomasz Trębski proposed openstack-infra/project-config master: Move log-api tempest gate to voting stage https://review.openstack.org/433018 | 11:54 |
| *** psilvad has quit IRC | 11:54 | |
| *** ralonsoh_ has quit IRC | 11:55 | |
| *** askb has joined #openstack-infra | 12:00 | |
| *** ccamacho has quit IRC | 12:02 | |
| openstackgerrit | Isaac Beckman proposed openstack-infra/project-config master: cleanup: remove gate-openstack-chef-repo builders https://review.openstack.org/432753 | 12:03 |
| *** sree_ has joined #openstack-infra | 12:03 | |
| *** ccamacho has joined #openstack-infra | 12:03 | |
| *** sree_ is now known as Guest75829 | 12:04 | |
| *** wolverineav has joined #openstack-infra | 12:04 | |
| *** dave-mccowan has joined #openstack-infra | 12:05 | |
| *** sree has quit IRC | 12:05 | |
| *** askb has quit IRC | 12:08 | |
| *** esikachev has quit IRC | 12:08 | |
| *** wolverineav has quit IRC | 12:09 | |
| *** rbuzatu has quit IRC | 12:10 | |
| jaosorior | Hey guys, mirror.regionone.osic-cloud1.openstack.org seems to be unreachable; is this a known issue? | 12:11 |
| *** psachin has joined #openstack-infra | 12:12 | |
| *** wolverineav has joined #openstack-infra | 12:14 | |
| *** gildub has quit IRC | 12:15 | |
| *** tqtran has joined #openstack-infra | 12:17 | |
| *** wolverineav has quit IRC | 12:18 | |
| *** isaacb has joined #openstack-infra | 12:20 | |
| *** isaacb_ has joined #openstack-infra | 12:20 | |
| *** isaacb_ has left #openstack-infra | 12:20 | |
| *** tqtran has quit IRC | 12:22 | |
| *** dims has quit IRC | 12:23 | |
| *** vsaienko has joined #openstack-infra | 12:26 | |
| *** dave-mcc_ has joined #openstack-infra | 12:28 | |
| *** eroux has joined #openstack-infra | 12:30 | |
| *** dave-mccowan has quit IRC | 12:30 | |
| *** rbuzatu has joined #openstack-infra | 12:32 | |
| *** wolverineav has joined #openstack-infra | 12:33 | |
| *** sdake_ has joined #openstack-infra | 12:36 | |
| *** eroux has quit IRC | 12:37 | |
| *** wolverineav has quit IRC | 12:38 | |
| *** eroux has joined #openstack-infra | 12:38 | |
| *** sdake has quit IRC | 12:39 | |
| prazumovsky | hi all! | 12:42 |
| prazumovsky | ping:) | 12:42 |
| *** sdake_ is now known as sdake | 12:48 | |
| *** thorst_ has joined #openstack-infra | 12:48 | |
| *** Madasi has quit IRC | 12:51 | |
| *** rhallisey has joined #openstack-infra | 12:51 | |
| *** lucasagomes is now known as lucas-hungry | 12:53 | |
| *** wolverineav has joined #openstack-infra | 12:53 | |
| *** wolverineav has quit IRC | 12:58 | |
| *** gouthamr has joined #openstack-infra | 12:58 | |
| *** russellb has joined #openstack-infra | 13:01 | |
| *** russellb has quit IRC | 13:01 | |
| *** russellb has joined #openstack-infra | 13:01 | |
| *** EricGonczer_ has joined #openstack-infra | 13:03 | |
| *** vsaienko has quit IRC | 13:03 | |
| *** wolverineav has joined #openstack-infra | 13:03 | |
| openstackgerrit | Russell Bryant proposed openstack-infra/irc-meetings master: Remove networking-ovn meeting. https://review.openstack.org/433046 | 13:05 |
| *** wolverineav has quit IRC | 13:08 | |
| *** armax has joined #openstack-infra | 13:08 | |
| *** esikachev has joined #openstack-infra | 13:09 | |
| *** shardy is now known as shardy_lunch | 13:10 | |
| *** vsaienko has joined #openstack-infra | 13:11 | |
| *** nmathew has joined #openstack-infra | 13:12 | |
| *** ociuhandu has quit IRC | 13:13 | |
| *** nmathew- has joined #openstack-infra | 13:14 | |
| *** dprince has joined #openstack-infra | 13:15 | |
| *** rlandy has joined #openstack-infra | 13:15 | |
| *** trown|outtypewww is now known as trown | 13:16 | |
| *** nmathew has quit IRC | 13:18 | |
| *** rfolco has quit IRC | 13:18 | |
| *** jaypipes has joined #openstack-infra | 13:20 | |
| *** dims has joined #openstack-infra | 13:20 | |
| *** eharney has quit IRC | 13:27 | |
| *** vsaienko has quit IRC | 13:27 | |
| *** salv-orlando has quit IRC | 13:29 | |
| *** timothyb89_ has joined #openstack-infra | 13:29 | |
| *** mikal_ has joined #openstack-infra | 13:30 | |
| *** rosmaita_ has joined #openstack-infra | 13:30 | |
| *** tristanC_ has joined #openstack-infra | 13:30 | |
| *** dotplus_ has joined #openstack-infra | 13:30 | |
| *** dotplus_ has quit IRC | 13:30 | |
| *** dotplus_ has joined #openstack-infra | 13:30 | |
| *** doffm_ has joined #openstack-infra | 13:31 | |
| *** fabo_ has joined #openstack-infra | 13:31 | |
| *** mikap has joined #openstack-infra | 13:31 | |
| *** doffm has quit IRC | 13:32 | |
| *** khappone has quit IRC | 13:32 | |
| *** dotplus has quit IRC | 13:32 | |
| *** mikal has quit IRC | 13:32 | |
| *** mika has quit IRC | 13:32 | |
| *** niedbalski has quit IRC | 13:32 | |
| *** dolphm has quit IRC | 13:32 | |
| *** hyakuhei has quit IRC | 13:32 | |
| *** hashar has quit IRC | 13:32 | |
| *** timothyb89 has quit IRC | 13:32 | |
| *** kevinbenton has quit IRC | 13:32 | |
| *** rosmaita has quit IRC | 13:32 | |
| *** tristanC has quit IRC | 13:32 | |
| *** fabo has quit IRC | 13:32 | |
| *** mikap is now known as mika | 13:32 | |
| *** dolphm_ has joined #openstack-infra | 13:32 | |
| *** niedbalski has joined #openstack-infra | 13:32 | |
| *** hyakuhei has joined #openstack-infra | 13:32 | |
| *** hyakuhei has quit IRC | 13:32 | |
| *** hyakuhei has joined #openstack-infra | 13:32 | |
| *** hyakuhei has quit IRC | 13:32 | |
| *** hyakuhei has joined #openstack-infra | 13:32 | |
| *** rfolco has joined #openstack-infra | 13:32 | |
| *** hashar has joined #openstack-infra | 13:32 | |
| *** wolverineav has joined #openstack-infra | 13:33 | |
| *** dolphm_ is now known as dolphm | 13:33 | |
| *** jheroux has joined #openstack-infra | 13:34 | |
| *** rosmaita_ is now known as rosmaita | 13:34 | |
| *** kevinbenton has joined #openstack-infra | 13:35 | |
| *** vsaienko has joined #openstack-infra | 13:35 | |
| *** mdrabe has joined #openstack-infra | 13:36 | |
| *** wolverineav has quit IRC | 13:37 | |
| EmilienM | pabelanger: hey, it seems like mirror.regionone.osic-cloud1.openstack.org is unreachable | 13:39 |
| *** yamamoto has quit IRC | 13:39 | |
| EmilienM | jaosorior: have you tried recheck? I just tried and could reach it | 13:39 |
| *** yamamoto has joined #openstack-infra | 13:41 | |
| jaosorior | EmilienM: I tried 10 min ago and it failed again | 13:42 |
| *** ccamacho is now known as ccamacho|lunch | 13:42 | |
| *** dprince has quit IRC | 13:42 | |
| EmilienM | pabelanger, jaosorior: http://logs.openstack.org/15/432515/1/check/gate-puppet-tripleo-puppet-syntax-4-centos-7/8bf8f48/console.html#_2017-02-13_12_18_33_438648 | 13:43 |
| EmilienM | sounds like some repodata | 13:43 |
| *** dims_ has joined #openstack-infra | 13:43 | |
| *** dims has quit IRC | 13:45 | |
| cloudnull | jaosorior: pabelanger: EmilienM: the osic mirror is mirror.regionone.osic-cloud1.openstack.org idk why this follows a different pattern but it is. | 13:45 |
| cloudnull | we've been tracking the mirrors here https://github.com/openstack/openstack-ansible/blob/master/scripts/fastest-infra-wheel-mirror.py#L32-L39 when gating our OSA jobs | 13:45 |
| cloudnull | seems like osic is an odd one when compared to the others. | 13:46 |
| *** yamamoto has quit IRC | 13:47 | |
| odyssey4me | cloudnull well, that's only actually used when *not* in the gate... the intent it to help people doing development | 13:47 |
| sdake | hey fellas | 13:47 |
| *** sbezverk has joined #openstack-infra | 13:47 | |
| sdake | have a problem | 13:47 |
| cloudnull | ah. its early . | 13:48 |
| sdake | getting the following gate failure: http://logs.openstack.org/56/431156/26/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-multi-nv/a62b417/console.html#_2017-02-13_13_13_13_653725 | 13:48 |
| sdake | oh right cloudnull - ppl may not be here | 13:48 |
| sdake | looks like EmilienM has already run into this | 13:48 |
| cloudnull | oh that was re odyssey4me | 13:48 |
| sdake | and jaosorior | 13:48 |
| *** nadya has quit IRC | 13:48 | |
| cloudnull | regarding gate failures. ¯\_(ツ)_/¯ | 13:48 |
| sdake | the above gate fialure looks like an osic cloud issue | 13:48 |
| * sdake giggles at cloudnull | 13:49 | |
| *** salv-orlando has joined #openstack-infra | 13:49 | |
| *** gordc has joined #openstack-infra | 13:49 | |
| sdake | well it sound slike its some kind of osic problem at minimum | 13:49 |
| *** mtanino has joined #openstack-infra | 13:49 | |
| sdake | I think we will have to wait a few hours for the west coast to wake up :) | 13:49 |
| sdake | pabelanger in the meantime, if you can take a look at our specific gate failure and see if it matches the scrollback reported, would appreciate it :) | 13:49 |
| sbezverk | sdake: it is not just kolla-kubernetes failing but kolla-ansible jobs as well | 13:49 |
| sdake | sbezverk yup its a systemic issue | 13:50 |
| sdake | sbezverk for us on the west coast, this is the "get the kids to school hour" | 13:50 |
| cloudnull | odyssey4me: were we being hit by the gate failures earlier ? | 13:50 |
| sdake | sbezverk and some of the infra peeps are on the west coast | 13:50 |
| cloudnull | I just suspected the mirror script would fire off and give us another one should one be done. | 13:50 |
| odyssey4me | cloudnull I haven't really been focusing on that. Maybe andymccr has seen something? | 13:50 |
| sdake | even east coast time its only 9 | 13:50 |
| sdake | odyssey4me i am pretty sure we all have the same failure | 13:51 |
| cloudnull | but in looking at the code it only runs if no pip.conf was present. | 13:51 |
| odyssey4me | cloudnull that script only executes outside of nodepool | 13:51 |
| *** ociuhandu has joined #openstack-infra | 13:51 | |
| odyssey4me | yep, if there's a pip.conf (which infra lays down) then that is preferred | 13:51 |
| *** ssbarnea has joined #openstack-infra | 13:51 | |
| *** esikache1 has joined #openstack-infra | 13:51 | |
| cloudnull | yup, so ignore me :) | 13:51 |
| sdake | Jeffrey4l__ may be we can debug this all together and give infra a head start :) | 13:52 |
| *** dizquierdo has joined #openstack-infra | 13:52 | |
| sbezverk | sdake: there is not much to debug, if the mirror is down ;) | 13:53 |
| sdake | cloudnull do you ahve a log to show | 13:53 |
| sdake | ya 404 on the mirror here cloudnull in kolla-ansible and kolla-kubernetes on bindep installation | 13:53 |
| sdake | (for osic cloud) | 13:53 |
| Jeffrey4l__ | i am checking the logs. | 13:53 |
| sbezverk | http://logs.openstack.org/56/431156/26/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-multi-nv/a62b417/console.html#_2017-02-13_13_13_13_650889 | 13:53 |
| sdake | sbezverk I was asking for the openstack-ansible logs to see if its the same failiure | 13:54 |
| *** esikachev has quit IRC | 13:54 | |
| odyssey4me | well, http://mirror.regionone.osic-cloud1.openstack.org is responding | 13:55 |
| cloudnull | ++ it is | 13:55 |
| sdake | EmilienM has same issue | 13:55 |
| odyssey4me | the wheel mirror appears to be responding | 13:55 |
| *** monester has quit IRC | 13:55 | |
| odyssey4me | it may just be the epel mirror | 13:55 |
| EmilienM | yeah but it's maybe a cached repodata or something | 13:55 |
| EmilienM | or just because it's Monday | 13:56 |
| sdake | monday funday :) | 13:56 |
| sdake | EmilienM fwiw the gate was working like a champ sunday :) | 13:56 |
| odyssey4me | the file in that log doesn't exist: http://mirror.regionone.osic-cloud1.openstack.org/epel/7/x86_64/repodata/ | 13:56 |
| sdake | and osic as well | 13:56 |
| cloudnull | so maybe the mirrors are out of sync | 13:56 |
| cloudnull | because monday | 13:56 |
| sdake | cloudnull mirror sync problems happen | 13:57 |
| *** ralonsoh__ is now known as ralonsoh | 13:57 | |
| cloudnull | EmilienM: you may be on to something there :) | 13:57 |
| sdake | cloudnull i know infra was giong to look at fixing that | 13:57 |
| sdake | cloudnull not sure if that got fixed | 13:57 |
| *** ihrachys has joined #openstack-infra | 13:57 | |
| odyssey4me | looks the same to me http://mirror.dfw.rax.openstack.org/epel/7/x86_64/repodata/ | 13:57 |
| sdake | cloudnull on a different topic, we are having a corss project collaboration day | 13:57 |
| sdake | its less a day and more a 1 hour session | 13:57 |
| odyssey4me | perhaps somehow the cache is stale? | 13:57 |
| sdake | i dont know why openstack-ansible wasn't in the tag line, however, anyone is welcome | 13:58 |
| sdake | odyssey4me wednesday 9am at the PTGs | 13:58 |
| sdake | for 1 hour | 13:59 |
| sdake | not sure if it will end up going all day - if it does sadly I wont be there as i am leaving at 11:40 | 13:59 |
| *** eharney has joined #openstack-infra | 13:59 | |
| *** amotoki has quit IRC | 13:59 | |
| odyssey4me | sdake probably because OSA is scheduled Mon/Tue as it's considered cross-project | 13:59 |
| odyssey4me | the scheduling could be better | 13:59 |
| sdake | odyssey4me ya agreed the scheduling is a nightmare for the ptg | 13:59 |
| sdake | odyssey4me this is cross-project around kuryr I think | 13:59 |
| sdake | moemnt let me find the mail | 13:59 |
| *** gongysh has joined #openstack-infra | 14:00 | |
| odyssey4me | anyway, not really for this channel | 14:00 |
| sdake | right - i'll go topenstack-ansibleon that topic | 14:01 |
| *** Goneri has joined #openstack-infra | 14:01 | |
| EmilienM | where are images stored now? it was on nodepool.openstack.org before | 14:02 |
| *** wolverineav has joined #openstack-infra | 14:02 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/jenkins-job-builder master: Add a publisher for MQTT https://review.openstack.org/427765 | 14:02 |
| openstackgerrit | Merged openstack-infra/odsreg master: Django 1.8+ compatibility https://review.openstack.org/412369 | 14:03 |
| *** eharney has quit IRC | 14:03 | |
| *** dave-mcc_ has quit IRC | 14:03 | |
| *** slaweq_ has quit IRC | 14:04 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Remove networking-ovn meeting. https://review.openstack.org/433046 | 14:04 |
| *** shardy_lunch is now known as shardy | 14:04 | |
| *** beagles is now known as beagles-mtg | 14:05 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: Installs openssh packages that try to workaround sshd start problem https://review.openstack.org/430688 | 14:05 |
| *** dave-mcc_ has joined #openstack-infra | 14:06 | |
| *** wolverineav has quit IRC | 14:06 | |
| *** amotoki has joined #openstack-infra | 14:08 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: Deploy novajoin in the TLS-everywhere job https://review.openstack.org/427481 | 14:09 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE - testing TLS everywhere job (novajoin) https://review.openstack.org/427482 | 14:09 |
| *** lucas-hungry is now known as lucasagomes | 14:10 | |
| *** dprince has joined #openstack-infra | 14:10 | |
| *** wolverineav has joined #openstack-infra | 14:12 | |
| *** srwilkers has joined #openstack-infra | 14:13 | |
| *** matt-borland has joined #openstack-infra | 14:13 | |
| *** EricGonczer_ has quit IRC | 14:14 | |
| *** pradk has joined #openstack-infra | 14:15 | |
| *** eharney has joined #openstack-infra | 14:16 | |
| *** wolverineav has quit IRC | 14:16 | |
| *** zz_dimtruck is now known as dimtruck | 14:18 | |
| *** hongbin_ has joined #openstack-infra | 14:19 | |
| dmsimard | Is there a particular script somewhere that does the log collection for devstack ? I'm probably not looking in the right place. | 14:21 |
| *** ccamacho|lunch is now known as ccamacho | 14:23 | |
| sdake | cloudnull gate still busted for you? | 14:24 |
| cloudnull | idk that it was ever busted for us? andymccr ? | 14:24 |
| cloudnull | i was just trying to help | 14:24 |
| andymccr | sdake: cloudnull: the only thing that broke our gate was the jinja2 req bump | 14:25 |
| sdake | cloudnull interesting - who was it busted for? | 14:25 |
| sdake | andymccr ya we had that happen too | 14:25 |
| andymccr | as far as im aware at least! | 14:25 |
| andymccr | although its still broken for some roles :( | 14:25 |
| andymccr | but working on that now | 14:25 |
| cloudnull | EmilienM: jaosorior | 14:25 |
| *** hongbin__ has joined #openstack-infra | 14:25 | |
| sdake | cloudnull thanks I thought it was broken for you if it works for you kolla and tripleo may be doing it wrong :) | 14:25 |
| cloudnull | likely :) | 14:26 |
| sdake | EmilienM mind providing a log of your gate jobs where itf fails? | 14:26 |
| * cloudnull is a helper | 14:26 | |
| sdake | or jaosorior? | 14:26 |
| jaosorior | sdake cloudnull: will verify | 14:26 |
| sdake | cloudnull me too :) | 14:26 |
| cloudnull | :) | 14:26 |
| sdake | cloudnull Jeffrey4l__ and sbezverk ar eactually8 sorting this problem out | 14:26 |
| *** dotplus_ is now known as dotplus | 14:27 | |
| sdake | or atleast Jeffrey4l__ (he is our release liason and we are trying to branch) | 14:27 |
| *** mriedem has joined #openstack-infra | 14:27 | |
| sdake | and we aren't going to branch on a cratered gate | 14:27 |
| sdake | jaosorior our gate job craters on bindep installation - although I'm not sure it is related to the epel mirror | 14:28 |
| *** hongbin_ has quit IRC | 14:29 | |
| jaosorior | sdake, cloudnull: still broken | 14:29 |
| jaosorior | EmilienM: ^ | 14:29 |
| *** gongysh has quit IRC | 14:29 | |
| jaosorior | sdake: same issue for us; fails on bindep install | 14:30 |
| sdake | jaosorior only on osic cloud? | 14:30 |
| jaosorior | sdake: that seems to be the case | 14:30 |
| *** beagles-mtg has quit IRC | 14:30 | |
| *** dargains has joined #openstack-infra | 14:31 | |
| *** nmathew- has quit IRC | 14:31 | |
| *** slaweq_ has joined #openstack-infra | 14:32 | |
| *** dargains_ has joined #openstack-infra | 14:33 | |
| *** tonytan4ever has joined #openstack-infra | 14:34 | |
| *** dargains has quit IRC | 14:37 | |
| *** xarses has quit IRC | 14:37 | |
| *** xarses has joined #openstack-infra | 14:37 | |
| *** amotoki has quit IRC | 14:37 | |
| *** jamielennox is now known as jamielennox|away | 14:39 | |
| *** Guest69121 has joined #openstack-infra | 14:39 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: Deploy novajoin in the TLS-everywhere job https://review.openstack.org/427481 | 14:39 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE - testing TLS everywhere job (novajoin) https://review.openstack.org/427482 | 14:39 |
| *** dizquierdo has quit IRC | 14:40 | |
| *** abregman has joined #openstack-infra | 14:41 | |
| *** wolverineav has joined #openstack-infra | 14:42 | |
| *** eroux has quit IRC | 14:43 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: tripleo: improve congress service coverage https://review.openstack.org/433092 | 14:43 |
| *** yamamoto has joined #openstack-infra | 14:44 | |
| *** eroux has joined #openstack-infra | 14:45 | |
| *** vgnbkr has quit IRC | 14:45 | |
| *** rbuzatu has quit IRC | 14:45 | |
| *** wolverineav has quit IRC | 14:46 | |
| *** HenryG has quit IRC | 14:46 | |
| openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate master: Upgrade devstack-tools to 0.2.0 https://review.openstack.org/433093 | 14:47 |
| *** vgnbkr has joined #openstack-infra | 14:48 | |
| *** yamamoto has quit IRC | 14:49 | |
| *** HenryG has joined #openstack-infra | 14:49 | |
| *** pcaruana has quit IRC | 14:50 | |
| *** rbuzatu has joined #openstack-infra | 14:52 | |
| *** nadya has joined #openstack-infra | 14:52 | |
| *** rossella_s has quit IRC | 14:53 | |
| *** baoli has joined #openstack-infra | 14:53 | |
| *** kjackal_ has joined #openstack-infra | 14:53 | |
| *** rossella_s has joined #openstack-infra | 14:53 | |
| *** claudiub|2 has joined #openstack-infra | 14:54 | |
| *** pcaruana has joined #openstack-infra | 14:55 | |
| *** prazumovsky has quit IRC | 14:55 | |
| *** zerick_ is now known as zerick | 14:56 | |
| *** jcoufal has joined #openstack-infra | 14:56 | |
| *** zzelle_hello is now known as ZZelle | 14:57 | |
| *** dave-mcc_ has quit IRC | 14:57 | |
| *** esikache1 has quit IRC | 15:00 | |
| *** anilvenkata has quit IRC | 15:00 | |
| *** wolverineav has joined #openstack-infra | 15:02 | |
| *** esikache1 has joined #openstack-infra | 15:05 | |
| *** dimtruck is now known as zz_dimtruck | 15:06 | |
| *** beekneemech is now known as bnemec | 15:08 | |
| *** burgerk has joined #openstack-infra | 15:09 | |
| *** esikache1 has quit IRC | 15:09 | |
| *** anilvenkata has joined #openstack-infra | 15:10 | |
| *** salv-orlando has quit IRC | 15:11 | |
| *** salv-orlando has joined #openstack-infra | 15:11 | |
| *** Guest69121 is now known as beagles | 15:11 | |
| *** dave-mccowan has joined #openstack-infra | 15:12 | |
| *** jcoufal_ has joined #openstack-infra | 15:12 | |
| *** fabo_ is now known as fabo | 15:12 | |
| *** esikache1 has joined #openstack-infra | 15:13 | |
| *** dane-fichter has joined #openstack-infra | 15:13 | |
| *** kgiusti has joined #openstack-infra | 15:13 | |
| dane-fichter | anyone have an idea / example of how to add an image file to a tempest gate? | 15:14 |
| *** jcoufal has quit IRC | 15:15 | |
| pabelanger | issue is likely EPE7 that is broken, not mirror.regionone.osic-cloud1.openstack.org | 15:15 |
| dane-fichter | as in, an image that'd be referenced for upload tests by CONF.scenario.img_dir and CONF.scenario.img_file | 15:15 |
| pabelanger | will dig into the logs shortly | 15:15 |
| *** rkukura has quit IRC | 15:17 | |
| *** hongbin__ has quit IRC | 15:17 | |
| *** esikache1 has quit IRC | 15:17 | |
| *** tlian has joined #openstack-infra | 15:18 | |
| pabelanger | looks like some issues with AFS this morning | 15:18 |
| pabelanger | Creating new volume 536870968 on replication site afs02.dfw.openstack.org: Failed to create the ro volume: : Input/output error | 15:18 |
| pabelanger | VOLSER: release could not be completed | 15:18 |
| *** tqtran has joined #openstack-infra | 15:19 | |
| sbezverk | hello, any news for fixing systematic gate failure for kolla-kubernetes jobs? | 15:20 |
| *** esikache1 has joined #openstack-infra | 15:22 | |
| *** rkukura has joined #openstack-infra | 15:23 | |
| *** tqtran has quit IRC | 15:24 | |
| *** zz_dimtruck is now known as dimtruck | 15:25 | |
| *** gcb has joined #openstack-infra | 15:26 | |
| gcb | Does anyone know about 'OpenStack Clean'? another robot ? | 15:27 |
| pabelanger | gcb: where do you see that? | 15:28 |
| gcb | pabelanger, https://review.openstack.org/#/c/415053/ | 15:28 |
| gcb | ah , is a person ? | 15:29 |
| pabelanger | looks like it | 15:29 |
| *** esikache1 has quit IRC | 15:29 | |
| *** dave-mccowan has quit IRC | 15:29 | |
| *** Guest75829 has quit IRC | 15:30 | |
| gcb | agree | 15:30 |
| *** esikache1 has joined #openstack-infra | 15:30 | |
| *** sree has joined #openstack-infra | 15:31 | |
| *** EricGonczer_ has joined #openstack-infra | 15:32 | |
| *** dave-mccowan has joined #openstack-infra | 15:33 | |
| *** rajinir_ has joined #openstack-infra | 15:34 | |
| *** sree has quit IRC | 15:35 | |
| *** mriedem1 has joined #openstack-infra | 15:35 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/jeepyb master: Make a few less github API calls https://review.openstack.org/433143 | 15:36 |
| *** sandanar has quit IRC | 15:37 | |
| *** mriedem has quit IRC | 15:38 | |
| EmilienM | pabelanger: I confirm we have some mirror sync issue | 15:38 |
| EmilienM | pabelanger: I saw a lot of reports from different teams, including ansible/puppet/tripleo groups | 15:38 |
| *** mkarpin has joined #openstack-infra | 15:39 | |
| sbezverk | EmilienM: kolla kubernetes as well | 15:39 |
| *** esikache1 has quit IRC | 15:39 | |
| pabelanger | yes, afs02.dfw.openstack.org likely needs a fsck | 15:39 |
| pabelanger | which is affecting AFS | 15:39 |
| pabelanger | I am going to see if I can disable replication to afs02.dfw.o.o for now | 15:41 |
| openstackgerrit | Radoslav Gerganov proposed openstack-infra/project-config master: Add new project: vmware-vspc https://review.openstack.org/433144 | 15:41 |
| mordred | pabelanger: while you're doing that, I'm going to start on the fsck | 15:42 |
| pabelanger | vos release mirror.epel -localauth worked | 15:43 |
| openstackgerrit | Radoslav Gerganov proposed openstack-infra/project-config master: Add new project: vmware-vspc https://review.openstack.org/433144 | 15:43 |
| pabelanger | however | 15:43 |
| Jeffrey4l__ | could anyone check this? http://logs.openstack.org/36/433136/1/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-multi-nv/20fe2b9/console.html#_2017-02-13_15_29_20_458919 , seems there are something wrong in mirror repo. | 15:43 |
| Jeffrey4l__ | sbezverk, ^^ | 15:43 |
| pabelanger | vos remsite afs02.dfw.openstack.org a mirror.epel | 15:43 |
| Jeffrey4l__ | "Not using downloaded repomd.xml because it is older than what we have:" | 15:43 |
| pabelanger | Deleting the replication site for volume 536870967 ...Removed replication site afs02.dfw.openstack.org /vicepa for volume mirror.epel | 15:44 |
| pabelanger | so, I don't think we want to do that | 15:44 |
| *** Jeffrey4l__ is now known as Jeffrey4l | 15:44 | |
| pabelanger | for all volumes | 15:44 |
| mordred | nope. I agree :) | 15:44 |
| *** psachin has quit IRC | 15:44 | |
| Jeffrey4l | seems it is cased by wrong timestampe? | 15:44 |
| *** timothyb89_ is now known as timothyb89 | 15:45 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Split up min-ready requests to 1 node per request https://review.openstack.org/433127 | 15:45 |
| mordred | pabelanger: fsck is running | 15:45 |
| *** rbrndt_ has joined #openstack-infra | 15:45 | |
| pabelanger | mordred: thanks | 15:45 |
| pabelanger | re-creating mirror.epel on afs02 too | 15:45 |
| mordred | pabelanger: the fileserver is stopped on afs02, fwiw | 15:46 |
| pabelanger | okay | 15:46 |
| pabelanger | #status notice We are currently investigating an issue with our AFS mirrors which is causing some projects jobs to fail. We are working to correct the issue. | 15:48 |
| openstackstatus | pabelanger: sending notice | 15:48 |
| *** mriedem1 is now known as mriedem | 15:48 | |
| *** sree has joined #openstack-infra | 15:49 | |
| -openstackstatus- NOTICE: We are currently investigating an issue with our AFS mirrors which is causing some projects jobs to fail. We are working to correct the issue. | 15:49 | |
| mordred | pabelanger: fsck done and volume mounted | 15:49 |
| mordred | (and fileserver started) | 15:50 |
| sdake | hey peeps, thanks for fixing the gating problem :) | 15:50 |
| sdake | i'll leave ya to your work | 15:50 |
| *** anilvenkata has quit IRC | 15:50 | |
| *** marst has quit IRC | 15:51 | |
| openstackstatus | pabelanger: finished sending notice | 15:51 |
| *** salv-orlando has quit IRC | 15:54 | |
| *** yamahata has joined #openstack-infra | 15:59 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Split up min-ready requests to 1 node per request https://review.openstack.org/433127 | 16:00 |
| *** abregman has quit IRC | 16:00 | |
| *** marst has joined #openstack-infra | 16:01 | |
| *** yuval has quit IRC | 16:02 | |
| *** jamesdenton has joined #openstack-infra | 16:04 | |
| *** jcoufal_ has quit IRC | 16:05 | |
| *** jcoufal has joined #openstack-infra | 16:06 | |
| pabelanger | mordred: ack | 16:06 |
| *** esikache1 has joined #openstack-infra | 16:07 | |
| sdague | mordred: how long after something is on pypi is it in our mirrors? | 16:08 |
| *** dtantsur is now known as dtantsur|mtg | 16:08 | |
| mordred | sdague: usually between 15 and 30 minutes depending on how lucky you get with timing | 16:08 |
| *** fguillot has joined #openstack-infra | 16:09 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: toci_gate_test: allow to run upgrade jobs for scenarios https://review.openstack.org/426370 | 16:09 |
| pabelanger | pypi is also affected by AFS issue | 16:09 |
| sdague | mordred: ok, I'll do a recheck, I guess I jumped the gun on the devstack-tools release | 16:09 |
| sdague | pabelanger: oh, there is an issue? | 16:09 |
| pabelanger | ya | 16:09 |
| mordred | sdague: afs issue this morning - working through it | 16:09 |
| pabelanger | we cannot release volumes, so data is getting stale quickly | 16:10 |
| notmyname | who is "OpenStack Clean", and how do I buy this bot a beer? | 16:10 |
| pabelanger | thought it was a user, but can dig into gerrit logs in a few | 16:11 |
| mordred | notmyname: I believe we just learned about our new friend OpenStack Clean this morning | 16:11 |
| notmyname | pabelanger: no idea. I'm just assuming a bot | 16:11 |
| notmyname | mordred: fun! | 16:11 |
| *** Julien-z_ has joined #openstack-infra | 16:11 | |
| *** Julien-zte has quit IRC | 16:12 | |
| notmyname | mordred: what other projects were blessed? I've got https://review.openstack.org/#/c/414881/ | 16:12 |
| notmyname | ah. looks like all the others with the same topic, at least | 16:13 |
| *** abregman has joined #openstack-infra | 16:13 | |
| openstackgerrit | Jørgen Edelbo proposed openstack-infra/git-review master: Add option to create upstream tracking branch https://review.openstack.org/433154 | 16:13 |
| *** spzala has joined #openstack-infra | 16:13 | |
| *** ociuhandu has quit IRC | 16:14 | |
| mordred | notmyname: I feel like this may be the beginnings of the "I can script the gerrit api" wars | 16:16 |
| *** mhickey has joined #openstack-infra | 16:17 | |
| mordred | notmyname: I mean, we've gotten almost 6 years of use out of it before battling bots, so that seems like a good run, right? | 16:17 |
| mordred | pabelanger: you gonna do another vos release? or want me to? | 16:18 |
| fungi | pabelanger: mordred: sorry, wasn't paying attention. was this related to my heads-up from friday (in here and in the infra status log) about cinder maintenance potentially impacting afs02.dfw over the weekend? | 16:18 |
| *** rcernin has quit IRC | 16:19 | |
| mordred | fungi: yup | 16:19 |
| fungi | oh, cool. so at least we knew it was coming | 16:19 |
| notmyname | mordred in one year: https://aahabershaw.files.wordpress.com/2013/07/a7ecff38c0e90bebd66b8e925fcba895c5cbd7fc.jpg "I remember the gerrit bot wars. Tell me again how difficult your open source project is" | 16:19 |
| mordred | fungi: volume shifter to read-only - afs became unhappy. since then I've fsck'd and remounted rewrite | 16:19 |
| *** dims_ has quit IRC | 16:19 | |
| mordred | notmyname: ++ | 16:19 |
| *** ramishra has quit IRC | 16:19 | |
| fungi | sounds about like the usual impact from the provider taking cinder volumes out of service temporarily | 16:19 |
| mordred | fungi: we have not yet starter rerunning release | 16:20 |
| fungi | k | 16:20 |
| *** pcaruana has quit IRC | 16:20 | |
| pabelanger | currently adding mirror.epel back to afs02 | 16:20 |
| mordred | ++ | 16:21 |
| *** esikache1 has quit IRC | 16:21 | |
| *** shardy has quit IRC | 16:21 | |
| *** armax has quit IRC | 16:21 | |
| *** shardy has joined #openstack-infra | 16:22 | |
| *** sree has quit IRC | 16:23 | |
| *** sree has joined #openstack-infra | 16:23 | |
| *** esikache1 has joined #openstack-infra | 16:24 | |
| *** dims has joined #openstack-infra | 16:24 | |
| *** tqtran has joined #openstack-infra | 16:25 | |
| *** dargains_ has quit IRC | 16:25 | |
| *** spzala has quit IRC | 16:26 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Split up min-ready requests to 1 node per request https://review.openstack.org/433127 | 16:27 |
| openstackgerrit | Yolande Amate proposed openstack-infra/jenkins-job-builder master: Update EnvInject to make use of convert_mapping_to_xml https://review.openstack.org/408801 | 16:27 |
| jeblair | mordred: need any help? | 16:27 |
| *** knangia has joined #openstack-infra | 16:27 | |
| *** sree has quit IRC | 16:28 | |
| *** jcoufal_ has joined #openstack-infra | 16:28 | |
| *** esikache1 has quit IRC | 16:29 | |
| openstackgerrit | Chris Holcombe proposed openstack-infra/project-config master: Add Ceph base charm layer https://review.openstack.org/429837 | 16:29 |
| *** esikache1 has joined #openstack-infra | 16:30 | |
| *** jcoufal has quit IRC | 16:30 | |
| *** srwilkers has quit IRC | 16:31 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Installs openssh packages that try to workaround sshd start problem https://review.openstack.org/430688 | 16:32 |
| *** matrohon has quit IRC | 16:32 | |
| *** pcaruana has joined #openstack-infra | 16:34 | |
| *** jcoufal has joined #openstack-infra | 16:34 | |
| mordred | jeblair: I believe pabelanger is on it | 16:34 |
| *** esikache1 has quit IRC | 16:34 | |
| *** e0ne has joined #openstack-infra | 16:35 | |
| jeblair | mordred, pabelanger: thanks, let me know if you need anything | 16:35 |
| pabelanger | mordred: jeblair: yes, data is still replicating to afs02 for mirror.epel, other volumes are still locked however. jeblair if you can peak at mirror.pypi, I'm currently waiting to see if the lock we release itself | 16:36 |
| *** jcoufal_ has quit IRC | 16:37 | |
| *** abregman is now known as abregman|afk | 16:38 | |
| *** nadya has quit IRC | 16:39 | |
| fungi | i didn't follow scrollback closely... are we currently serving from afs01.dfw instead of the broken afs02.dfw? | 16:40 |
| *** ociuhandu has joined #openstack-infra | 16:41 | |
| *** eroux has quit IRC | 16:41 | |
| *** esikache1 has joined #openstack-infra | 16:42 | |
| *** isaacb has quit IRC | 16:42 | |
| jeblair | pabelanger: 'vos status afs02.dfw.openstack.org' shows http://paste.openstack.org/show/598681/ (those times are my local time which is currently 8:43) | 16:43 |
| jeblair | pabelanger: so i take it that the vos release that mirror-update is running is continuing | 16:43 |
| jeblair | pabelanger: (a vos release includes a 'restore' transaction iirc) | 16:43 |
| *** vhosakot has joined #openstack-infra | 16:44 | |
| pabelanger | fungi: serving content is still okay, we are just failing to 'vos release' new content. So the mirrors are stale ATM | 16:44 |
| jeblair | fungi: i also don't have the full background; hopefully pabelanger can answer, but from what i see, i would expect us to be serving old data from afs01.dfw | 16:44 |
| *** kjackal_ has quit IRC | 16:44 | |
| *** john-davidge has quit IRC | 16:45 | |
| pabelanger | yes, that is how I understand it too | 16:45 |
| *** john-davidge has joined #openstack-infra | 16:45 | |
| *** kjackal_ has joined #openstack-infra | 16:46 | |
| fungi | okay, cool | 16:46 |
| pabelanger | Released volume mirror.epel successfully | 16:47 |
| fungi | i guess i'm still a bit vague on the degree of redundancy between the fileservers | 16:47 |
| *** esikache1 has quit IRC | 16:47 | |
| pabelanger | jeblair: okay, so does that mean AFS is currently trying to restore volume: 536870932 to the last good know state? | 16:49 |
| openstackgerrit | Andy McCrae proposed openstack-infra/project-config master: Set os_zaqar role for OpenStack-Ansible to vote on xenial https://review.openstack.org/433182 | 16:49 |
| jeblair | fungi: read-only volumes are completely redundant. but when updating a ro volume (releasing it), clients use the old volume for consistency until all the replicas are updated. | 16:49 |
| jeblair | pabelanger: no, 'restore' is an implementation detail for vos release. it's implemented as an incremental backup and restore operation. | 16:50 |
| *** john-davidge has quit IRC | 16:50 | |
| jeblair | pabelanger: so it's trying to bring afs02.dfw up to date with what's on afs01.dfw | 16:50 |
| *** dims has quit IRC | 16:51 | |
| pabelanger | got it | 16:51 |
| fungi | jeblair: aha, thanks. so unless the problem replica is taken out of service the ro volume on the primary or other replicas can't be updated | 16:53 |
| *** jcoufal_ has joined #openstack-infra | 16:53 | |
| jeblair | fungi: that's my understanding, yes | 16:53 |
| *** srwilkers has joined #openstack-infra | 16:53 | |
| jeblair | fungi: i don't know if there's a 'mark temporarily out of service' option, or if the only options are "fix" or "remove entirely" | 16:54 |
| *** e0ne has quit IRC | 16:54 | |
| *** spzala has joined #openstack-infra | 16:54 | |
| fungi | makes sense | 16:54 |
| *** annegentle has joined #openstack-infra | 16:54 | |
| *** dims has joined #openstack-infra | 16:55 | |
| *** sdake_ has joined #openstack-infra | 16:55 | |
| *** tesseract-RH has quit IRC | 16:55 | |
| *** jcoufal has quit IRC | 16:56 | |
| *** annegentle has quit IRC | 16:56 | |
| *** annegentle has joined #openstack-infra | 16:56 | |
| *** sdake__ has joined #openstack-infra | 16:57 | |
| *** dtantsur|mtg is now known as dtantsur | 16:58 | |
| pabelanger | cool, mirror.centos appears to have recovered | 16:58 |
| *** pcaruana has quit IRC | 16:58 | |
| *** pcaruana has joined #openstack-infra | 16:58 | |
| *** sdake has quit IRC | 16:59 | |
| *** esikache1 has joined #openstack-infra | 16:59 | |
| *** mriedem has quit IRC | 17:00 | |
| *** sdake_ has quit IRC | 17:00 | |
| *** mriedem has joined #openstack-infra | 17:01 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: Deploy novajoin in the TLS-everywhere job https://review.openstack.org/427481 | 17:02 |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack-infra/tripleo-ci master: DO NOT MERGE - testing TLS everywhere job (novajoin) https://review.openstack.org/427482 | 17:02 |
| openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate master: Upgrade devstack-tools to 0.2.0 https://review.openstack.org/433093 | 17:03 |
| *** esikache1 has quit IRC | 17:03 | |
| openstackgerrit | Merged openstack-dev/pbr master: Updated from global requirements https://review.openstack.org/431922 | 17:04 |
| *** esikache1 has joined #openstack-infra | 17:06 | |
| *** dims_ has joined #openstack-infra | 17:06 | |
| *** dims has quit IRC | 17:07 | |
| *** tphummel has joined #openstack-infra | 17:08 | |
| *** jcoufal has joined #openstack-infra | 17:08 | |
| *** jcoufal__ has joined #openstack-infra | 17:09 | |
| *** esikache1 has quit IRC | 17:10 | |
| *** monester has joined #openstack-infra | 17:11 | |
| *** jcoufal_ has quit IRC | 17:11 | |
| *** jgriffiths has joined #openstack-infra | 17:12 | |
| openstackgerrit | Zhenguo Niu proposed openstack-infra/project-config master: Move senlin-dashboard py35 job to non-voting https://review.openstack.org/433192 | 17:13 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add action plugins to restrict untrusted execution https://review.openstack.org/428798 | 17:13 |
| *** jcoufal has quit IRC | 17:13 | |
| *** abregman|afk has quit IRC | 17:13 | |
| *** esikache1 has joined #openstack-infra | 17:14 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Updated Member serialization https://review.openstack.org/433195 | 17:16 |
| fungi | hoping to tag a new bindep release once the dust settles. the only outstanding change which potentially looks ready to merge is mordred's https://review.openstack.org/428394 if anybody else wants to review that | 17:17 |
| mordred | fungi: spekaing of things - I pushed up a new jeepyb patch to try to deal with gh rate limits | 17:17 |
| pabelanger | #status notice AFS replication issue has been addressed. Mirrors are currently re-syncing and coming back online. | 17:17 |
| openstackstatus | pabelanger: sending notice | 17:17 |
| fungi | mordred: ooh, thanks. will have a look | 17:18 |
| *** esikache1 has quit IRC | 17:18 | |
| mordred | jeblair: re: afs - next time we get a notice like the one from friday that a volume is going to go offline - should we take that fileserver out of rotation before the maint event then reenable it after to avoid being read only for a period? | 17:18 |
| mordred | jeblair: or would that be worse / more disruptive? | 17:19 |
| -openstackstatus- NOTICE: AFS replication issue has been addressed. Mirrors are currently re-syncing and coming back online. | 17:19 | |
| *** monester has quit IRC | 17:19 | |
| *** tnovacik has joined #openstack-infra | 17:19 | |
| *** nadya has joined #openstack-infra | 17:19 | |
| fungi | i was wondering the same | 17:20 |
| *** bnemec has quit IRC | 17:20 | |
| openstackstatus | pabelanger: finished sending notice | 17:20 |
| openstackgerrit | Carlos Camacho proposed openstack-infra/tripleo-ci master: Removing multinode_major_upgrade.yaml https://review.openstack.org/433201 | 17:21 |
| *** esikache1 has joined #openstack-infra | 17:21 | |
| jeblair | fungi, mordred: i don't know | 17:21 |
| *** trown is now known as trown|lunch | 17:21 | |
| *** panda is now known as panda|bbl | 17:23 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Split up min-ready requests to 1 node per request https://review.openstack.org/433127 | 17:23 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Collect only logs we need https://review.openstack.org/425459 | 17:23 |
| *** dargains has joined #openstack-infra | 17:24 | |
| *** bnemec has joined #openstack-infra | 17:24 | |
| *** tosky has quit IRC | 17:25 | |
| *** esikache1 has quit IRC | 17:26 | |
| *** ihrachys_ has joined #openstack-infra | 17:26 | |
| *** ihrachys has quit IRC | 17:26 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/tripleo-ci master: Replace "ara generate" by "ara generate html" https://review.openstack.org/433206 | 17:28 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/devstack-gate master: Replace "ara generate" by "ara generate html" https://review.openstack.org/433207 | 17:28 |
| fungi | pabelanger: what was the primary observed impact? centos package indices were referring to nonexistent package files? skimming the scrollback i'm not seeing much else | 17:29 |
| fungi | if that was the extent of the outward-facing symptoms, seems like it generally dealt with the situation well | 17:30 |
| jeblair | (also, why would that be? shouldn't the index and packages be consistent?) | 17:30 |
| pabelanger | fungi: Ya, initial reports we osic-cloud1 mirror was down | 17:30 |
| pabelanger | in fact it was not | 17:31 |
| pabelanger | so, I then looked at mirror-update.o.o for the epel.mirror log, and noticed vos release errors | 17:31 |
| pabelanger | Creating new volume 536870968 on replication site afs02.dfw.openstack.org: Failed to create the ro volume: : Input/output error | 17:32 |
| pabelanger | was the error | 17:32 |
| pabelanger | then, I remembered about the outage to afs02.dfw.o.o | 17:32 |
| *** rbrndt_ has quit IRC | 17:32 | |
| *** rbrndt has joined #openstack-infra | 17:32 | |
| *** vhosakot has quit IRC | 17:33 | |
| pabelanger | jeblair: usually yes for centos / epel, but we still need to write our index validation before we vos release | 17:33 |
| *** jpich has quit IRC | 17:33 | |
| jeblair | pabelanger: ah, so we just happened to have had a 'bad' update right before the outage i guess? | 17:33 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: upgrades: switch 'newton' to $UPGRADE_RELEASE https://review.openstack.org/433209 | 17:33 |
| pabelanger | jeblair: yes, that is what I am thinking | 17:33 |
| fungi | okay, so each hour when we rsync rpm mirrors we risk having them temporarily out of sync inex-wise? | 17:33 |
| fungi | er, index-wise | 17:33 |
| pabelanger | fungi: every 2 hours, but yes. Currently, it hasn't been much of an issue. I can only remember 1 other time, since bringing the mirror online, where I had to manually run our sync | 17:35 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Updated Member serialization https://review.openstack.org/433195 | 17:35 |
| *** mhickey has quit IRC | 17:35 | |
| pabelanger | on my list to write the validation, but have been pushing it off, because things work so well | 17:35 |
| mordred | ++ | 17:35 |
| *** john-davidge has joined #openstack-infra | 17:37 | |
| pabelanger | BTW: wouldn't mind a few more reviews on https://review.openstack.org/#/q/topic:infra-ptg-pike Would like to stand up nl01.o.o this week, before travelling to PTG | 17:37 |
| *** esikache1 has joined #openstack-infra | 17:37 | |
| *** nadya has quit IRC | 17:37 | |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack-infra/project-config master: Skip unittests of cinder if not related to code https://review.openstack.org/432423 | 17:39 |
| openstackgerrit | Ken'ichi Ohmichi proposed openstack-infra/project-config master: Skip a dsvm job of cinder if not related to code https://review.openstack.org/433211 | 17:39 |
| *** Apoorva has joined #openstack-infra | 17:39 | |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Use helper function to instantiate ProviderManager https://review.openstack.org/433212 | 17:39 |
| *** vsaienko has quit IRC | 17:40 | |
| AJaeger | pabelanger, could you review https://review.openstack.org/431120 for vmware-nsx python-db change, please? | 17:41 |
| *** esikache1 has quit IRC | 17:41 | |
| *** inc0 has joined #openstack-infra | 17:43 | |
| pabelanger | done | 17:43 |
| AJaeger | thanks | 17:43 |
| AJaeger | mordred: the change you approved yesterday is not completely done - how do we look on the jeepyb cache site? https://github.com/openstack/charm-keystone-ldap and https://github.com/openstack/charm-interface-keystone-domain-backend are empty | 17:44 |
| *** baoli has quit IRC | 17:44 | |
| *** pcaruana has quit IRC | 17:44 | |
| *** baoli has joined #openstack-infra | 17:45 | |
| *** baoli has quit IRC | 17:45 | |
| AJaeger | mordred: and https://github.com/openstack/charm-interface-ceph-mds which merged today is also empty | 17:45 |
| AJaeger | infra-root, something seems wrong with manage-projects, please see ^ | 17:45 |
| *** yamahata has quit IRC | 17:45 | |
| *** kmartin has quit IRC | 17:46 | |
| *** esikache1 has joined #openstack-infra | 17:47 | |
| pabelanger | I can take a look quickly | 17:47 |
| fungi | AJaeger: if it hit github api timeouts, then 433143 should hopefully help (i haven't reviewed it yet) | 17:47 |
| fungi | pabelanger: note that mordred's manage-projects caching changes merged, and were possibly used by that run | 17:47 |
| mordred | AJaeger: it's github rate limit stuff | 17:47 |
| pabelanger | okay, that is what I was checking | 17:48 |
| mordred | and yes - they were used - so we got to a different place :) | 17:48 |
| *** adrian_otto has joined #openstack-infra | 17:48 | |
| mordred | if that change looks good to y'all, I can run it again with the patch applied and watch for new errors | 17:48 |
| pabelanger | +2, looks right | 17:49 |
| openstackgerrit | Michael Johnson proposed openstack-infra/project-config master: Enable api-ref jobs for Octavia https://review.openstack.org/432493 | 17:49 |
| *** dprince has quit IRC | 17:49 | |
| openstackgerrit | Merged openstack-infra/project-config master: vmware-nsx: Use normal python jobs https://review.openstack.org/431120 | 17:50 |
| fungi | reviewing now | 17:51 |
| *** armax has joined #openstack-infra | 17:51 | |
| *** derekh has quit IRC | 17:52 | |
| jeblair | pabelanger, fungi: it looks like afs02dfw has a pretty constant 400mbit incoming stream. it's possible that we're hitting a bw limit. we may want to consider not replicating all of our volumes at once after an incident like this (it looks like 4 volumes currently have open transactions) | 17:52 |
| fungi | mordred: yeah, looks fine. go for it | 17:53 |
| fungi | jeblair: good point, that sounds like the bw cap for that flavor | 17:53 |
| mordred | fungi: cool. | 17:53 |
| *** markus_z has quit IRC | 17:53 | |
| jeblair | (i probably wouldn't be as worried except for the fact that the pypi release is running in a session that could timeout and need manual cleanup) | 17:54 |
| *** strigazi is now known as strigazi_AFK | 17:54 | |
| *** monester has joined #openstack-infra | 17:54 | |
| *** salv-orlando has joined #openstack-infra | 17:55 | |
| pabelanger | jeblair: fungi: maybe we should discuss how to prepare for an outage in the future. I haven't been able to find a way to disable replication to afs02.dfw.o.o, aside from removing the site from the volume | 17:56 |
| *** esikache1 has quit IRC | 17:56 | |
| *** ralonsoh has quit IRC | 17:56 | |
| openstackgerrit | Merged openstack-infra/jeepyb master: Make a few less github API calls https://review.openstack.org/433143 | 17:57 |
| *** david-lyle has joined #openstack-infra | 17:58 | |
| *** ZZelle_ has joined #openstack-infra | 17:58 | |
| *** armax has quit IRC | 17:59 | |
| oomichi | ianw: hi, thanks for your comment on https://review.openstack.org/#/c/432423/ | 17:59 |
| oomichi | ianw: I proposed another one for skiping heavy job on https://review.openstack.org/#/c/433211/ , but it is fine also to merge them into a single if we want | 18:00 |
| *** armax has joined #openstack-infra | 18:00 | |
| jeblair | pabelanger: i just meant to stop our vos releases (pause the cronjobs, etc) | 18:01 |
| *** lucasagomes is now known as lucas-afk | 18:01 | |
| *** kgiusti has left #openstack-infra | 18:01 | |
| *** spzala has quit IRC | 18:01 | |
| *** armax has quit IRC | 18:02 | |
| *** dtantsur is now known as dtantsur|afk | 18:02 | |
| pabelanger | jeblair: Ya, I wasn't sure if there was another way to do it using openafs. | 18:02 |
| pabelanger | but I would agree | 18:02 |
| *** nadya has joined #openstack-infra | 18:03 | |
| *** dargains has quit IRC | 18:03 | |
| *** fandi has joined #openstack-infra | 18:03 | |
| *** hashar has quit IRC | 18:04 | |
| *** nadya has quit IRC | 18:04 | |
| *** nadya has joined #openstack-infra | 18:04 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: Supply label name to Nodes https://review.openstack.org/431719 | 18:04 |
| *** Swami has joined #openstack-infra | 18:06 | |
| *** hrybacki has quit IRC | 18:06 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Fixed member feedback https://review.openstack.org/433221 | 18:07 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Use helper function to instantiate ProviderManager https://review.openstack.org/433212 | 18:08 |
| *** david-lyle_ has joined #openstack-infra | 18:08 | |
| *** jaosorior has quit IRC | 18:08 | |
| *** kgiusti has joined #openstack-infra | 18:09 | |
| *** david-lyle has quit IRC | 18:09 | |
| *** david-lyle_ is now known as david-lyle | 18:12 | |
| *** fandi has quit IRC | 18:14 | |
| *** xarses_ has joined #openstack-infra | 18:17 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Supply label name to Nodes https://review.openstack.org/431719 | 18:17 |
| *** dhinesh has joined #openstack-infra | 18:21 | |
| jeblair | pabelanger, fungi: down to 3 open transactions | 18:22 |
| jeblair | looks like mirror.fedora just finished | 18:23 |
| jeblair | still at 400mb | 18:23 |
| pabelanger | can confirm | 18:23 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Update nodepool 'list' command https://review.openstack.org/431647 | 18:23 |
| *** baoli has joined #openstack-infra | 18:24 | |
| jeblair | ftr: debian, ubuntu, and pypi are still running | 18:24 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Update nodepool hold to use zookeeper https://review.openstack.org/431756 | 18:24 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Re-enable test_dib_image_pause / test_dib_image_upload_pause https://review.openstack.org/432447 | 18:24 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Re-enable test_dib_image_delete test https://review.openstack.org/432606 | 18:24 |
| *** baoli has quit IRC | 18:25 | |
| jeblair | pabelanger, fungi: looking at 'vos listvldb' it seems a bunch of the wheel mirrors have partial releases; when the rest of the mirrors are finished, maybe we should manually release them (if a zuul job hasn't gotten to it first) | 18:25 |
| *** yamahata has joined #openstack-infra | 18:25 | |
| *** baoli has joined #openstack-infra | 18:25 | |
| fungi | interesting. presumably related to this incident and not just a general issue with the jobs | 18:26 |
| *** rossella_s has quit IRC | 18:26 | |
| *** gouthamr has quit IRC | 18:26 | |
| *** gouthamr has joined #openstack-infra | 18:27 | |
| *** Hal has quit IRC | 18:28 | |
| openstackgerrit | Tomasz Trębski proposed openstack-infra/project-config master: Move log-api tempest gate to voting stage https://review.openstack.org/433018 | 18:29 |
| *** e0ne has joined #openstack-infra | 18:30 | |
| AJaeger | mordred: so, does an infra-root need to manually adjust the newly created github repos now? | 18:30 |
| mordred | AJaeger: sorry - still iterating on it - gimme just a little longer | 18:30 |
| fungi | gonna go grab some lunch, bbiab | 18:31 |
| AJaeger | mordred: no worries ;) btw. https://github.com/openstack/python-tricircleclient is completely missing - so, that's 4 repos... | 18:31 |
| *** esikache1 has joined #openstack-infra | 18:32 | |
| mordred | AJaeger: awesome | 18:33 |
| *** maishsk has joined #openstack-infra | 18:33 | |
| openstackgerrit | Tomasz Trębski proposed openstack-infra/project-config master: Move log-api tempest gate to voting stage https://review.openstack.org/433018 | 18:34 |
| *** esikache1 has quit IRC | 18:36 | |
| *** john-dav_ has joined #openstack-infra | 18:40 | |
| *** john-davidge has quit IRC | 18:40 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Re-enable test_node_net_name test https://review.openstack.org/433227 | 18:41 |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Wait for main loop when stopping https://review.openstack.org/433230 | 18:42 |
| *** sdake has joined #openstack-infra | 18:42 | |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool feature/zuulv3: Wait for main loop when stopping https://review.openstack.org/433230 | 18:43 |
| *** sdake__ has quit IRC | 18:43 | |
| *** eroux has joined #openstack-infra | 18:43 | |
| *** Goneri has quit IRC | 18:43 | |
| *** e0ne has quit IRC | 18:44 | |
| mordred | infra-root, AJaeger: Ok. I found the final jeepyb issue | 18:45 |
| *** hrybacki has joined #openstack-infra | 18:45 | |
| *** sdake has quit IRC | 18:45 | |
| *** sdake has joined #openstack-infra | 18:47 | |
| *** trown|lunch is now known as trown | 18:47 | |
| *** sdake has quit IRC | 18:47 | |
| *** dane-fichter has quit IRC | 18:47 | |
| *** baoli has quit IRC | 18:48 | |
| AJaeger | final? Cool ;) | 18:48 |
| *** baoli has joined #openstack-infra | 18:48 | |
| *** ekhugen has quit IRC | 18:49 | |
| *** zz_ja has quit IRC | 18:49 | |
| *** vsaienko has joined #openstack-infra | 18:49 | |
| *** patriciadomin has quit IRC | 18:50 | |
| openstackgerrit | Tomasz Trębski proposed openstack-infra/project-config master: Move log-api tempest gate to voting stage https://review.openstack.org/433018 | 18:50 |
| *** esikache1 has joined #openstack-infra | 18:53 | |
| *** jcoufal has joined #openstack-infra | 18:53 | |
| *** ekhugen has joined #openstack-infra | 18:53 | |
| *** zz_ja has joined #openstack-infra | 18:54 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: murano: Use normal python jobs https://review.openstack.org/429050 | 18:54 |
| *** jcoufal__ has quit IRC | 18:56 | |
| *** esikache1 has quit IRC | 18:57 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Re-enable test_node_net_name test https://review.openstack.org/433227 | 18:58 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Re-enable test_node_vhd_image test https://review.openstack.org/433233 | 18:58 |
| AJaeger | fungi, I documented tools/test-setup.sh - please put https://review.openstack.org/432786 on your infra-manual review queue | 19:00 |
| *** salv-orl_ has joined #openstack-infra | 19:00 | |
| *** shardy has quit IRC | 19:00 | |
| *** rbuzatu has quit IRC | 19:00 | |
| *** salv-orlando has quit IRC | 19:03 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Installs openssh packages that try to workaround sshd start problem https://review.openstack.org/430688 | 19:03 |
| *** esikache1 has joined #openstack-infra | 19:04 | |
| *** jamesdenton has quit IRC | 19:05 | |
| *** sean-k-mooney has quit IRC | 19:09 | |
| *** esikache1 has quit IRC | 19:11 | |
| *** john-dav_ has quit IRC | 19:13 | |
| *** ociuhandu has quit IRC | 19:13 | |
| *** esikache1 has joined #openstack-infra | 19:13 | |
| *** john-davidge has joined #openstack-infra | 19:13 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Re-enable test_dib_upload_fail test https://review.openstack.org/433235 | 19:14 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/shade master: Remove mock of keystoneclient for test_caching https://review.openstack.org/433236 | 19:15 |
| *** rbrndt has quit IRC | 19:15 | |
| *** pvaneck has joined #openstack-infra | 19:15 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/shade master: Remove mock of keystoneclient for test_caching for projects https://review.openstack.org/433236 | 19:15 |
| *** vsaienko has quit IRC | 19:17 | |
| *** john-davidge has quit IRC | 19:18 | |
| *** esikache1 has quit IRC | 19:18 | |
| *** dharinic is now known as dharinic|lunch | 19:19 | |
| openstackgerrit | Lucas Alvares Gomes proposed openstack-infra/project-config master: Add a new project Sushy https://review.openstack.org/432285 | 19:20 |
| *** salv-orl_ has quit IRC | 19:20 | |
| *** salv-orlando has joined #openstack-infra | 19:21 | |
| *** MasterOfBugs has joined #openstack-infra | 19:21 | |
| *** esikache1 has joined #openstack-infra | 19:23 | |
| openstackgerrit | John Trowbridge proposed openstack-infra/tripleo-ci master: update promotion scripts for ocata https://review.openstack.org/427385 | 19:24 |
| *** srwilkers has quit IRC | 19:24 | |
| *** ssbarnea has quit IRC | 19:25 | |
| *** cody-somerville has quit IRC | 19:25 | |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Wait for main loop when stopping https://review.openstack.org/433230 | 19:26 |
| *** dprince has joined #openstack-infra | 19:26 | |
| *** electrofelix has quit IRC | 19:27 | |
| *** ihrachys has joined #openstack-infra | 19:29 | |
| mordred | AJaeger, infra-root: running with a locally modified jeepyb on review.o.o | 19:30 |
| *** cody-somerville has joined #openstack-infra | 19:31 | |
| *** srwilkers has joined #openstack-infra | 19:31 | |
| AJaeger | mordred: thanks. did you push the change up already? | 19:31 |
| *** ihrachys_ has quit IRC | 19:31 | |
| *** hashar has joined #openstack-infra | 19:32 | |
| *** csomerville has joined #openstack-infra | 19:32 | |
| *** ihrachys_ has joined #openstack-infra | 19:33 | |
| *** esikache1 has quit IRC | 19:33 | |
| *** esikache1 has joined #openstack-infra | 19:34 | |
| *** ihrachys has quit IRC | 19:34 | |
| *** gk-1wm-su has joined #openstack-infra | 19:34 | |
| *** gk-1wm-su has left #openstack-infra | 19:34 | |
| mordred | AJaeger: not yet | 19:35 |
| AJaeger | ok | 19:35 |
| *** cody-somerville has quit IRC | 19:35 | |
| AJaeger | team, could you review https://review.openstack.org/431154 - updates tuning-box for python-db change, please? The dependencies have merged | 19:36 |
| jeblair | down to 2 open transactions | 19:36 |
| *** jcoufal_ has joined #openstack-infra | 19:39 | |
| *** jcoufal has quit IRC | 19:39 | |
| openstackgerrit | Sachin proposed openstack-infra/project-config master: Introducing CI changed for Omni https://review.openstack.org/433241 | 19:41 |
| *** matrohon has joined #openstack-infra | 19:41 | |
| *** rfolco has quit IRC | 19:44 | |
| *** rbuzatu has joined #openstack-infra | 19:44 | |
| *** panda|bbl is now known as panda | 19:45 | |
| pabelanger | stepping a way to run an errand, shouldn't be to long | 19:45 |
| openstackgerrit | Radoslav Gerganov proposed openstack-infra/project-config master: Add new project: vmware-vspc https://review.openstack.org/433144 | 19:46 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Set Node image_id and launcher attributes https://review.openstack.org/433242 | 19:48 |
| *** baoli has quit IRC | 19:50 | |
| *** ociuhandu has joined #openstack-infra | 19:51 | |
| *** baoli has joined #openstack-infra | 19:51 | |
| *** jcoufal_ has quit IRC | 19:54 | |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Re-enable test_node test https://review.openstack.org/432607 | 19:54 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/shade master: Remove keystoneclient mocks in test_caching for users https://review.openstack.org/433244 | 19:55 |
| *** ociuhandu has quit IRC | 19:56 | |
| *** salv-orlando has quit IRC | 19:57 | |
| *** maishsk has quit IRC | 19:57 | |
| *** dharinic|lunch is now known as dharinic | 19:57 | |
| *** dane-fichter has joined #openstack-infra | 19:59 | |
| dane-fichter | AJaeger: any idea how to add an image file to a gate that runs tempest tests? | 20:00 |
| AJaeger | dane-fichter: what exactly do you want to do? | 20:00 |
| dane-fichter | AJaeger: we need a disc image for upload | 20:01 |
| *** esikachev has joined #openstack-infra | 20:01 | |
| dane-fichter | so that the config options CONF.scenario.img_dir/img_file will point to a valid disc image in the gate | 20:01 |
| clarkb | as a heads up I have contracted some version of child crud and have an appointment to do taxes so mostly not here today | 20:01 |
| dane-fichter | AJaeger: see https://review.openstack.org/#/c/431241/ | 20:02 |
| AJaeger | dane-fichter: can't you create a simple one on the fly? | 20:02 |
| dane-fichter | AJaeger: how do you mean? | 20:03 |
| dane-fichter | AJaeger: locally im just using a CirrOs image and configuring tempest to use it | 20:03 |
| *** esikache1 has quit IRC | 20:04 | |
| AJaeger | dane-fichter: do you really need an image or will a random file be enough for this test? | 20:04 |
| dane-fichter | AJaeger: it needs to be an actual, bootable image | 20:04 |
| dane-fichter | AJaeger: preferably a CirrOs for latency reasons | 20:05 |
| *** armax has joined #openstack-infra | 20:07 | |
| AJaeger | dane-fichter: openstack-dev/devstack downloads cirros AFAIK | 20:07 |
| *** jcoufal has joined #openstack-infra | 20:08 | |
| AJaeger | dane-fichter: I'm not aware that we cache it anywhere but others here might know more | 20:08 |
| dane-fichter | AJager: doesn't appear to be, judging by the results of my gate | 20:08 |
| dane-fichter | AJaeger: or at least, if it is downloading CirrOs, it's not putting it anywhere Tempest can use it. | 20:09 |
| *** spzala has joined #openstack-infra | 20:09 | |
| AJaeger | dane-fichter: better ask others for help on this | 20:10 |
| dane-fichter | AJaeger: Anybody in particular you think would be useful? | 20:10 |
| *** srwilkers has quit IRC | 20:11 | |
| *** dimtruck is now known as zz_dimtruck | 20:12 | |
| *** ociuhandu has joined #openstack-infra | 20:12 | |
| dmsimard | At what frequency does the internal pypi mirror update ? And also, what's the URL to query it ? Looks like it's lagging behind. | 20:12 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh https://review.openstack.org/431029 | 20:13 |
| AJaeger | dane-fichter: just ask in the round here and hope something can help ;) | 20:13 |
| *** rbrndt has joined #openstack-infra | 20:16 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Installs openssh packages that try to workaround sshd start problem https://review.openstack.org/430688 | 20:16 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Split up min-ready requests to 1 node per request https://review.openstack.org/433127 | 20:18 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Use helper function to instantiate ProviderManager https://review.openstack.org/433212 | 20:18 |
| *** vsaienko has joined #openstack-infra | 20:20 | |
| *** ijw has joined #openstack-infra | 20:20 | |
| *** ociuhandu has quit IRC | 20:21 | |
| *** gouthamr has quit IRC | 20:24 | |
| *** jcoufal has quit IRC | 20:24 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/shade master: WIP: Remove keystoneclient mocks in test_caching for users https://review.openstack.org/433244 | 20:25 |
| *** ijw has quit IRC | 20:25 | |
| *** ijw has joined #openstack-infra | 20:25 | |
| *** gouthamr has joined #openstack-infra | 20:25 | |
| *** adrian_otto has quit IRC | 20:26 | |
| *** Goneri has joined #openstack-infra | 20:26 | |
| *** kong has joined #openstack-infra | 20:27 | |
| *** jcoufal has joined #openstack-infra | 20:29 | |
| *** rfolco has joined #openstack-infra | 20:29 | |
| *** rfolco has quit IRC | 20:29 | |
| *** ociuhandu has joined #openstack-infra | 20:32 | |
| *** zz_dimtruck is now known as dimtruck | 20:32 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add generator API method for node iteration https://review.openstack.org/433252 | 20:32 |
| *** srwilkers has joined #openstack-infra | 20:33 | |
| *** esikachev has quit IRC | 20:33 | |
| dane-fichter | does anyone know how to add a disc image file to a gate??? | 20:33 |
| *** tonytan4ever has quit IRC | 20:36 | |
| *** tonytan4ever has joined #openstack-infra | 20:36 | |
| *** ociuhandu has quit IRC | 20:36 | |
| pabelanger | dmsimard: known issue, we are trying to recover from an outage in our AFS replication | 20:37 |
| pabelanger | it is a waiting game now | 20:37 |
| *** rfolco has joined #openstack-infra | 20:38 | |
| *** rfolco has quit IRC | 20:38 | |
| AJaeger | pabelanger: , could you add https://review.openstack.org/431154 to your review queue, please? - updates tuning-box for python-db change. The dependencies have merged. | 20:38 |
| fungi | dane-fichter: http://paste.openstack.org/show/598701/ (taken from a ready job node( | 20:38 |
| fungi | dane-fichter: found by reading through https://git.openstack.org/cgit/openstack-dev/devstack/tree/stackrc#n595 followed by http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/elements/cache-devstack/extra-data.d/55-cache-devstack-repos | 20:39 |
| *** jamesdenton has joined #openstack-infra | 20:40 | |
| fungi | dane-fichter: will one of those cirros images (0.3.0, 0.3.2, 0.3.4) suffice? | 20:40 |
| *** sdake has joined #openstack-infra | 20:41 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh https://review.openstack.org/431029 | 20:41 |
| dane-fichter | fungi: yeah those would work, it's a matter of getting the gate to download those and the tempest config to point to them | 20:42 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/devstack-gate master: WIP/DNM: Testing grenade with Ironic multi-node job https://review.openstack.org/431777 | 20:42 |
| fungi | dane-fichter: those are pre-cached at that location before jobs start, because they're declared by devstack as something it might need | 20:42 |
| *** ianychoi has quit IRC | 20:43 | |
| fungi | dane-fichter: so shouldn't need to download them--they're already there | 20:43 |
| dane-fichter | in ~/jenkins/cache/files ? | 20:43 |
| fungi | ~jenkins/cache/files but yes | 20:44 |
| fungi | looks like they're only readable by the jenkins user, so you may need to so something to copy them somewhere the stack or tempest user expects (if devstack doesn't do this for you already) | 20:44 |
| fungi | s/so something/do something/ | 20:44 |
| dane-fichter | devstack appears to do some copying/downloading of image files | 20:45 |
| dane-fichter | doesn't seem all that stable though since it isn't downloading and storing the image file that tempest expects | 20:45 |
| fungi | normal devstack+tempest jobs use one of those to populate glance so nova has something to boot... might double-check what it's doing and follow a similar pattern? | 20:45 |
| dane-fichter | fungi: how do I check what those are doing? | 20:46 |
| fungi | this also might be a discussion for #openstack-qa if devstack or tempest aren't behaving as expected. those aren't infra deliverables | 20:46 |
| dane-fichter | fungi: hmm ok good point | 20:46 |
| dane-fichter | the gate template thing is confusing me | 20:46 |
| fungi | basically devstack maintains that IMAGE_URLS list which we parse through an entrypoint it provides and then download and make them all available on disk for jobs to consume | 20:47 |
| dane-fichter | hard to see how you'd modify the behavior of a gate that is spawned from one of these templates | 20:47 |
| panda | is there a way to check why a patch failed to merge on gerrit ? I'm unable to reproduce the merge failure locally | 20:47 |
| dane-fichter | fungi: yup i see that | 20:47 |
| sdague | fungi: ok, really confused, why don't the pypi mirrors have devstack-tools 0.2.0 yet? | 20:47 |
| fungi | dane-fichter: and devstack-gate tells devstack where to find its file cache, afaik everything else is up to devstack to take care of | 20:47 |
| sdague | http://logs.openstack.org/93/433093/2/check/gate-grenade-dsvm-neutron-ubuntu-xenial/76b6c8c/logs/grenade.sh.txt.gz#_2017-02-13_19_36_31_606 | 20:48 |
| dane-fichter | fungi: agreed, maybe this is an issue w tempest defaults | 20:48 |
| fungi | sdague: rackspace was doing cinder maintenance over the weekend and tanked one of our afs servers so we've been serving up the read-only volume all day while it's in the process of being rectified | 20:48 |
| dane-fichter | because tempest is looking in a non-existent folder for images | 20:48 |
| sdague | fungi: ah, ok, is there an eta? | 20:49 |
| fungi | sdague: not sure, but sounds like it's very close to finished? pabelanger and jeblair have been watching its progress i think | 20:49 |
| pabelanger | yes, still watching | 20:49 |
| pabelanger | mirror.pypi is last one running ATM | 20:50 |
| *** ianychoi has joined #openstack-infra | 20:51 | |
| *** nadya has quit IRC | 20:51 | |
| *** gildub has joined #openstack-infra | 20:52 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add generator API method for node iteration https://review.openstack.org/433252 | 20:54 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Set Node image_id and launcher attributes https://review.openstack.org/433242 | 20:54 |
| *** jcoufal has quit IRC | 20:54 | |
| *** mikal_ is now known as mikal | 20:54 | |
| *** Swami has quit IRC | 20:55 | |
| *** Swami has joined #openstack-infra | 20:56 | |
| *** srwilkers has quit IRC | 20:56 | |
| *** jcoufal has joined #openstack-infra | 20:56 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Re-enable working test_builder.py tests https://review.openstack.org/433262 | 20:57 |
| *** monester has quit IRC | 20:58 | |
| openstackgerrit | Merged openstack-infra/project-config master: Skip unittests of cinder if not related to code https://review.openstack.org/432423 | 20:58 |
| openstackgerrit | Merged openstack-infra/project-config master: Skip a dsvm job of cinder if not related to code https://review.openstack.org/433211 | 20:58 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/shade master: WIP: Remove keystoneclient mocks in test_caching for users https://review.openstack.org/433244 | 20:58 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/shade master: Remove keystoneclient mocks in test_caching for users https://review.openstack.org/433244 | 20:59 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Re-enable test_diskimage_build_only test https://review.openstack.org/433265 | 21:00 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Re-enable test_image_upload_fail test https://review.openstack.org/433270 | 21:01 |
| *** jamielennox|away is now known as jamielennox | 21:02 | |
| *** rkukura has quit IRC | 21:04 | |
| panda | pabelanger: any hints ? ^ | 21:06 |
| pabelanger | panda: which patch? | 21:08 |
| panda | pabelanger: https://review.openstack.org/431029 | 21:08 |
| *** ihrachys_ is now known as ihrachys | 21:10 | |
| pabelanger | panda: likey an issue with a cross repo dependency | 21:11 |
| pabelanger | that patch looks okay, but the depends-on might have problems | 21:11 |
| pabelanger | https://review.openstack.org/#/c/425451/ for example | 21:11 |
| *** jamesden_ has joined #openstack-infra | 21:11 | |
| *** yamamoto has joined #openstack-infra | 21:12 | |
| *** jamesdenton has quit IRC | 21:12 | |
| fungi | which is why we added "or one of its cross-repo dependencies" to the failure message for those | 21:13 |
| *** sambetts is now known as sambetts|afk | 21:14 | |
| panda | fungi: pabelanger aaaahhh mmmhh looking thanks | 21:18 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/project-config master: Kill gate-tempest-dsvm-platform-ubuntu-trusty https://review.openstack.org/433274 | 21:19 |
| *** monester has joined #openstack-infra | 21:20 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Set Node image_id and launcher attributes https://review.openstack.org/433242 | 21:21 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add generator API method for node iteration https://review.openstack.org/433252 | 21:21 |
| *** patriciadomin has joined #openstack-infra | 21:22 | |
| *** jcoufal has quit IRC | 21:24 | |
| *** jcoufal has joined #openstack-infra | 21:27 | |
| *** yamamoto has quit IRC | 21:27 | |
| *** pramodrj07 has joined #openstack-infra | 21:28 | |
| *** pramodrj07 has quit IRC | 21:30 | |
| *** pramodrj07 has joined #openstack-infra | 21:30 | |
| *** sree_ has joined #openstack-infra | 21:31 | |
| *** sree_ is now known as Guest34695 | 21:31 | |
| *** MasterOfBugs has quit IRC | 21:32 | |
| *** pramodrj07 has quit IRC | 21:32 | |
| *** MasterOfBugs has joined #openstack-infra | 21:32 | |
| *** jkilpatr has quit IRC | 21:32 | |
| *** jgriffiths has quit IRC | 21:32 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Save images according to their branch https://review.openstack.org/429782 | 21:33 |
| *** fguillot has quit IRC | 21:33 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Re-enable test_diskimage_build_only test https://review.openstack.org/433265 | 21:33 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Re-enable test_dib_upload_fail test https://review.openstack.org/433235 | 21:33 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Re-enable test_node_vhd_image test https://review.openstack.org/433233 | 21:33 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Re-enable test_image_upload_fail test https://review.openstack.org/433270 | 21:33 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Re-enable test_node_net_name test https://review.openstack.org/433227 | 21:33 |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool feature/zuulv3: Re-enable working test_builder.py tests https://review.openstack.org/433262 | 21:33 |
| *** dprince has quit IRC | 21:34 | |
| *** rbuzatu has quit IRC | 21:35 | |
| *** Guest34695 has quit IRC | 21:35 | |
| *** Jeffrey4l has quit IRC | 21:36 | |
| *** Jeffrey4l_ has joined #openstack-infra | 21:36 | |
| *** dave-mccowan has quit IRC | 21:36 | |
| *** srwilkers has joined #openstack-infra | 21:37 | |
| *** monester has quit IRC | 21:42 | |
| *** jaypipes has quit IRC | 21:44 | |
| *** salv-orlando has joined #openstack-infra | 21:46 | |
| jhesketh | Morning | 21:48 |
| *** sshnaidm is now known as sshnaidm|off | 21:51 | |
| *** dane-fichter has quit IRC | 21:53 | |
| *** jcoufal_ has joined #openstack-infra | 21:53 | |
| *** jcoufal has quit IRC | 21:55 | |
| *** jcoufal_ has quit IRC | 21:56 | |
| *** jheroux has quit IRC | 21:56 | |
| *** vsaienko has quit IRC | 21:57 | |
| *** gouthamr has quit IRC | 21:59 | |
| jeblair | it's zuul meeting time in #openstack-meeting-alt | 22:01 |
| fungi | so it is! | 22:01 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci master: Stop mirroring centos images https://review.openstack.org/430372 | 22:02 |
| sbezverk | any ETA for the gate fix please? | 22:03 |
| *** thorst_ has quit IRC | 22:03 | |
| *** pvaneck_ has joined #openstack-infra | 22:05 | |
| *** hashar has quit IRC | 22:06 | |
| fungi | sbezverk: what "gate fix" are you looking for? | 22:07 |
| openstackgerrit | Sachin proposed openstack-infra/project-config master: Introducing CI changes for Omni https://review.openstack.org/433241 | 22:07 |
| fungi | as far as i know the versions of packages available on out pypi mirror are about 18 hours behind, but are there other problems we're not aware of? | 22:07 |
| *** jamesden_ has quit IRC | 22:08 | |
| fungi | s/out/our/ | 22:08 |
| *** pvaneck has quit IRC | 22:08 | |
| *** vsaienko has joined #openstack-infra | 22:11 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: [WIP] Quickstart transition: replace toci_gate_test.sh https://review.openstack.org/431029 | 22:11 |
| *** yamamoto has joined #openstack-infra | 22:12 | |
| *** jkilpatr has joined #openstack-infra | 22:13 | |
| *** eharney has quit IRC | 22:15 | |
| *** trown is now known as trown|outtypewww | 22:15 | |
| *** ihrachys has quit IRC | 22:16 | |
| *** monester has joined #openstack-infra | 22:17 | |
| sbezverk | fungi: on kolla-kubernetes project all gates are constantly failing | 22:18 |
| fungi | sbezverk: early on that was reported due to centos package repository indices being out of sync on our mirrors, but i thought that was resolved hours ago | 22:20 |
| fungi | pabelanger: ^ ? | 22:20 |
| *** rajinir_ has quit IRC | 22:20 | |
| openstackgerrit | Sachin proposed openstack-infra/project-config master: Introducing CI changes for Omni https://review.openstack.org/433241 | 22:21 |
| pabelanger | fungi: yes, they should be. We'd need to see a recent failure to see what is going on | 22:21 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/shade master: Remove keystoneclient mocks in test_caching for users https://review.openstack.org/433244 | 22:21 |
| pabelanger | mirror.pypi is still ongoing | 22:21 |
| sbezverk | pabelanger: here is the log from PS which was passing perfectl las night, now it is constantly failing. http://logs.openstack.org/56/431156/29/check/gate-kolla-kubernetes-python27-ubuntu-xenial/4eea945/console.html#_2017-02-13_20_55_41_594585 | 22:24 |
| *** Goneri has quit IRC | 22:25 | |
| *** tnovacik has quit IRC | 22:25 | |
| pabelanger | sbezverk: this looks to be a failure of your template syntax, I am not sure how that would related to the AFS issues we had today | 22:27 |
| pabelanger | unless a new package was installed that broke something in your project | 22:27 |
| *** ZZelle_ has quit IRC | 22:28 | |
| *** burgerk_ has joined #openstack-infra | 22:28 | |
| *** matrohon has quit IRC | 22:30 | |
| openstackgerrit | Matthieu Huin proposed openstack-infra/jenkins-job-builder master: Add a publisher for MQTT https://review.openstack.org/427765 | 22:31 |
| *** burgerk has quit IRC | 22:31 | |
| *** thorst_ has joined #openstack-infra | 22:33 | |
| *** burgerk_ has quit IRC | 22:34 | |
| *** monester has quit IRC | 22:35 | |
| *** rbuzatu has joined #openstack-infra | 22:35 | |
| *** gouthamr has joined #openstack-infra | 22:37 | |
| *** thorst_ has quit IRC | 22:38 | |
| *** Hal has joined #openstack-infra | 22:39 | |
| clarkb | sbezverk: reading that error it is a jinja2 template error happening from your test templates test | 22:40 |
| clarkb | sbezverk: I don't think that is an infrastructure issue | 22:40 |
| sbezverk | clarkb: I would agree with you if the same PS without any modification has not been passing all green last night | 22:41 |
| clarkb | sbezverk: I don't know what else to tell you, your test is failing because the jinja2 template is bad according to the logs.That tells me an issue in the template or jinja2 | 22:42 |
| clarkb | but not something related to where the actual thing is running | 22:42 |
| sbezverk | clarkb: funny thing is we see the same error on all PS | 22:43 |
| clarkb | sbezverk: so maybe jinja2 updated and something is no longer valid? or maybe the test broke somehow? | 22:43 |
| pabelanger | that would be my guess | 22:43 |
| sbezverk | clarkb: ok we will dounle check the master | 22:46 |
| *** tonytan4ever has quit IRC | 22:46 | |
| sbezverk | thanks for suggestion | 22:46 |
| sc` | dumb question time: what's the best way to test enhancements to my project's ci without spamming people with changeset noise when it fails? | 22:47 |
| *** panda is now known as panda|zZ | 22:47 | |
| *** wolverineav has quit IRC | 22:47 | |
| openstackgerrit | Russell Bryant proposed openstack-infra/project-config master: networking-ovn: Reorganize CI jobs. https://review.openstack.org/394684 | 22:48 |
| fungi | sc`: if it's zuul you can change the report mechanism to e-mail you instead of commenting on the review | 22:48 |
| *** salv-orlando has quit IRC | 22:49 | |
| clarkb | what I like to do is run enough of it locally that I am confident in it then push up | 22:51 |
| clarkb | and then clean up any remaining issues | 22:51 |
| *** gordc has quit IRC | 22:51 | |
| sc` | there within lies the rub: i'm going to be testing multinode deployments - something we've never had in the gates before | 22:52 |
| sc` | being chef, not everything from puppet-openstack or tripleo-ci would translate, necessitating in some triangle-shaped wheels | 22:53 |
| *** matt-borland has quit IRC | 22:54 | |
| *** vsaienko has quit IRC | 22:55 | |
| *** gk--1wm- has joined #openstack-infra | 22:58 | |
| *** gk--1wm- has left #openstack-infra | 22:58 | |
| pabelanger | clarkb: fungi: https://review.openstack.org/#/c/431122/ we missed this from bup work last week, removes hieradata from backups | 22:58 |
| pabelanger | clarkb: also, when you have free time, do you mind reviewing nl01.o.o again: https://review.openstack.org/#/q/topic:infra-ptg-pike | 22:59 |
| clarkb | pabelanger: ya | 23:00 |
| pabelanger | danke | 23:00 |
| *** wolverineav has joined #openstack-infra | 23:00 | |
| clarkb | and for other review requests https://review.openstack.org/#/c/326585/ would be a good one to get reviewed so hopefully sdague can stop rebasing it every week | 23:01 |
| *** thorst_ has joined #openstack-infra | 23:01 | |
| *** mriedem is now known as mriedem_afk | 23:02 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Move coverage jobs to non-voting gate jobs https://review.openstack.org/432836 | 23:03 |
| *** aeng has quit IRC | 23:04 | |
| *** dhellmann has quit IRC | 23:05 | |
| *** wolverineav has quit IRC | 23:05 | |
| clarkb | ianw: re ^ any idea how long nova and neutron's coverage jobs take? (should be roughly captured by console log) | 23:05 |
| ianw | clarkb: can check ... i'd hope the same as the unit tests | 23:06 |
| *** dhellmann has joined #openstack-infra | 23:06 | |
| clarkb | ianw: part of the original reason for running the coverage jobs in post was neutron regular job took 40-60 minutes and coverage was much longer due to measurement overhead | 23:07 |
| *** annegentle has quit IRC | 23:07 | |
| clarkb | ianw: that said since then I think neutron (and nova) have test suites that run much quicker overall | 23:07 |
| *** dizquierdo has joined #openstack-infra | 23:08 | |
| fungi | i thought a big part of the coverage overhead at the time was being unable to parallelize tests too? | 23:08 |
| clarkb | fungi: no we have always been able to parallelize them with testr too | 23:09 |
| clarkb | (so as long as testr + coverage was used it was fine) | 23:09 |
| *** rbuzatu has quit IRC | 23:10 | |
| fungi | ahh, maybe i'm remembering a nosetests+coverage limitation | 23:10 |
| ianw | ok, the latest one i can find is http://logs.openstack.org/c3/c3671ee7da154e251c2915d4aced2b1a2bd8dfa9/post/nova-coverage-db-ubuntu-xenial/bc21639/console.html | 23:10 |
| *** marst has quit IRC | 23:11 | |
| ianw | 2017-02-09 12:39:10.723156 - 2017-02-09 12:49:43.557147 | 23:11 |
| clarkb | so not bad for nova | 23:11 |
| *** marst has joined #openstack-infra | 23:11 | |
| ianw | i do wonder if anyone finds them in the post phase, since they go under the merge commit | 23:13 |
| fungi | it's been suggested coverage trending would be a good improvement to the health dashboard | 23:14 |
| ianw | /dev/xvda1 40G 40G 0 100% / | 23:15 |
| ianw | umm, that seems bad | 23:15 |
| ianw | static.openstack.org | 23:15 |
| fungi | ouch | 23:15 |
| pabelanger | ya, think it is apache2 logs | 23:15 |
| *** jamielennox is now known as jamielennox|away | 23:15 | |
| fungi | that was going to be my first guess | 23:15 |
| ianw | root@static:/var/log/apache2# du -h | 23:15 |
| ianw | 4.5G. | 23:15 |
| fungi | huh. maybe something else then | 23:16 |
| ianw | 2.5G Feb 5 06:25 logs.openstack.org_access.log.1 | 23:16 |
| ianw | we should probably make that compress | 23:16 |
| fungi | oh, it's the mlocate cache!?! | 23:17 |
| fungi | i could have sworn we uninstalled that entirely | 23:17 |
| pabelanger | fungi: Oh, yes. I actually mentioned that the other day | 23:17 |
| *** dizquierdo has quit IRC | 23:18 | |
| fungi | 32g in /var/lib/mlocate | 23:18 |
| fungi | not only does it add lengthy i/o load, but apparently also fills your rootfs while you're not looking :/ | 23:19 |
| JayF | eek | 23:19 |
| bkero | but hey, if you ever need to locate a file.... | 23:19 |
| * JayF adds that to the reasons mlocate is undesirable | 23:19 | |
| *** jamielennox|away is now known as jamielennox | 23:20 | |
| fungi | it's probably useful on the right systems, but a fileserver with many millions of logfiles is not the right place for it | 23:20 |
| ianw | did puppet put it back? something drags it in? | 23:20 |
| *** aeng has joined #openstack-infra | 23:21 | |
| *** marst has quit IRC | 23:21 | |
| fungi | i don't recall how we disabled it in the past. probably by crippling its cronjob | 23:21 |
| fungi | but maybe that never made it into configuration manglement | 23:21 |
| *** annegentle has joined #openstack-infra | 23:21 | |
| *** spzala has quit IRC | 23:22 | |
| *** annegentle has quit IRC | 23:22 | |
| ianw | i guess we can just stuff things into updatedb.conf | 23:23 |
| *** EricGonczer_ has quit IRC | 23:24 | |
| ianw | it looks like findutils depends on it? | 23:24 |
| persia | findutils used to provide `locate`, which mlocate replaces. | 23:24 |
| ianw | anyone mind if i rm /var/lib/mlocate/*.db ? | 23:24 |
| persia | adding entries to PRUNEPATHS is usually the best way to make it not kill a system | 23:24 |
| ianw | cool, i can add that to puppet | 23:25 |
| *** EricGonczer_ has joined #openstack-infra | 23:25 | |
| ianw | ok, i just killed the >24 hour indexing process cron had started, and it cleaned up it's own temp file | 23:27 |
| ianw | so we've got a bit of space back for now | 23:27 |
| *** EricGonczer_ has quit IRC | 23:29 | |
| *** jtomasek has quit IRC | 23:31 | |
| jeblair | mirror.pypi release just finished | 23:33 |
| *** askb has joined #openstack-infra | 23:33 | |
| *** rwsu has quit IRC | 23:33 | |
| *** dizquierdo has joined #openstack-infra | 23:34 | |
| jeblair | so i think we're caught up to saturday; next release should catch up from there | 23:35 |
| jeblair | and bandersnatch just started again | 23:35 |
| *** jtomasek has joined #openstack-infra | 23:35 | |
| *** Swami has quit IRC | 23:36 | |
| pabelanger | can confirm | 23:39 |
| pabelanger | wheel mirrors still need to be released | 23:39 |
| *** wolverineav has joined #openstack-infra | 23:41 | |
| *** dimtruck is now known as zz_dimtruck | 23:42 | |
| *** zz_dimtruck is now known as dimtruck | 23:42 | |
| *** rerobot is now known as redrobot | 23:43 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/jeepyb master: Continue to optimize manage-projects for resiliency https://review.openstack.org/433333 | 23:44 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Remove mlocate from static.o.o https://review.openstack.org/433334 | 23:45 |
| *** wolverineav has quit IRC | 23:46 | |
| *** MasterOfBugs has quit IRC | 23:46 | |
| ianw | now, where was i ... i was going to check logs to see if anything much looks at periodic coverage job output | 23:46 |
| *** srwilkers_ has joined #openstack-infra | 23:48 | |
| *** srwilkers has quit IRC | 23:48 | |
| *** srwilkers_ is now known as srwilkers | 23:48 | |
| clarkb | mordred: so we'll only update the github description/homepage if some other flag indicates we should update things? | 23:49 |
| *** Swami has joined #openstack-infra | 23:49 | |
| *** spzala has joined #openstack-infra | 23:52 | |
| clarkb | mordred: left comment releated to ^ on change | 23:53 |
| *** Julien-z_ has quit IRC | 23:53 | |
| mordred | clarkb: right now we will not update them at all | 23:54 |
| mordred | oh - yeah | 23:54 |
| mordred | what you said is a better way to say that | 23:54 |
| mordred | testing their current state is the thing that kill us with gh rate limiting | 23:54 |
| clarkb | mordred: right, so are we just asserting issues/wiki/downloads is more important? or is that something we don't actiively check we just assert? | 23:54 |
| clarkb | mordred: trying to understand how those are different from homepage/description | 23:55 |
| mordred | nope. all of the values for openstack are the same as the default - which is "False" | 23:55 |
| mordred | so the checks for those will skip actually checking the github values | 23:55 |
| mordred | for us | 23:55 |
| mordred | clarkb: we're _still_ not perfect, fwiw | 23:56 |
| clarkb | mordred: gotcha we basically noop when false and only need to do things if true? | 23:56 |
| *** spzala has quit IRC | 23:56 | |
| clarkb | which means we won't coverage an errant project but those shouldn't exist in the first place | 23:56 |
| *** kgiusti has left #openstack-infra | 23:56 | |
| *** wolverineav has joined #openstack-infra | 23:57 | |
| clarkb | pabelanger: looking at https://review.openstack.org/#/c/430329/7/manifests/init.pp and the stack behind it doesn't that still require us to set install launcher to true for v2? | 23:57 |
| clarkb | pabelanger: however in the stack behind we aren't doing that anymore | 23:57 |
| *** aeng has quit IRC | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
