| jeblair | pabelanger: i don't understand -- we shouldn't be changing ~jenkins/.ssh/authorized_keys | 00:00 |
|---|---|---|
| *** tkelsey has joined #openstack-infra | 00:01 | |
| *** _ari_ has joined #openstack-infra | 00:01 | |
| fungi | yeah, at worst, the job should just `cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys` | 00:01 |
| *** _ari_ has quit IRC | 00:01 | |
| *** thorst has quit IRC | 00:02 | |
| *** aeng has quit IRC | 00:03 | |
| *** camunoz has quit IRC | 00:03 | |
| *** pahuang has quit IRC | 00:03 | |
| *** dingyichen has joined #openstack-infra | 00:04 | |
| *** ihrachys has quit IRC | 00:04 | |
| fungi | jeblair: notmorgan: quick question inline on 326758. if you can confirm, i'm happy to approve | 00:04 |
| *** tkelsey has quit IRC | 00:05 | |
| *** EricGonczer_ has joined #openstack-infra | 00:05 | |
| *** Swami has quit IRC | 00:05 | |
| *** sflanigan has quit IRC | 00:06 | |
| fungi | pabelanger: at http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/macros.yaml#n812 it looks like we only append to the known_hosts file, so that shouldn't be the cause? | 00:06 |
| *** Sukhdev has quit IRC | 00:07 | |
| jeblair | fungi: i *think* that's the case, though i would like notmorgan to confirm. | 00:07 |
| pabelanger | jeblair: fungi: Ya, scratch that. | 00:07 |
| *** IlyaG has joined #openstack-infra | 00:08 | |
| pabelanger | but it is related to our public key | 00:08 |
| notmorgan | fungi: looking | 00:08 |
| fungi | jeblair: notmorgan: also, comments are entirely valid in ~/.ssh/config | 00:08 |
| fungi | if you wanted to include some explaining what's there | 00:08 |
| pabelanger | Warning: Permanently added '104.130.204.34' (RSA) to the list of known hosts.\r\nPermission denied (publickey) | 00:08 |
| *** Sukhdev has joined #openstack-infra | 00:08 | |
| fungi | but i'm fine approving without too | 00:08 |
| notmorgan | fungi: correct | 00:08 |
| notmorgan | fungi: that is supposed to be slaves | 00:08 |
| notmorgan | fungi: it is relying on the assumption you outlined. | 00:08 |
| fungi | okay, did you want to add comments? now or in a separate change? | 00:08 |
| jeblair | fungi, pabelanger: i see that appending to *root's* authorized_keys file, which is fine -- it's appending (which is good) and zuul logs in as *jenkins* anyway... | 00:08 |
| notmorgan | fungi: happy to add a comment. | 00:09 |
| pabelanger | jeblair: I agree | 00:09 |
| fungi | notmorgan: if you feel a comment is appropriate there (i.e. expect people to scratch their heads down the road when looking at it) then please add a comment, but i could just be overly questiony | 00:10 |
| fungi | jeblair: pabelanger: yep, not even the jenkins user, so doubly irrelevant | 00:10 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/system-config: Add zuul-launcher ssh config file https://review.openstack.org/326758 | 00:11 |
| *** tphummel has quit IRC | 00:11 | |
| notmorgan | fungi: ^ | 00:11 |
| notmorgan | whoopse that is abad comment | 00:11 |
| notmorgan | i say ip address twice :P | 00:11 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/system-config: Add zuul-launcher ssh config file https://review.openstack.org/326758 | 00:11 |
| notmorgan | there | 00:11 |
| pabelanger | fungi: jeblair: This is the only jobs where we restart sshd, to pick up new config options. That's the next place to check I think | 00:12 |
| fungi | notmorgan: thanks--apprividado! | 00:12 |
| *** ffrank has quit IRC | 00:13 | |
| fungi | pabelanger: the log collection is a separate ssh call, so presumably checking logs (server side) and debug output (client side) would elucidate? | 00:13 |
| *** ffrank has joined #openstack-infra | 00:13 | |
| *** PollyZ has joined #openstack-infra | 00:15 | |
| *** sflanigan has joined #openstack-infra | 00:16 | |
| pabelanger | fungi: Ya, we need to see server side and why it is rejecting our key | 00:17 |
| *** kaisers1 has joined #openstack-infra | 00:17 | |
| *** kaisers has quit IRC | 00:19 | |
| *** PollyZ has quit IRC | 00:20 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Don't skip errors in postci https://review.openstack.org/326759 | 00:21 |
| *** claudiub has quit IRC | 00:21 | |
| *** aeng has joined #openstack-infra | 00:21 | |
| *** claudiub has joined #openstack-infra | 00:21 | |
| *** EricGonczer_ has quit IRC | 00:21 | |
| *** ihrachys has joined #openstack-infra | 00:22 | |
| *** pahuang has joined #openstack-infra | 00:23 | |
| pabelanger | #status log nodepool.o.o restarted to pick up review 326114 | 00:24 |
| *** Jeffrey4l has quit IRC | 00:24 | |
| openstackstatus | pabelanger: finished logging | 00:24 |
| *** mtanino has quit IRC | 00:27 | |
| *** david-lyle has quit IRC | 00:28 | |
| *** david-lyle has joined #openstack-infra | 00:28 | |
| *** PollyZ has joined #openstack-infra | 00:29 | |
| openstackgerrit | Merged openstack-infra/system-config: Add zuul-launcher ssh config file https://review.openstack.org/326758 | 00:29 |
| *** amotoki has joined #openstack-infra | 00:31 | |
| *** david-lyle has quit IRC | 00:33 | |
| *** PollyZ has quit IRC | 00:33 | |
| *** ihrachys has quit IRC | 00:33 | |
| *** amotoki has quit IRC | 00:36 | |
| *** anteaya has quit IRC | 00:36 | |
| *** Sukhdev has quit IRC | 00:37 | |
| openstackgerrit | Paul Van Eck proposed openstack-infra/project-config: Add pushMerge access to refstack-release group https://review.openstack.org/326770 | 00:38 |
| *** ihrachys has joined #openstack-infra | 00:39 | |
| *** pvaneck has quit IRC | 00:40 | |
| *** amitgandhinz has joined #openstack-infra | 00:40 | |
| *** PollyZ has joined #openstack-infra | 00:42 | |
| *** Goneri has joined #openstack-infra | 00:42 | |
| *** amitgandhinz has quit IRC | 00:45 | |
| *** PollyZ has quit IRC | 00:47 | |
| *** thorst has joined #openstack-infra | 00:48 | |
| *** rhallisey has quit IRC | 00:48 | |
| openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate: DNM: experimenting with local.conf https://review.openstack.org/326585 | 00:51 |
| *** salv-orlando has quit IRC | 00:53 | |
| *** salv-orlando has joined #openstack-infra | 00:53 | |
| *** nwkarsten has joined #openstack-infra | 00:55 | |
| *** nwkarste_ has joined #openstack-infra | 00:56 | |
| *** PollyZ has joined #openstack-infra | 00:56 | |
| *** baoli has joined #openstack-infra | 00:57 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool: Add nodepool.manager.MANAGER.nodes.STATE also https://review.openstack.org/326774 | 00:58 |
| pabelanger | jeblair: jhesketh: ^ follow up patch to for more zuul-launcher stats. I believe that is everything | 00:59 |
| *** nwkarsten has quit IRC | 00:59 | |
| *** tphummel has joined #openstack-infra | 01:00 | |
| *** PollyZ has quit IRC | 01:00 | |
| jhesketh | pabelanger: looking | 01:01 |
| *** ihrachys has quit IRC | 01:01 | |
| *** shashank_hegde has quit IRC | 01:02 | |
| *** ams__ has quit IRC | 01:03 | |
| *** ihrachys has joined #openstack-infra | 01:04 | |
| *** zhurong has joined #openstack-infra | 01:07 | |
| *** Goneri has quit IRC | 01:07 | |
| *** ihrachys has quit IRC | 01:09 | |
| *** PollyZ has joined #openstack-infra | 01:09 | |
| *** jerryz has quit IRC | 01:10 | |
| *** ffrank has quit IRC | 01:12 | |
| *** markvoelker has joined #openstack-infra | 01:13 | |
| *** PollyZ has quit IRC | 01:14 | |
| *** hongbin has joined #openstack-infra | 01:15 | |
| *** tqtran has quit IRC | 01:16 | |
| *** zz_dimtruck is now known as dimtruck | 01:16 | |
| *** markvoelker has quit IRC | 01:18 | |
| *** Apoorva has quit IRC | 01:23 | |
| *** PollyZ has joined #openstack-infra | 01:23 | |
| *** psilvad has quit IRC | 01:23 | |
| *** Daisy has joined #openstack-infra | 01:25 | |
| *** ramishra has joined #openstack-infra | 01:26 | |
| *** sputnik13 has quit IRC | 01:27 | |
| *** sputnik13 has joined #openstack-infra | 01:28 | |
| openstackgerrit | Eli Qiao proposed openstack-infra/project-config: Zun: rename higgins to zun https://review.openstack.org/326306 | 01:28 |
| *** PollyZ has quit IRC | 01:28 | |
| *** psilvad has joined #openstack-infra | 01:31 | |
| *** amotoki has joined #openstack-infra | 01:32 | |
| *** sputnik13 has quit IRC | 01:33 | |
| *** amotoki has quit IRC | 01:36 | |
| *** PollyZ has joined #openstack-infra | 01:37 | |
| *** tzn has joined #openstack-infra | 01:38 | |
| *** gouthamr has quit IRC | 01:40 | |
| *** amitgandhinz has joined #openstack-infra | 01:41 | |
| *** PollyZ has quit IRC | 01:41 | |
| *** signed8bit_Zzz is now known as signed8bit | 01:43 | |
| *** tzn has quit IRC | 01:43 | |
| *** signed8b_ has joined #openstack-infra | 01:44 | |
| *** amitgandhinz has quit IRC | 01:45 | |
| *** Daisy has quit IRC | 01:46 | |
| *** Daisy has joined #openstack-infra | 01:46 | |
| *** signed8bit has quit IRC | 01:47 | |
| *** banix has joined #openstack-infra | 01:47 | |
| *** Daisy has quit IRC | 01:51 | |
| *** IlyaG has quit IRC | 01:51 | |
| *** mriedem has quit IRC | 01:52 | |
| *** yamahata has quit IRC | 01:53 | |
| *** r-mibu has quit IRC | 01:57 | |
| *** r-mibu has joined #openstack-infra | 01:57 | |
| *** gildub has joined #openstack-infra | 01:58 | |
| *** thorst has quit IRC | 01:59 | |
| *** signed8b_ is now known as signed8bit_Zzz | 02:00 | |
| *** thorst has joined #openstack-infra | 02:00 | |
| *** psilvad has quit IRC | 02:00 | |
| *** akshai has quit IRC | 02:01 | |
| *** tkelsey has joined #openstack-infra | 02:02 | |
| *** tkelsey has quit IRC | 02:06 | |
| *** Julien-zte has joined #openstack-infra | 02:08 | |
| *** thorst has quit IRC | 02:08 | |
| *** PollyZ has joined #openstack-infra | 02:09 | |
| *** mdrabe has joined #openstack-infra | 02:10 | |
| *** cody-somerville has joined #openstack-infra | 02:10 | |
| *** tonytan4ever has joined #openstack-infra | 02:11 | |
| *** Julien-zte has quit IRC | 02:11 | |
| *** akshai has joined #openstack-infra | 02:12 | |
| *** shashank_hegde has joined #openstack-infra | 02:12 | |
| *** pblaho has joined #openstack-infra | 02:15 | |
| *** aeng has quit IRC | 02:16 | |
| *** PollyZ has quit IRC | 02:24 | |
| *** Daisy has joined #openstack-infra | 02:26 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool: Add nodepool.manager.MANAGER.nodes.STATE also https://review.openstack.org/326774 | 02:27 |
| *** LaT0rtue has quit IRC | 02:27 | |
| *** PollyZ has joined #openstack-infra | 02:28 | |
| *** cody-somerville has quit IRC | 02:28 | |
| *** aeng has joined #openstack-infra | 02:29 | |
| *** yamamoto_ has joined #openstack-infra | 02:29 | |
| *** sdake has joined #openstack-infra | 02:32 | |
| *** amotoki has joined #openstack-infra | 02:32 | |
| *** PollyZ has quit IRC | 02:33 | |
| *** ramishra_ has joined #openstack-infra | 02:33 | |
| openstackgerrit | Merged openstack-infra/puppet-jeepyb: Revert "Add missing packages to jeepyb install" https://review.openstack.org/326731 | 02:34 |
| *** mtanino has joined #openstack-infra | 02:34 | |
| *** sdake has quit IRC | 02:34 | |
| *** ramishra has quit IRC | 02:36 | |
| *** amotoki has quit IRC | 02:37 | |
| *** gildub has quit IRC | 02:38 | |
| *** woodster_ has quit IRC | 02:38 | |
| *** jyuso1 has joined #openstack-infra | 02:40 | |
| *** amitgandhinz has joined #openstack-infra | 02:42 | |
| *** Daisy has quit IRC | 02:45 | |
| *** Daisy has joined #openstack-infra | 02:46 | |
| *** Daisy_ has joined #openstack-infra | 02:46 | |
| *** amitgandhinz has quit IRC | 02:47 | |
| *** openstackgerrit has quit IRC | 02:47 | |
| *** openstackgerrit has joined #openstack-infra | 02:48 | |
| *** Daisy has quit IRC | 02:50 | |
| *** gyee has quit IRC | 02:50 | |
| *** grassazz has joined #openstack-infra | 02:51 | |
| *** shashank_hegde has quit IRC | 02:52 | |
| *** ihrachys has joined #openstack-infra | 02:53 | |
| *** dchen has joined #openstack-infra | 02:55 | |
| *** ihrachys has quit IRC | 02:59 | |
| *** shashank_hegde has joined #openstack-infra | 03:03 | |
| *** rossella_s has quit IRC | 03:04 | |
| *** mdrabe has quit IRC | 03:05 | |
| *** thorst has joined #openstack-infra | 03:06 | |
| *** mtanino has quit IRC | 03:08 | |
| *** PollyZ has joined #openstack-infra | 03:09 | |
| *** thorst has quit IRC | 03:13 | |
| *** PollyZ has quit IRC | 03:14 | |
| *** shashank_hegde has quit IRC | 03:17 | |
| *** hongbin has quit IRC | 03:18 | |
| *** Daisy_ has quit IRC | 03:20 | |
| *** Daisy has joined #openstack-infra | 03:20 | |
| *** PollyZ has joined #openstack-infra | 03:23 | |
| openstackgerrit | Tin Lam proposed openstack-infra/project-config: Add releasenotes jobs for tacker https://review.openstack.org/326801 | 03:23 |
| *** gildub has joined #openstack-infra | 03:25 | |
| *** shashank_hegde has joined #openstack-infra | 03:25 | |
| *** tzn has joined #openstack-infra | 03:27 | |
| *** pblaho has quit IRC | 03:27 | |
| *** grassazz has quit IRC | 03:27 | |
| *** PollyZ has quit IRC | 03:28 | |
| *** grassaSS has joined #openstack-infra | 03:28 | |
| *** sdake has joined #openstack-infra | 03:28 | |
| *** dimtruck is now known as zz_dimtruck | 03:28 | |
| *** Daisy_ has joined #openstack-infra | 03:30 | |
| *** tzn has quit IRC | 03:31 | |
| *** Daisy has quit IRC | 03:33 | |
| *** gildub has quit IRC | 03:33 | |
| *** amotoki has joined #openstack-infra | 03:34 | |
| *** gildub has joined #openstack-infra | 03:34 | |
| *** sflanigan has quit IRC | 03:34 | |
| *** aeng has quit IRC | 03:34 | |
| *** links has joined #openstack-infra | 03:35 | |
| *** PollyZ has joined #openstack-infra | 03:36 | |
| *** tlian has quit IRC | 03:37 | |
| *** amotoki has quit IRC | 03:38 | |
| *** binbincong has joined #openstack-infra | 03:40 | |
| *** PollyZ has quit IRC | 03:41 | |
| *** furlongm has quit IRC | 03:41 | |
| *** furlongm has joined #openstack-infra | 03:42 | |
| *** amitgandhinz has joined #openstack-infra | 03:43 | |
| *** shashank_hegde has quit IRC | 03:43 | |
| *** julim has quit IRC | 03:45 | |
| *** amitgandhinz has quit IRC | 03:47 | |
| *** rlandy has quit IRC | 03:48 | |
| *** PollyZ has joined #openstack-infra | 03:50 | |
| *** sflanigan has joined #openstack-infra | 03:50 | |
| *** aeng has joined #openstack-infra | 03:51 | |
| *** sdake has quit IRC | 03:53 | |
| *** PollyZ has quit IRC | 03:54 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: disable voting on puppet-ceph beaker jobs https://review.openstack.org/326591 | 03:58 |
| *** Daisy_ has quit IRC | 03:59 | |
| *** Daisy has joined #openstack-infra | 04:00 | |
| *** shashank_hegde has joined #openstack-infra | 04:05 | |
| *** tphummel has quit IRC | 04:05 | |
| jamielennox | hey, i just got openstack-dev/os-http onto infra - can someone add me to the os-http-core group? | 04:06 |
| *** tonytan_brb has joined #openstack-infra | 04:07 | |
| *** tonytan4ever has quit IRC | 04:08 | |
| *** claudiub has quit IRC | 04:10 | |
| *** devananda has quit IRC | 04:11 | |
| *** thorst has joined #openstack-infra | 04:11 | |
| jamielennox | oh, and os-http-release group too please | 04:14 |
| *** Daisy has quit IRC | 04:18 | |
| *** Daisy has joined #openstack-infra | 04:18 | |
| *** thorst has quit IRC | 04:18 | |
| *** devananda has joined #openstack-infra | 04:23 | |
| *** Daisy has quit IRC | 04:23 | |
| *** salv-orl_ has joined #openstack-infra | 04:24 | |
| *** psachin has joined #openstack-infra | 04:25 | |
| *** salv-orlando has quit IRC | 04:27 | |
| *** PollyZ has joined #openstack-infra | 04:31 | |
| *** grassaSS has quit IRC | 04:33 | |
| *** Sukhdev has joined #openstack-infra | 04:33 | |
| *** baoli has quit IRC | 04:34 | |
| *** amotoki has joined #openstack-infra | 04:34 | |
| *** PollyZ has quit IRC | 04:35 | |
| *** tonytan_brb has quit IRC | 04:36 | |
| *** amotoki has quit IRC | 04:39 | |
| *** tphummel has joined #openstack-infra | 04:41 | |
| *** ihrachys has joined #openstack-infra | 04:41 | |
| *** amitgandhinz has joined #openstack-infra | 04:43 | |
| *** ihrachys has quit IRC | 04:45 | |
| *** amitgandhinz has quit IRC | 04:48 | |
| *** jaosorior has joined #openstack-infra | 04:50 | |
| *** savihou has joined #openstack-infra | 04:50 | |
| *** banix has quit IRC | 04:50 | |
| *** sshnaidm|afk has quit IRC | 04:51 | |
| *** njohnston is now known as njohnston|sleep | 04:53 | |
| *** yamahata has joined #openstack-infra | 04:55 | |
| *** sarob has joined #openstack-infra | 04:55 | |
| *** PollyZ has joined #openstack-infra | 04:58 | |
| *** sarob has quit IRC | 05:00 | |
| *** sdake has joined #openstack-infra | 05:01 | |
| *** shashank_hegde has quit IRC | 05:02 | |
| *** PollyZ has quit IRC | 05:02 | |
| openstackgerrit | Nate Johnston proposed openstack-infra/project-config: Add accessbot for #openstack-firewall https://review.openstack.org/326821 | 05:02 |
| *** salv-orl_ has quit IRC | 05:03 | |
| *** salv-orlando has joined #openstack-infra | 05:03 | |
| *** tonytan4ever has joined #openstack-infra | 05:03 | |
| *** Daisy has joined #openstack-infra | 05:05 | |
| *** tonytan4ever has quit IRC | 05:09 | |
| *** oanson has joined #openstack-infra | 05:09 | |
| *** gmann has quit IRC | 05:10 | |
| *** PollyZ has joined #openstack-infra | 05:11 | |
| *** gmann has joined #openstack-infra | 05:12 | |
| *** tzn has joined #openstack-infra | 05:15 | |
| *** savihou has quit IRC | 05:15 | |
| *** nwkarste_ has quit IRC | 05:15 | |
| *** PollyZ has quit IRC | 05:16 | |
| *** savihou has joined #openstack-infra | 05:16 | |
| *** thorst has joined #openstack-infra | 05:16 | |
| *** savihou has quit IRC | 05:17 | |
| *** shashank_hegde has joined #openstack-infra | 05:18 | |
| *** tzn has quit IRC | 05:20 | |
| *** savihou has joined #openstack-infra | 05:21 | |
| *** thorst has quit IRC | 05:23 | |
| *** PollyZ has joined #openstack-infra | 05:25 | |
| *** hdaniel has joined #openstack-infra | 05:25 | |
| *** PollyZ has quit IRC | 05:30 | |
| notmorgan | jamielennox: you can also email openstack-infra ML and someone will def. add you | 05:32 |
| *** sdake has quit IRC | 05:32 | |
| notmorgan | jamielennox: if you don't want to rely on async nature of IRC | 05:32 |
| *** fawadkhaliq has joined #openstack-infra | 05:33 | |
| *** tonytan4ever has joined #openstack-infra | 05:33 | |
| *** amotoki has joined #openstack-infra | 05:35 | |
| *** tonytan4ever has quit IRC | 05:38 | |
| *** PollyZ has joined #openstack-infra | 05:39 | |
| *** amotoki has quit IRC | 05:39 | |
| *** tphummel has quit IRC | 05:39 | |
| *** PollyZ has quit IRC | 05:43 | |
| *** amitgandhinz has joined #openstack-infra | 05:44 | |
| *** amitgandhinz has quit IRC | 05:49 | |
| nibalizer | jamielennox: is done | 05:49 |
| notmorgan | jamielennox: or nibalizer is on it | 05:50 |
| notmorgan | cause #nibalizer | 05:50 |
| notmorgan | :P | 05:50 |
| *** PollyZ has joined #openstack-infra | 05:52 | |
| *** mtanino has joined #openstack-infra | 05:56 | |
| *** PollyZ has quit IRC | 05:57 | |
| *** binbincong has quit IRC | 05:58 | |
| *** lvdongbing has joined #openstack-infra | 05:58 | |
| *** jtomasek has quit IRC | 06:01 | |
| lvdongbing | Hello, everyone:), I proposed a commit to add Bilean team meeting to irc-meeting, could anyone help me review it: https://review.openstack.org/#/c/326271/ | 06:02 |
| *** tkelsey has joined #openstack-infra | 06:03 | |
| *** andreas_s has joined #openstack-infra | 06:04 | |
| *** PollyZ has joined #openstack-infra | 06:06 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/326838 | 06:07 |
| *** tkelsey has quit IRC | 06:08 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Move puppet-congress to OpenStack https://review.openstack.org/326720 | 06:10 |
| *** pfallenop has quit IRC | 06:11 | |
| *** PollyZ has quit IRC | 06:11 | |
| *** Daisy has quit IRC | 06:12 | |
| *** lifeless has quit IRC | 06:13 | |
| *** rcernin has joined #openstack-infra | 06:13 | |
| *** Daisy has joined #openstack-infra | 06:13 | |
| *** LaT0rtue has joined #openstack-infra | 06:13 | |
| *** pfallenop has joined #openstack-infra | 06:14 | |
| *** sflanigan has quit IRC | 06:16 | |
| AJaeger | infra-root, https://github.com/openstack-dev/os-http has not been imported yet - could somebody check what's blocking and retrigger, please? It exists fine here: http://git.openstack.org/cgit/openstack-dev/os-http | 06:17 |
| *** Daisy_ has joined #openstack-infra | 06:17 | |
| *** Daisy has quit IRC | 06:17 | |
| *** aeng has quit IRC | 06:18 | |
| *** Daisy_ has quit IRC | 06:19 | |
| *** Daisy has joined #openstack-infra | 06:19 | |
| *** abregman|nb has quit IRC | 06:19 | |
| *** PollyZ has joined #openstack-infra | 06:19 | |
| *** bhavik has joined #openstack-infra | 06:20 | |
| *** thorst has joined #openstack-infra | 06:21 | |
| *** Daisy has quit IRC | 06:21 | |
| *** Daisy_ has joined #openstack-infra | 06:21 | |
| *** e0ne has joined #openstack-infra | 06:21 | |
| *** nadya has joined #openstack-infra | 06:22 | |
| *** PollyZ has quit IRC | 06:25 | |
| yolanda | hi AJaeger, should be the normal groups issue | 06:25 |
| yolanda | going to check | 06:25 |
| AJaeger | thanks, yolanda | 06:28 |
| yolanda | ok added | 06:28 |
| *** thorst has quit IRC | 06:29 | |
| *** sflanigan has joined #openstack-infra | 06:29 | |
| *** sflanigan has joined #openstack-infra | 06:29 | |
| *** purp has quit IRC | 06:30 | |
| *** faizy has joined #openstack-infra | 06:31 | |
| *** aeng has joined #openstack-infra | 06:31 | |
| *** PollyZ has joined #openstack-infra | 06:33 | |
| *** Sukhdev has quit IRC | 06:33 | |
| *** purp has joined #openstack-infra | 06:34 | |
| *** PollyZ has quit IRC | 06:37 | |
| *** nadya has quit IRC | 06:41 | |
| *** sflanigan has quit IRC | 06:42 | |
| openstackgerrit | Merged openstack-infra/project-config: Move puppet-congress to OpenStack https://review.openstack.org/326720 | 06:42 |
| *** claudiub has joined #openstack-infra | 06:44 | |
| openstackgerrit | Merged openstack-infra/project-config: Add experimental searchlight ES 2.x https://review.openstack.org/326081 | 06:45 |
| openstackgerrit | Merged openstack-infra/project-config: Fix span for jenkins / zuul launcher to a single row https://review.openstack.org/326761 | 06:45 |
| *** amitgandhinz has joined #openstack-infra | 06:45 | |
| *** nadya has joined #openstack-infra | 06:45 | |
| *** jpeeler has quit IRC | 06:47 | |
| *** openstackgerrit has quit IRC | 06:48 | |
| *** ajmiller_ has quit IRC | 06:48 | |
| *** openstackgerrit has joined #openstack-infra | 06:48 | |
| *** ajmiller_ has joined #openstack-infra | 06:48 | |
| *** amitgandhinz has quit IRC | 06:49 | |
| *** Daisy_ has quit IRC | 06:51 | |
| *** Daisy has joined #openstack-infra | 06:52 | |
| *** gildub has quit IRC | 06:52 | |
| openstackgerrit | Merged openstack-infra/project-config: Add pushMerge access to refstack-release group https://review.openstack.org/326770 | 06:52 |
| *** Julien-zte has joined #openstack-infra | 06:53 | |
| *** nadya has quit IRC | 06:53 | |
| *** sdake has joined #openstack-infra | 06:54 | |
| *** Daisy has quit IRC | 06:54 | |
| *** Daisy has joined #openstack-infra | 06:54 | |
| *** jpeeler has joined #openstack-infra | 06:55 | |
| *** jpeeler has joined #openstack-infra | 06:55 | |
| openstackgerrit | Christian Schwede proposed openstack-infra/devstack-gate: Use subnodes for Swift storage nodes in a multinode setup https://review.openstack.org/304465 | 06:55 |
| *** sorantis has joined #openstack-infra | 06:56 | |
| *** sflanigan has joined #openstack-infra | 06:57 | |
| *** tesseract has joined #openstack-infra | 07:00 | |
| *** pahuang has quit IRC | 07:01 | |
| *** gildub has joined #openstack-infra | 07:01 | |
| openstackgerrit | Merged openstack-infra/project-config: disable voting on puppet-ceph beaker jobs https://review.openstack.org/326591 | 07:02 |
| *** gildub has quit IRC | 07:02 | |
| *** salv-orlando has quit IRC | 07:02 | |
| *** gildub has joined #openstack-infra | 07:03 | |
| openstackgerrit | Merged openstack-infra/project-config: puppet: rename PUPPET_VERSION to PUPPET_MAJ_VERSION https://review.openstack.org/326738 | 07:03 |
| *** tzn has joined #openstack-infra | 07:03 | |
| *** ajmiller_ has quit IRC | 07:03 | |
| *** ajmiller has joined #openstack-infra | 07:03 | |
| *** salv-orlando has joined #openstack-infra | 07:03 | |
| *** amoralej|off is now known as amoralej | 07:04 | |
| openstackgerrit | Merged openstack-infra/project-config: Fix branch for releasenotes job of oslo.messaging https://review.openstack.org/326687 | 07:05 |
| *** lvdongbing has quit IRC | 07:05 | |
| openstackgerrit | Merged openstack-infra/project-config: Add security-analysis documentation repository https://review.openstack.org/325049 | 07:06 |
| *** e0ne has quit IRC | 07:06 | |
| openstackgerrit | Merged openstack-infra/project-config: add new project meghdwar https://review.openstack.org/319466 | 07:07 |
| *** tzn has quit IRC | 07:07 | |
| *** sdake has quit IRC | 07:10 | |
| *** pahuang has joined #openstack-infra | 07:14 | |
| *** PollyZ has joined #openstack-infra | 07:14 | |
| *** zeih has joined #openstack-infra | 07:15 | |
| *** vincentll has joined #openstack-infra | 07:16 | |
| *** esikachev has joined #openstack-infra | 07:18 | |
| *** savihou has quit IRC | 07:18 | |
| *** Daisy has quit IRC | 07:18 | |
| *** Daisy has joined #openstack-infra | 07:19 | |
| *** flepied has joined #openstack-infra | 07:19 | |
| *** lvdongbing has joined #openstack-infra | 07:19 | |
| *** PollyZ has quit IRC | 07:19 | |
| *** mtanino has quit IRC | 07:19 | |
| *** ifarkas has joined #openstack-infra | 07:20 | |
| *** watanabe_isao has joined #openstack-infra | 07:20 | |
| *** Daisy has quit IRC | 07:22 | |
| *** sarob has joined #openstack-infra | 07:22 | |
| *** Daisy has joined #openstack-infra | 07:22 | |
| *** abregman|mtg has joined #openstack-infra | 07:23 | |
| *** LaT0rtue has quit IRC | 07:23 | |
| *** fawadkhaliq has quit IRC | 07:23 | |
| *** mixos has quit IRC | 07:24 | |
| *** mikelk has joined #openstack-infra | 07:25 | |
| AJaeger | yolanda: https://github.com/openstack-dev/os-http is still not synced - is anything broken or am I not waiting long enough? | 07:26 |
| *** thorst has joined #openstack-infra | 07:26 | |
| *** sarob has quit IRC | 07:26 | |
| openstackgerrit | LisaLi proposed openstack-infra/project-config: Add new job template for dsvm plugins with os-brick https://review.openstack.org/326879 | 07:27 |
| *** Daisy has quit IRC | 07:27 | |
| yolanda | AJaeger, i added permissions. I'll check the replication logs | 07:27 |
| *** matrohon has joined #openstack-infra | 07:29 | |
| *** jlanoux has joined #openstack-infra | 07:29 | |
| yolanda | i'm launching replication for that project manually | 07:30 |
| ianw | AJaeger: in scrollback, there was an issue with puppet this morning my time, i think pabelanger was looking at it | 07:31 |
| *** watanabe_isao has quit IRC | 07:31 | |
| *** ccamacho has joined #openstack-infra | 07:32 | |
| *** mnencia has quit IRC | 07:32 | |
| *** salv-orlando has quit IRC | 07:32 | |
| *** mnencia has joined #openstack-infra | 07:32 | |
| *** salv-orlando has joined #openstack-infra | 07:32 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update EnvInject plugin https://review.openstack.org/324140 | 07:32 |
| ianw | AJaeger / yolanda: i think it was https://review.openstack.org/326731 | 07:32 |
| ianw | but not sure what follow-up had to be done | 07:33 |
| yolanda | in the manual replication, it seems to be stuck syncing to github | 07:33 |
| *** thorst has quit IRC | 07:33 | |
| yolanda | ianw, i'll check proper package on centos 7, and repurpose it | 07:34 |
| *** lvdongbing has quit IRC | 07:34 | |
| *** ajmiller has quit IRC | 07:35 | |
| *** lvdongbing has joined #openstack-infra | 07:35 | |
| openstackgerrit | Merged openstack-infra/project-config: Add releasenotes jobs for tacker https://review.openstack.org/326801 | 07:36 |
| yolanda | mm, i see, starting on centos 6, libssl-dev has been replaced with openssl-devel | 07:36 |
| *** amotoki has joined #openstack-infra | 07:36 | |
| *** Liuqing has joined #openstack-infra | 07:37 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update EnvInject plugin https://review.openstack.org/324140 | 07:38 |
| *** shardy has joined #openstack-infra | 07:39 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-jeepyb: Add missing packages to jeepyb install The pip install currently failed because of missing libffi-dev and libssl-dev/openssl-devel packages https://review.openstack.org/326885 | 07:40 |
| *** amotoki has quit IRC | 07:41 | |
| ianw | yolanda: ^ commit msg looks a bit funny | 07:41 |
| AJaeger | ianw, yolanda : Thanks. | 07:41 |
| AJaeger | yolanda: seems you missed the empty line ;) | 07:41 |
| yolanda | ok amending | 07:41 |
| openstackgerrit | yolanda.robla proposed openstack-infra/puppet-jeepyb: Add missing packages to jeepyb install https://review.openstack.org/326885 | 07:42 |
| *** Daisy has joined #openstack-infra | 07:42 | |
| yolanda | i also want to take a deeper look at jeepyb, pip install needs a require all these packages. Currently has a race there, and jeepyb only installs at second puppet run | 07:42 |
| *** dtantsur|afk is now known as dtantsur | 07:44 | |
| *** rossella_s has joined #openstack-infra | 07:44 | |
| openstackgerrit | Di XiaoLi proposed openstack-infra/project-config: Add a heat job to use devstack plugin https://review.openstack.org/317811 | 07:44 |
| *** amitgandhinz has joined #openstack-infra | 07:46 | |
| *** arxcruz has joined #openstack-infra | 07:47 | |
| yolanda | AJaeger, still checking. There may be some problem with github and this project. Because replication to other github projects is fine, and for this one, it times out | 07:47 |
| *** Daisy has quit IRC | 07:48 | |
| *** openstackgerrit has quit IRC | 07:48 | |
| *** Daisy has joined #openstack-infra | 07:48 | |
| *** openstackgerrit has joined #openstack-infra | 07:48 | |
| yolanda | worked now | 07:49 |
| AJaeger | thanks, yolanda ! | 07:50 |
| *** amitgandhinz has quit IRC | 07:50 | |
| *** Jeffrey4l has joined #openstack-infra | 07:50 | |
| ianw | great, that will save the dib builds tomorrow too | 07:50 |
| *** jlanoux_ has joined #openstack-infra | 07:50 | |
| *** Liuqing has quit IRC | 07:52 | |
| *** hferenc has joined #openstack-infra | 07:53 | |
| openstackgerrit | Di XiaoLi proposed openstack-infra/project-config: [WIP]Switch all heat jobs to use devstack plugin https://review.openstack.org/317817 | 07:53 |
| openstackgerrit | Di XiaoLi proposed openstack-infra/project-config: Add a heat job to use devstack plugin https://review.openstack.org/317811 | 07:53 |
| *** Daisy has quit IRC | 07:54 | |
| *** jlanoux has quit IRC | 07:54 | |
| *** PollyZ has joined #openstack-infra | 07:55 | |
| *** sorantis has quit IRC | 07:57 | |
| *** nmagnezi has joined #openstack-infra | 07:57 | |
| *** Daisy has joined #openstack-infra | 07:58 | |
| *** PollyZ has quit IRC | 08:00 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** zqfan has joined #openstack-infra | 08:00 | |
| *** sorantis has joined #openstack-infra | 08:01 | |
| *** zzzeek has joined #openstack-infra | 08:02 | |
| penguinolog | Hi! I seen "WARNING: The following packages cannot be authenticated!" in python coverage gate log - it's not looks good | 08:03 |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/326838 | 08:04 |
| *** ajmiller has joined #openstack-infra | 08:05 | |
| clarkb | penguinolog: we dont currently sign our apt package mirror indexes which causes the warning | 08:05 |
| *** jtomasek has joined #openstack-infra | 08:05 | |
| penguinolog | clarkb, thanks. Just always read warnings as pending errors | 08:06 |
| *** fawadkhaliq has joined #openstack-infra | 08:09 | |
| mordred | penguinolog: yes. it's a good way to think | 08:09 |
| *** salv-orlando has quit IRC | 08:10 | |
| *** ajmiller_ has joined #openstack-infra | 08:10 | |
| *** lucas-dinner is now known as lucasagomes | 08:10 | |
| *** salv-orlando has joined #openstack-infra | 08:11 | |
| *** dizquierdo has joined #openstack-infra | 08:11 | |
| *** Daisy has quit IRC | 08:11 | |
| *** Daisy has joined #openstack-infra | 08:12 | |
| *** Daisy has quit IRC | 08:13 | |
| *** hashar has joined #openstack-infra | 08:13 | |
| *** Daisy has joined #openstack-infra | 08:13 | |
| *** ajmiller has quit IRC | 08:14 | |
| *** dchen has quit IRC | 08:16 | |
| *** abregman|mtg has quit IRC | 08:16 | |
| *** yamahata has quit IRC | 08:16 | |
| *** dingyichen has quit IRC | 08:16 | |
| *** Daisy has quit IRC | 08:18 | |
| openstackgerrit | Merged openstack-infra/infra-ansible: Change the old meta puppetmaster group to the expected 'puppetmaster' group https://review.openstack.org/326389 | 08:19 |
| *** gildub has quit IRC | 08:19 | |
| openstackgerrit | Merged openstack-infra/infra-ansible: Change the old meta gerrit group to the expected 'gerrit' group https://review.openstack.org/326515 | 08:19 |
| openstackgerrit | Merged openstack-infra/infra-ansible: Change the old meta zuul group to the expected 'zuul' group https://review.openstack.org/326518 | 08:19 |
| *** Julien-zte has quit IRC | 08:19 | |
| openstackgerrit | Merged openstack-infra/infra-ansible: Change the old meta jenkins group to the expected 'jenkins' group https://review.openstack.org/326569 | 08:19 |
| openstackgerrit | Merged openstack-infra/project-config: New project request for SwiftOnHPSS https://review.openstack.org/316098 | 08:19 |
| openstackgerrit | Merged openstack-infra/project-config: Improve skip for Heat some more https://review.openstack.org/326444 | 08:19 |
| *** claudiub has quit IRC | 08:20 | |
| *** Daisy has joined #openstack-infra | 08:21 | |
| openstackgerrit | Merged openstack-infra/project-config: Repair ironic-inspector ability to run its own tempest job https://review.openstack.org/325825 | 08:22 |
| *** jaosorior has quit IRC | 08:25 | |
| *** jaosorior has joined #openstack-infra | 08:25 | |
| *** aarefiev_ is now known as aarefiev | 08:26 | |
| *** abregman|mtg has joined #openstack-infra | 08:28 | |
| clarkb | yolanda: was that jeepyb thing related to a dep like cryptography updating? | 08:35 |
| *** PollyZ has joined #openstack-infra | 08:36 | |
| *** dmk0202 has joined #openstack-infra | 08:37 | |
| *** amotoki has joined #openstack-infra | 08:37 | |
| *** abregman|mtg is now known as abregman | 08:38 | |
| yolanda | clarkb, let me check install logs | 08:38 |
| *** PollyZ has quit IRC | 08:41 | |
| *** amotoki has quit IRC | 08:42 | |
| *** tzn has joined #openstack-infra | 08:42 | |
| *** derekh has joined #openstack-infra | 08:42 | |
| *** tzn has quit IRC | 08:43 | |
| *** fawadkhaliq has quit IRC | 08:45 | |
| *** amitgandhinz has joined #openstack-infra | 08:46 | |
| *** tzn has joined #openstack-infra | 08:47 | |
| *** openstackgerrit has quit IRC | 08:48 | |
| *** openstackgerrit has joined #openstack-infra | 08:48 | |
| *** PollyZ has joined #openstack-infra | 08:49 | |
| *** andreww has joined #openstack-infra | 08:49 | |
| *** e0ne has joined #openstack-infra | 08:50 | |
| *** amitgandhinz has quit IRC | 08:51 | |
| openstackgerrit | Gal Sagie proposed openstack-infra/project-config: Add networking-l2gw fullstack job https://review.openstack.org/326300 | 08:51 |
| yolanda | clarkb http://paste.openstack.org/show/508870/ - cryptography deps | 08:51 |
| openstackgerrit | Merged openstack-infra/project-config: remove duplicate jobs for aodh and gnocchi https://review.openstack.org/326589 | 08:51 |
| *** xarses_ has quit IRC | 08:52 | |
| *** kzaitsev_mb has joined #openstack-infra | 08:53 | |
| clarkb | yolanda: ok, thats what I figured but wanted to make sure we understood the underlying cause | 08:53 |
| *** tkelsey has joined #openstack-infra | 08:54 | |
| *** PollyZ has quit IRC | 08:54 | |
| *** dtantsur is now known as dtantsur|bbl | 08:57 | |
| *** dtardivel has joined #openstack-infra | 09:00 | |
| *** jyuso1 has quit IRC | 09:02 | |
| *** tosky has joined #openstack-infra | 09:02 | |
| *** tosky has quit IRC | 09:02 | |
| *** sorantis has quit IRC | 09:02 | |
| *** jtomasek has quit IRC | 09:04 | |
| *** markusry has joined #openstack-infra | 09:05 | |
| *** tosky has joined #openstack-infra | 09:05 | |
| *** arxcruz has quit IRC | 09:05 | |
| *** arxcruz has joined #openstack-infra | 09:06 | |
| *** electrofelix has joined #openstack-infra | 09:13 | |
| *** esikachev has quit IRC | 09:14 | |
| *** sdake has joined #openstack-infra | 09:16 | |
| *** PollyZ has joined #openstack-infra | 09:16 | |
| *** esikachev has joined #openstack-infra | 09:17 | |
| *** jtomasek has joined #openstack-infra | 09:18 | |
| *** shashank_hegde has quit IRC | 09:20 | |
| *** PollyZ has quit IRC | 09:21 | |
| *** ajmiller__ has joined #openstack-infra | 09:22 | |
| skraynev | yolanda: hi. could you help me with one magic thing: we have a periodic job http://logs.openstack.org/periodic/gate-rally-dsvm-fakevirt-heat/ and it fails due to strange error always: /opt/stack/new/devstack-gate/devstack-vm-gate.sh: No such file or directory may be periodic job should have some specific configuration ? | 09:24 |
| yolanda | skraynev, looking | 09:25 |
| *** ajmiller_ has quit IRC | 09:26 | |
| *** Daisy has quit IRC | 09:26 | |
| *** nadya has joined #openstack-infra | 09:27 | |
| *** Daisy has joined #openstack-infra | 09:27 | |
| *** PollyZ has joined #openstack-infra | 09:30 | |
| *** Daisy_ has joined #openstack-infra | 09:31 | |
| *** fawadkhaliq has joined #openstack-infra | 09:32 | |
| *** Daisy has quit IRC | 09:32 | |
| sambetts|afk | bkero: Thanks for the update on the SSO service :) | 09:32 |
| *** Daisy has joined #openstack-infra | 09:33 | |
| *** fawadkhaliq has quit IRC | 09:33 | |
| *** fawadkhaliq has joined #openstack-infra | 09:33 | |
| *** fawadkhaliq has quit IRC | 09:33 | |
| *** asettle has joined #openstack-infra | 09:33 | |
| *** Daisy__ has joined #openstack-infra | 09:34 | |
| *** vgridnev has joined #openstack-infra | 09:34 | |
| yolanda | skraynev, i'm thinking there may be some problem with the branch... i'm seeing an error trying to use branch=None, and falling back to master | 09:34 |
| yolanda | this is the only rally job running on periodical pipeline? | 09:34 |
| skraynev | yes. | 09:35 |
| skraynev | fir Heat | 09:35 |
| *** PollyZ has quit IRC | 09:35 | |
| skraynev | *for | 09:35 |
| *** nadya_ has joined #openstack-infra | 09:35 | |
| *** Daisy_ has quit IRC | 09:36 | |
| yolanda | try adding that | 09:36 |
| yolanda | # ZUUL Doesn't set these for periodic jobs | 09:36 |
| yolanda | export ZUUL_BRANCH=${{ZUUL_BRANCH:-master}} | 09:36 |
| yolanda | export ZUUL_REF=${{ZUUL_REF:-None}} | 09:36 |
| yolanda | the thing is that this job is a template, used for check and periodic as well, right? | 09:36 |
| *** nadya has quit IRC | 09:36 | |
| AJaeger | skraynev: it's the only rally job *at all* in periodic pipeline. | 09:36 |
| *** Daisy has quit IRC | 09:37 | |
| yolanda | AJaeger, so i'm thinking that he shall detect if the job is periodic somehow, and export these vars if needed, am i right? | 09:37 |
| AJaeger | yolanda: yes - like done in devstack.yaml with the BRANCH_OVERRIDE logic. | 09:37 |
| *** amotoki has joined #openstack-infra | 09:38 | |
| *** Daisy__ has quit IRC | 09:38 | |
| AJaeger | this implies creating a specific periodic job that has BRANCH_OVERRIDE=master - and the gate job uses "default" | 09:39 |
| skraynev | AJaeger: yes. it's only one for periodic | 09:39 |
| AJaeger | skraynev: that's why you're the first to hit this. The rally jobs are not setup to handle periodic jobs *at all*. | 09:40 |
| skraynev | AJaeger: heh. naked true ;) | 09:40 |
| kashyap | Those using Gertty (I'm using python-gertty-1.3.0-2.fc24.noarch) , when trying to sync a project, I see this "KeyError: '_sortkey'" - http://paste.openstack.org/show/508876/ | 09:40 |
| kashyap | Anyone else notice too? | 09:41 |
| *** sambetts|afk is now known as sambetts | 09:41 | |
| kashyap | Maybe there's a fix for it in Git master, I should try it I guess... | 09:41 |
| AJaeger | kashyap: what is the value of change-list-options' sort-by option for you? Might be that that is wrong... | 09:42 |
| skraynev | AJaeger: ok. so I will try fix suggested by yolanda for Rally job template. | 09:42 |
| AJaeger | kashyap: in your ~/.gertty.yaml | 09:42 |
| kashyap | AJaeger: Yeah, looking | 09:42 |
| *** fawadkhaliq has joined #openstack-infra | 09:42 | |
| AJaeger | skraynev: not the fix by yolanda - do it like done in devstack.yaml. For devstack jobs we do this idifferently | 09:42 |
| *** amotoki has quit IRC | 09:42 | |
| kashyap | AJaeger: I don't even have that option | 09:42 |
| kashyap | AJaeger: That's my sanitized config - http://paste.openstack.org/show/508878/ | 09:43 |
| skraynev | AJaeger: aha. got it. thank you | 09:43 |
| *** kzaitsev_mb has quit IRC | 09:43 | |
| AJaeger | kashyap: can't help further - you might want to ask jeblair later today (US west coast morning) | 09:44 |
| *** PollyZ has joined #openstack-infra | 09:44 | |
| kashyap | AJaeger: No worries, I'll try some heuristics meanwhile. | 09:44 |
| *** sorantis has joined #openstack-infra | 09:44 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/infra-ansible: Wait for server to be ssh reachable before bootstrap/configuration https://review.openstack.org/326957 | 09:46 |
| *** amitgandhinz has joined #openstack-infra | 09:47 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/infra-ansible: Set resolv.conf on nodes before bootstrap https://review.openstack.org/326959 | 09:48 |
| *** PollyZ has quit IRC | 09:49 | |
| *** sdake has quit IRC | 09:49 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/infra-ansible: Add infra_config.yml to .gitignore https://review.openstack.org/326962 | 09:51 |
| *** amitgandhinz has quit IRC | 09:52 | |
| *** mhickey has joined #openstack-infra | 09:52 | |
| *** markvoelker has joined #openstack-infra | 09:53 | |
| *** amotoki has joined #openstack-infra | 09:56 | |
| *** PollyZ has joined #openstack-infra | 09:57 | |
| *** lvdongbing has quit IRC | 09:59 | |
| *** PollyZ has quit IRC | 10:02 | |
| *** rossella_s has quit IRC | 10:02 | |
| *** nadya_ has quit IRC | 10:05 | |
| *** zhurong has quit IRC | 10:08 | |
| *** dtantsur|bbl is now known as dtantsur | 10:11 | |
| *** tzn has quit IRC | 10:11 | |
| *** PollyZ has joined #openstack-infra | 10:11 | |
| *** sorantis has quit IRC | 10:15 | |
| *** PollyZ has quit IRC | 10:16 | |
| *** jogo has quit IRC | 10:17 | |
| *** matrohon has quit IRC | 10:21 | |
| *** PollyZ has joined #openstack-infra | 10:25 | |
| *** salv-orl_ has joined #openstack-infra | 10:25 | |
| odyssey4me | yolanda If you have a moment, could you please add me to each of these groups? https://review.openstack.org/#/q/project:openstack-infra/project-config+branch:master+topic:osa-additional-core-teams | 10:26 |
| clarkb | kashyap: I think most people using gertty run it out of a virtualenv | 10:28 |
| *** salv-orlando has quit IRC | 10:28 | |
| *** matbu is now known as matbu|lch | 10:29 | |
| *** PollyZ has quit IRC | 10:29 | |
| kashyap | clarkb: I see. I use the packaged version normally, but okay, I can try the vitualenv route. | 10:30 |
| openstackgerrit | Luigi Toscano proposed openstack-infra/project-config: Run tempest tests from sahara-tests for sahara jobs https://review.openstack.org/326980 | 10:30 |
| *** lvdongbing has joined #openstack-infra | 10:30 | |
| yolanda | odyssey4me, i'll do | 10:30 |
| *** lvdongbing has quit IRC | 10:32 | |
| *** salv-orl_ has quit IRC | 10:34 | |
| *** salv-orlando has joined #openstack-infra | 10:34 | |
| odyssey4me | yolanda thanks, I appreciate it | 10:34 |
| *** lifeless has joined #openstack-infra | 10:35 | |
| kashyap | clarkb: Do you have the 'change-list-options' section? | 10:36 |
| yolanda | odyssey4me, done. Please verify if all groups are correct | 10:36 |
| *** bhavik has quit IRC | 10:36 | |
| odyssey4me | yolanda thanks, I'll ping you for any that were missed | 10:36 |
| clarkb | mtreinish: for the translation job statuses in openstack health dashboard that isn't using subunit is it? Are you querying graphite instead? | 10:37 |
| *** weshay has joined #openstack-infra | 10:37 | |
| clarkb | kashyap: I don't currently use gertty I probably should but my device use is much more fluid than it used to be making it hard to have a single local gertty | 10:37 |
| *** lvdongbing has joined #openstack-infra | 10:38 | |
| *** jogo has joined #openstack-infra | 10:38 | |
| kashyap | clarkb: No worries. I'm trying it out in a virtualenv | 10:38 |
| *** PollyZ has joined #openstack-infra | 10:38 | |
| *** vincentll has quit IRC | 10:38 | |
| clarkb | my old config doesn't have that option set | 10:39 |
| clarkb | but I also typically just copy paste the provided example and add in the auth info | 10:39 |
| *** tkelsey has quit IRC | 10:39 | |
| openstackgerrit | Merged openstack-infra/infra-ansible: Add infra_config.yml to .gitignore https://review.openstack.org/326962 | 10:39 |
| *** nadya has joined #openstack-infra | 10:41 | |
| *** rhallisey has joined #openstack-infra | 10:41 | |
| *** nadya_ has joined #openstack-infra | 10:42 | |
| *** sdague has joined #openstack-infra | 10:42 | |
| *** nadya has quit IRC | 10:42 | |
| *** andreas_s has quit IRC | 10:42 | |
| *** javeriak has joined #openstack-infra | 10:43 | |
| openstackgerrit | Stephen Finucane proposed openstack-infra/project-config: Disable 'docs' gate test for 'intel-nfv-ci-tests' https://review.openstack.org/326989 | 10:43 |
| kashyap | clarkb: Okay, syncing well in a virtualenv. I'll just stick to it. | 10:43 |
| *** PollyZ has quit IRC | 10:43 | |
| *** yamamoto_ has quit IRC | 10:43 | |
| *** lifeless has quit IRC | 10:47 | |
| *** lifeless_ has joined #openstack-infra | 10:47 | |
| AJaeger | clarkb: the translation job status use subunit | 10:48 |
| AJaeger | clarkb: see https://review.openstack.org/311459 | 10:48 |
| *** amitgandhinz has joined #openstack-infra | 10:48 | |
| *** kzaitsev_mb has joined #openstack-infra | 10:48 | |
| clarkb | hrm, not sure if I like that. Will have to think about whether or not it makes sense to bake that into individual jobs that way | 10:50 |
| *** tesseract has quit IRC | 10:50 | |
| *** nwkarsten has joined #openstack-infra | 10:50 | |
| *** nadya_ has quit IRC | 10:50 | |
| clarkb | looks like we install os-testr but we never use it? | 10:50 |
| AJaeger | clarkb: we use it - in the finish call... | 10:51 |
| *** nadya has joined #openstack-infra | 10:51 | |
| odyssey4me | yolanda all confirmed done, thank you very much | 10:51 |
| *** dizquierdo has quit IRC | 10:51 | |
| clarkb | AJaeger: oh is generate subunit from os-testr not from subunit? | 10:51 |
| *** tesseract has joined #openstack-infra | 10:51 | |
| AJaeger | clarkb: yes, it is AFAIU | 10:51 |
| clarkb | AJaeger: I see | 10:52 |
| clarkb | (subunit also comes with a bunch of utilities like that) | 10:52 |
| *** PollyZ has joined #openstack-infra | 10:52 | |
| AJaeger | clarkb: mtreinish and myself dsicussed this in Austin, and I talked with jeblair about the problem I have as well: With shutdown of jenkins external access, there's no easy way to figure out whether translation jobs succeed or not. So, this was the best idea moving forward. | 10:53 |
| *** amitgandhinz has quit IRC | 10:53 | |
| AJaeger | clarkb: If you have a better idea to figure out which post and periodic jobs fail/succeed, please tell me. | 10:53 |
| *** lvdongbing has quit IRC | 10:53 | |
| clarkb | yup Inagree we need a way to expose the info,I mostly dont like installi gos-testr when we never run testr | 10:53 |
| *** Daisy has joined #openstack-infra | 10:53 | |
| clarkb | but thats my bias against the tool.in general probably | 10:54 |
| clarkb | (wish it was more upstream less opemstack special) | 10:54 |
| *** lvdongbing has joined #openstack-infra | 10:54 | |
| *** nwkarsten has quit IRC | 10:54 | |
| AJaeger | clarkb: if installing subunit will work as well, I'm happy to switch... | 10:54 |
| clarkb | AJaeger: I am not sure if it will, depends on if gemerate script comes from aubunut | 10:55 |
| clarkb | I am being dragged to lunch though | 10:55 |
| AJaeger | os-testr setup.cfg has: generate-subunit = os_testr.generate_subunit:main | 10:55 |
| AJaeger | clarkb: enjoy lunch! | 10:55 |
| *** PollyZ has quit IRC | 10:56 | |
| *** nadya has quit IRC | 10:57 | |
| *** rossella_s has joined #openstack-infra | 10:58 | |
| *** Daisy has quit IRC | 10:58 | |
| openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate: DNM: experimenting with local.conf https://review.openstack.org/326585 | 10:59 |
| *** rhallisey has quit IRC | 10:59 | |
| *** d0ugal has quit IRC | 10:59 | |
| *** andreww has quit IRC | 10:59 | |
| *** yolanda has quit IRC | 10:59 | |
| *** d0ugal has joined #openstack-infra | 11:00 | |
| *** nadya has joined #openstack-infra | 11:00 | |
| *** ihrachys has joined #openstack-infra | 11:02 | |
| *** banix has joined #openstack-infra | 11:04 | |
| *** nadya has quit IRC | 11:04 | |
| *** afazekas is now known as __afazekas | 11:05 | |
| *** weshay has quit IRC | 11:05 | |
| *** yujunz has joined #openstack-infra | 11:05 | |
| *** PollyZ has joined #openstack-infra | 11:05 | |
| *** yujunz has quit IRC | 11:08 | |
| *** d0ugal has quit IRC | 11:09 | |
| *** doug-fish has joined #openstack-infra | 11:09 | |
| *** PollyZ has quit IRC | 11:10 | |
| *** lvdongbing has quit IRC | 11:10 | |
| odyssey4me | It looks like the zuul events/results are simply climbing? | 11:11 |
| *** e0ne_ has joined #openstack-infra | 11:11 | |
| *** weshay has joined #openstack-infra | 11:11 | |
| *** matbu|lch is now known as matbu | 11:12 | |
| *** fawadkhaliq has quit IRC | 11:12 | |
| *** banix has quit IRC | 11:13 | |
| *** e0ne has quit IRC | 11:13 | |
| *** dizquierdo has joined #openstack-infra | 11:17 | |
| AJaeger | odyssey4me: there are times when jobs merge and a possible reconfiguration happens where this happens, might take a couple of minutes to go through all events | 11:18 |
| AJaeger | odyssey4me: I've seen it go up to 10000 already ;( | 11:19 |
| *** PollyZ has joined #openstack-infra | 11:19 | |
| odyssey4me | AJaeger I've been watching a particular job for over 10 mins now and it hasn't changed. Usually when it takes this long something's gone wrong. | 11:19 |
| *** fawadkhaliq has joined #openstack-infra | 11:19 | |
| odyssey4me | AJaeger https://review.openstack.org/325507 completed its job quite some time ago. | 11:20 |
| AJaeger | odyssey4me: it might take 30mins for this ;( | 11:20 |
| AJaeger | odyssey4me: Let's drink a coffee ;) | 11:20 |
| AJaeger | infra-root, I see a failing job in the post queue - but no log files. Links are http://logs.openstack.org/f2/f20de1013003414a35abd22a31516921a9d8d202/post/openstack-manuals-releasenotes/ff790b9/ and http://logs.openstack.org/39/392bdecc981ce1f07afb91f4927938d570692340/post/openstack-manuals-releasenotes/7e4bb6d/ . Any ideas? | 11:20 |
| odyssey4me | AJaeger yeah, another 3 of those for release notes: http://logs.openstack.org/3b/3bc293f77c84cad297f8870c1898c685fae5c44f/post/openstack-ansible-security-releasenotes/de645d8/ / http://logs.openstack.org/4a/4a5b482ebac51a423a2e317ff876b965c3b16d81/post/openstack-ansible-os_tempest-releasenotes/b3ab7ed/ / http://logs.openstack.org/9c/9c15bfee5a187668fe536795b5a8a5333235be01/post/openstack-ansible-os_gnocchi-releasenotes | 11:21 |
| odyssey4me | /0b808d8/ | 11:21 |
| * odyssey4me makes a pot of coffee | 11:22 | |
| *** javeriak has quit IRC | 11:23 | |
| *** javeriak has joined #openstack-infra | 11:23 | |
| *** LaT0rtue has joined #openstack-infra | 11:24 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: translations: Handle early exit https://review.openstack.org/327021 | 11:24 |
| *** sorantis has joined #openstack-infra | 11:24 | |
| *** PollyZ has quit IRC | 11:24 | |
| *** fawadkhaliq has quit IRC | 11:25 | |
| AJaeger | odyssey4me: the coffee helped, queue is empty again... | 11:26 |
| odyssey4me | AJaeger haha, ok - I was just a little concerned | 11:26 |
| AJaeger | ;) | 11:27 |
| AJaeger | odyssey4me: my rule of thumb: as long as the numbers change, it's fine ;) But yeah, if those change for 30mins, it's scary... | 11:27 |
| *** moravec has quit IRC | 11:27 | |
| *** nadya has joined #openstack-infra | 11:28 | |
| *** vincentll has joined #openstack-infra | 11:29 | |
| *** bhavik has joined #openstack-infra | 11:31 | |
| *** kzaitsev_mb has quit IRC | 11:32 | |
| *** d0ugal has joined #openstack-infra | 11:32 | |
| *** yamamoto has joined #openstack-infra | 11:33 | |
| * krotscheck patiently waits for the next dib image upload to happen | 11:33 | |
| *** zerda2 has joined #openstack-infra | 11:34 | |
| *** yamamoto_ has joined #openstack-infra | 11:34 | |
| *** ccamacho is now known as ccamacho|lunch | 11:34 | |
| *** lucasagomes is now known as lucas-afk | 11:35 | |
| *** ldnunes has joined #openstack-infra | 11:37 | |
| *** fawadkhaliq has joined #openstack-infra | 11:38 | |
| *** yamamoto has quit IRC | 11:38 | |
| *** baoli has joined #openstack-infra | 11:40 | |
| *** fawadkhaliq has quit IRC | 11:40 | |
| *** fawadkhaliq has joined #openstack-infra | 11:40 | |
| *** rhallisey has joined #openstack-infra | 11:40 | |
| *** doug-fish has quit IRC | 11:41 | |
| *** baoli_ has joined #openstack-infra | 11:41 | |
| *** tpsilva has joined #openstack-infra | 11:41 | |
| zerda2 | Hello. I'm trying to use openstack-infra/logstash-filters for merging multiline traces, but stacktraces from errors are merged with previous unrelated log message. In filter there is a note saying "stack traces always contain process ID", but other log lines also have it, which is why I guess this happens. | 11:42 |
| *** Goneri has joined #openstack-infra | 11:42 | |
| zerda2 | Question is - what can be done about it? | 11:42 |
| *** moravec has joined #openstack-infra | 11:42 | |
| *** banix has joined #openstack-infra | 11:43 | |
| *** gordc has joined #openstack-infra | 11:43 | |
| *** fawadk has joined #openstack-infra | 11:43 | |
| *** javeriak_ has joined #openstack-infra | 11:43 | |
| *** sdake has joined #openstack-infra | 11:44 | |
| *** fawadkhaliq has quit IRC | 11:44 | |
| krotscheck | There's a dib image build log error: fatal: remote error: access denied or repository not exported: /openstack-dev/os-http.git | 11:44 |
| krotscheck | http://nodepool.openstack.org/dib.debian-jessie.log.2016-06-07_10 | 11:44 |
| *** fawadkhaliq has joined #openstack-infra | 11:44 | |
| *** javeria__ has joined #openstack-infra | 11:44 | |
| *** nadya_ has joined #openstack-infra | 11:44 | |
| *** kzaitsev_mb has joined #openstack-infra | 11:44 | |
| *** javeriak has quit IRC | 11:44 | |
| *** baoli has quit IRC | 11:45 | |
| krotscheck | Same in fedora: http://nodepool.openstack.org/dib.fedora-23.log.2016-06-07_10 | 11:45 |
| krotscheck | Repo's still there. | 11:45 |
| *** nadya has quit IRC | 11:46 | |
| *** PollyZ has joined #openstack-infra | 11:46 | |
| *** banix has quit IRC | 11:46 | |
| *** javeriak_ has quit IRC | 11:48 | |
| *** fawadk has quit IRC | 11:48 | |
| *** amitgandhinz has joined #openstack-infra | 11:49 | |
| *** yolanda has joined #openstack-infra | 11:49 | |
| *** __afazekas is now known as afazekas | 11:49 | |
| *** gildub has joined #openstack-infra | 11:51 | |
| *** PollyZ has quit IRC | 11:51 | |
| *** tzn has joined #openstack-infra | 11:53 | |
| *** faizy has quit IRC | 11:53 | |
| *** amitgandhinz has quit IRC | 11:53 | |
| *** xarses has joined #openstack-infra | 11:55 | |
| *** markvoelker has quit IRC | 11:56 | |
| *** LaT0rtue has quit IRC | 11:57 | |
| *** ifarkas has quit IRC | 11:58 | |
| *** amrith is now known as _amrith_ | 11:58 | |
| *** salv-orlando has quit IRC | 11:59 | |
| *** zeih has quit IRC | 11:59 | |
| clarkb | zerda2: can you paste example inpuy and output? | 12:00 |
| *** moravec has quit IRC | 12:00 | |
| clarkb | but my guess is some of our rules are stale and need updates | 12:00 |
| *** salv-orlando has joined #openstack-infra | 12:00 | |
| *** PollyZ has joined #openstack-infra | 12:00 | |
| *** aysyd has joined #openstack-infra | 12:00 | |
| *** kzaitsev_mb has quit IRC | 12:01 | |
| *** HeOS has quit IRC | 12:01 | |
| zerda2 | clarkb, I found out that in devstack there are some redefinitions of logging_* options, most notably context and exception, which make these rules work | 12:02 |
| zerda2 | So I guess I'll go the same way | 12:02 |
| *** HeOS has joined #openstack-infra | 12:02 | |
| *** kzaitsev_mb has joined #openstack-infra | 12:03 | |
| *** fawadkhaliq has quit IRC | 12:03 | |
| *** PollyZ has quit IRC | 12:05 | |
| *** vincentll has quit IRC | 12:05 | |
| *** ifarkas has joined #openstack-infra | 12:06 | |
| *** ccamacho|lunch is now known as ccamacho | 12:06 | |
| zerda2 | But still that rules won't work reliably on "vanilla" installations, and maybe there should be a note about that in logstash-filters | 12:06 |
| *** ayoung has quit IRC | 12:07 | |
| *** ihrachys has quit IRC | 12:10 | |
| *** zhurong has joined #openstack-infra | 12:10 | |
| zerda2 | clarkb, should I file a bug/rfe for this, or just a review? | 12:10 |
| *** nadya_ has quit IRC | 12:10 | |
| *** ihrachys has joined #openstack-infra | 12:10 | |
| *** vincentll has joined #openstack-infra | 12:10 | |
| *** sfinucan has joined #openstack-infra | 12:11 | |
| clarkb | zerda2: gotcha I think the best thing here would be to make devstack stop doing that then update the rules to work | 12:12 |
| clarkb | basically devstack default logging should reflect real world defaults | 12:12 |
| *** markvoelker has joined #openstack-infra | 12:12 | |
| *** zqfan has quit IRC | 12:13 | |
| *** sdake has quit IRC | 12:13 | |
| zerda2 | clarkb, well, I tried to find a general solution, but since oslo.log changed severity of stacktraces there is no simple way to distinguish stacktrace from other error | 12:13 |
| *** moravec has joined #openstack-infra | 12:14 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Ensure extracted log file are readable https://review.openstack.org/327053 | 12:14 |
| *** nadya has joined #openstack-infra | 12:14 | |
| *** salv-orl_ has joined #openstack-infra | 12:15 | |
| *** dprince has joined #openstack-infra | 12:15 | |
| *** dizquierdo has quit IRC | 12:16 | |
| *** nadya_ has joined #openstack-infra | 12:17 | |
| *** zhurong has quit IRC | 12:19 | |
| *** nadya has quit IRC | 12:19 | |
| *** nadya_ has quit IRC | 12:19 | |
| *** salv-orlando has quit IRC | 12:19 | |
| *** zerda2 has quit IRC | 12:20 | |
| *** sshnaidm|afk has joined #openstack-infra | 12:21 | |
| *** zhurong has joined #openstack-infra | 12:21 | |
| *** fawadkhaliq has joined #openstack-infra | 12:22 | |
| *** jcoufal has joined #openstack-infra | 12:23 | |
| *** mriedem has joined #openstack-infra | 12:24 | |
| *** banix has joined #openstack-infra | 12:24 | |
| *** yamamoto_ has quit IRC | 12:26 | |
| *** fawadkhaliq has quit IRC | 12:26 | |
| sfinucan | AJaeger: Quick question? | 12:27 |
| *** zeih has joined #openstack-infra | 12:27 | |
| *** PollyZ has joined #openstack-infra | 12:27 | |
| openstackgerrit | Stephen Finucane proposed openstack-infra/project-config: Disable 'docs' gate test for 'intel-nfv-ci-tests' https://review.openstack.org/326989 | 12:28 |
| *** yamamoto has joined #openstack-infra | 12:28 | |
| *** yamamoto has quit IRC | 12:30 | |
| *** pradk has quit IRC | 12:30 | |
| *** nadya has joined #openstack-infra | 12:30 | |
| *** trown|outtypewww is now known as trown | 12:31 | |
| *** PollyZ has quit IRC | 12:32 | |
| *** zeih has quit IRC | 12:32 | |
| *** fawadkhaliq has joined #openstack-infra | 12:33 | |
| openstackgerrit | Stephen Finucane proposed openstack-infra/project-config: Simplify gate tests for 'intel-nfv-ci-tests' https://review.openstack.org/326989 | 12:34 |
| openstackgerrit | Bob Ball proposed openstack-infra/system-config: Update third party recheck syntax recommendations https://review.openstack.org/327065 | 12:34 |
| *** signed8bit_Zzz is now known as signed8b_ | 12:36 | |
| *** rossella_s has quit IRC | 12:36 | |
| sfinucan | Hey. Could someone briefly describe the difference between 'jenkins/jobs/projects.yaml' and 'zuul/layout.yaml' in the 'project-config' project, please? They appear to be quite similar? | 12:37 |
| clarkb | sfinucan: jenkins jobs define what jobs to configure in jenkins. zuul layout declares what jobs zuul should run | 12:38 |
| clarkb | you can run jjb (jenkins/jobs) without zuul and you can run zuul without jjb | 12:38 |
| clarkb | we happen to use them togetger but that is an implementation choice and not a requiremwnt | 12:39 |
| *** javeria__ has quit IRC | 12:39 | |
| *** rlandy has joined #openstack-infra | 12:39 | |
| *** PollyZ has joined #openstack-infra | 12:41 | |
| *** gildub has quit IRC | 12:41 | |
| *** kgiusti has joined #openstack-infra | 12:41 | |
| *** eharney has joined #openstack-infra | 12:42 | |
| *** rossella_s has joined #openstack-infra | 12:42 | |
| *** lucas-afk is now known as lucasagomes | 12:44 | |
| *** thorst has joined #openstack-infra | 12:45 | |
| sfinucan | clarkb: OK, so to me that means the jobs that I see on the Gerrit "Checks" table should be configured in the 'jenkins/jobs/project.yaml' file, but AJaeger's comments suggest otherwise https://review.openstack.org/#/c/326989/1/jenkins/jobs/projects.yaml | 12:45 |
| sfinucan | and when I look at 'zuul/layout.yaml', there is indeed an entry for the same project that states pretty much the same jobs | 12:45 |
| *** PollyZ has quit IRC | 12:45 | |
| clarkb | sfinucan: the names do map between them because zuul triggers the jobs in jenkins | 12:46 |
| clarkb | the distinction is each configures a different service and you need not use both services together | 12:46 |
| AJaeger | sfinucan: jenkins is a library of jobs you can run, zuul "calls" them. | 12:46 |
| clarkb | we happen to use tgem together | 12:47 |
| AJaeger | sfinucan: in jenkins you configure what to run exactly, in zuul you configure when - and whether - to run them | 12:47 |
| *** kzaitsev_mb has quit IRC | 12:48 | |
| *** esikachev has quit IRC | 12:48 | |
| *** thorst has quit IRC | 12:49 | |
| *** amitgandhinz has joined #openstack-infra | 12:49 | |
| *** xarses has quit IRC | 12:50 | |
| sfinucan | AJaeger: clarkb: But, assuming I'm reading it correctly, in the zuul config we say run a merge-check job and run the standard python-jobs. For the Jenkins config, we say run just the python-jobs | 12:50 |
| sfinucan | So would it be fair to say there's no real reuse here, other than a reuse of that python-jobs configuration. zuul isn't using any jenkins config in this case? | 12:51 |
| clarkb | sfinucan: no in the jenkins config we dont say run any jobs | 12:51 |
| clarkb | we just say make sure that those jobs exist and that they are configured this way | 12:51 |
| clarkb | there is no reuse. jenkins is an executor and zuul is a schedulor | 12:52 |
| sfinucan | clarkb: Ahh | 12:52 |
| AJaeger | sfinucan: tread jenkins config as a library. You enable all python-jobs so that you can run them if needed in jenkins. And then in zuul you use one of the enabled ones | 12:52 |
| AJaeger | sfinucan: or as clarkb said ;) | 12:52 |
| sfinucan | OK, I get you now. That makes much more sense :) | 12:53 |
| sfinucan | Is that documented anywhere? If not, is it something I could quickly document? | 12:53 |
| sfinucan | (for my own future reference, as much as anything :)) | 12:53 |
| clarkb | each of the tools should make it clear | 12:53 |
| clarkb | jjb configures jenkins. zuul is a build scheduler | 12:53 |
| *** amitgandhinz has quit IRC | 12:54 | |
| *** PollyZ has joined #openstack-infra | 12:54 | |
| *** matt-borland has joined #openstack-infra | 12:54 | |
| *** julim has joined #openstack-infra | 12:55 | |
| AJaeger | http://docs.openstack.org/infra/manual/creators.html#add-basic-jenkins-jobs ? | 12:55 |
| AJaeger | and http://docs.openstack.org/infra/manual/creators.html#configure-zuul-to-run-jobs | 12:55 |
| AJaeger | sfinucan: and http://docs.openstack.org/infra/manual/drivers.html#running-jobs-with-zuul | 12:56 |
| *** rodrigods has quit IRC | 12:56 | |
| *** amoralej is now known as amoralej|lunch | 12:56 | |
| *** rodrigods has joined #openstack-infra | 12:56 | |
| sfinucan | AJaeger: clarkb: That's perfect. Thank you very much :) | 12:57 |
| *** tlian has joined #openstack-infra | 12:58 | |
| AJaeger | fungi, pabelanger : Is https://review.openstack.org/#/c/325658 still the right way to do this - together with a change for bindep-fallback? Could you both double check this, please? | 12:58 |
| *** PollyZ has quit IRC | 12:59 | |
| *** _shaps_ has joined #openstack-infra | 12:59 | |
| *** thorst has joined #openstack-infra | 13:00 | |
| *** edmondsw has joined #openstack-infra | 13:00 | |
| *** claudiub has joined #openstack-infra | 13:00 | |
| *** nadya has quit IRC | 13:01 | |
| *** oanson has quit IRC | 13:01 | |
| *** gouthamr has joined #openstack-infra | 13:02 | |
| *** zeih has joined #openstack-infra | 13:03 | |
| *** links has quit IRC | 13:03 | |
| openstackgerrit | Stephen Finucane proposed openstack-infra/project-config: Simplify gate tests for 'intel-nfv-ci-tests' https://review.openstack.org/326989 | 13:03 |
| *** fawadkhaliq has quit IRC | 13:04 | |
| *** amitgandhinz has joined #openstack-infra | 13:04 | |
| *** amitgandhinz has quit IRC | 13:05 | |
| *** amitgandhinz has joined #openstack-infra | 13:05 | |
| openstackgerrit | Stephen Finucane proposed openstack-infra/project-config: Simplify gate tests for 'intel-nfv-ci-tests' https://review.openstack.org/326989 | 13:06 |
| *** pblaho has joined #openstack-infra | 13:07 | |
| *** esikachev has joined #openstack-infra | 13:08 | |
| *** pblaho has quit IRC | 13:08 | |
| *** pblaho has joined #openstack-infra | 13:09 | |
| *** otherwiseguy has joined #openstack-infra | 13:09 | |
| pabelanger | AJaeger: agreed, I'd like to see that move to bindep as I plan to delete that code as soon as we migrate to zuul-launcher | 13:11 |
| *** mdrabe has joined #openstack-infra | 13:13 | |
| *** kzaitsev_mb has joined #openstack-infra | 13:13 | |
| openstackgerrit | Merged openstack-infra/system-config: Add zuul-worker public key to nodepool.o.o https://review.openstack.org/323539 | 13:17 |
| *** samuelBartel has joined #openstack-infra | 13:18 | |
| *** links has joined #openstack-infra | 13:18 | |
| *** signed8b_ is now known as signed8bit_Zzz | 13:19 | |
| *** esberglu has joined #openstack-infra | 13:20 | |
| *** watanabe_isao has joined #openstack-infra | 13:20 | |
| *** kushal has joined #openstack-infra | 13:20 | |
| SotK | jeblair: I'm looking at https://storyboard.openstack.org/#!/story/2000620 and wondering if you have time to expand on which events you think should be hidden by default, and which you find moderately interesting? | 13:21 |
| *** zz_dimtruck is now known as dimtruck | 13:22 | |
| *** vincentll has quit IRC | 13:23 | |
| *** vincentll has joined #openstack-infra | 13:23 | |
| *** jamesmcarthur has joined #openstack-infra | 13:24 | |
| mnaser | alright, this might be out of scope but i figure i'd poke minds of people who use puppet often. i'm getting "Duplicate declaration: Package[python-memcache] is already declared" at a line where ensure_packages() is used... | 13:24 |
| clarkb | pleia2: pabelanger: jamesmcarthur noticed that we still have spam things on our wiki and we are getting flagged by the Google appearnetly. eg https://wiki.openstack.org/w/images/6/61/Kindle_Fire_models_@@1844-844-2433_Kindle_Fire_tech_support_number_Cisco_customer_support_phone_number.pdf | 13:24 |
| mnaser | any thoughts? | 13:24 |
| jamesmcarthur | clarkb: pleia2: pabelenger: Another set of pages that you can find via search on the wiki: https://wiki.openstack.org/w/index.php?search=Kindle | 13:25 |
| clarkb | pleia2: pabelanger: also if you search for "Kindle" ~9 pages come up. Any chance you can work your wiki cleanup magic again and make sure that we aren't currently falling prey to new spammers? | 13:25 |
| crinkle | mnaser: ensure_packages() doesn't work if the other declaration doesn't use ensure_packages() and happened to be interpreted first | 13:25 |
| clarkb | mnaser: ya, puppet enforces unique namevars so if you have two packages with that same name (even if resources rae defined identically) then it fails | 13:26 |
| crinkle | mnaser: and #puppet is a good place for questions like that | 13:26 |
| *** ayoung has joined #openstack-infra | 13:26 | |
| jamesmcarthur | Looks like the problem isn't limited to that search. Also try https://wiki.openstack.org/w/index.php?search=Customer%20Service and https://wiki.openstack.org/w/index.php?search=Kodak | 13:26 |
| mnaser | crinkle: i see. i was just poking brains here, didnt get much at #puppet .. the weird thing is i cannot for my life find anything that is defining it directly | 13:26 |
| mnaser | grep'd all through modules, but ill try to see if it's sneaking its way somewhere else.. | 13:26 |
| crinkle | mnaser: it could also be that it's inside a define that is getting called twice | 13:27 |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Don't skip errors in postci https://review.openstack.org/326759 | 13:27 |
| *** _ari_ has joined #openstack-infra | 13:27 | |
| jamesmcarthur | clarkb: pleia2: pabelenger: looks like it goes pretty deep. https://wiki.openstack.org/w/index.php?search=tech%20support, https://wiki.openstack.org/w/index.php?search=Phone%20Number | 13:28 |
| mnaser | crinkle: that's a a far more interesting/probably idea, i'll check. | 13:28 |
| *** njohnston|sleep is now known as njohnston | 13:28 | |
| clarkb | jamesmcarthur: pleia2 pabelanger "customer" is another good one | 13:28 |
| jamesmcarthur | clarkb: It does appear that most of them are dated February, at first glance. | 13:29 |
| pabelanger | jamesmcarthur: clarkb: pleia2: I can try looking later today. | 13:29 |
| clarkb | pleia2: pabelanger but basically I am super distracted and traveling and conferencing and not in a great position to learn me a wiki. If you have any bandwidth at all it would be great to get this cleaned up | 13:29 |
| jamesmcarthur | pabelanger: clarkb: I'd appreciate anything you can do. Big problem is that Google has started flagging www.openstack.org as part of wiki.openstack.org | 13:30 |
| jamesmcarthur | pabelanger: clarkb: So we're losing ranking for our main site as well, which has pretty deep implications. | 13:30 |
| pabelanger | jamesmcarthur: clarkb: Personally, I'd be find updating robots.txt on wiki.o.o to Disallow: / | 13:30 |
| jamesmcarthur | pabelanger: but a lot of people still rely on the wiki. It gets 30% of our traffic. | 13:31 |
| clarkb | jamesmcarthur: they could use the wiki search | 13:31 |
| clarkb | but the jump via google wouldn't be there | 13:31 |
| pabelanger | Right, and if I understand all our important data should be out of the wiki moving forward? | 13:32 |
| *** samuelBartel has quit IRC | 13:32 | |
| clarkb | pabelanger: I think that ttx is still working through how that will work | 13:32 |
| clarkb | in nay case we should clear this out, looks like ~800 pages at least | 13:32 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: [WIP] Add mulitnode CI job support https://review.openstack.org/324777 | 13:32 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci: Remove using a swap partition https://review.openstack.org/327094 | 13:32 |
| pabelanger | I have to double check robots.txt, but we could white list the openstack namespace in the wiki | 13:33 |
| *** fifieldt has joined #openstack-infra | 13:33 | |
| pabelanger | and only focus our efforts there for the spam | 13:33 |
| fifieldt | pabelanger: $wgUseRCPatrol = true; | 13:34 |
| clarkb | pabelanger: the issue is any content on the wiki that google finds can be flagged | 13:34 |
| fifieldt | if that's in our mediawiki config file (not sure if it is in 1.25) | 13:34 |
| fifieldt | might be worth seeting | 13:34 |
| clarkb | pabelanger: so if we clean up openstack/ we would still be in trouble | 13:34 |
| fifieldt | allows us to do things with https://meta.wikimedia.org/wiki/Help:Patrolled_edit | 13:34 |
| *** dimtruck is now known as zz_dimtruck | 13:34 | |
| fifieldt | so we can filter out known good edits and people from various things when fighting spamz | 13:35 |
| *** PollyZ has joined #openstack-infra | 13:35 | |
| *** zz_dimtruck is now known as dimtruck | 13:36 | |
| ttx | pabelanger: re: wiki http://lists.openstack.org/pipermail/openstack-dev/2016-June/096481.html | 13:36 |
| fifieldt | (option is benign enough its enabled by default from 1.4) | 13:36 |
| frickler | is nodepool having some issues or is there just high load currently? | 13:37 |
| krotscheck | Another fatal error in dib, this time precise: fatal: remote error: access denied or repository not exported: /openstack-dev/os-http.git | 13:39 |
| krotscheck | frickler: The queue's pretty full. | 13:39 |
| pabelanger | fifieldt: We can add patrolled edits, if everybody is fine with it. The next step would be to figure out who is going to review them | 13:40 |
| clarkb | frickler: it looks like we are using our available capacity, you can see the nodepool node graph on the zuul status page (http://status.openstack.org/zuul) | 13:40 |
| *** _amrith_ is now known as amrith | 13:40 | |
| krotscheck | clarkb, pabelanger: dib builds appear broken. | 13:40 |
| clarkb | krotscheck: they ran when that repo was not available | 13:40 |
| clarkb | krotscheck: its a thing | 13:40 |
| fifieldt | hmm, looks like it should be on by default in our version, so we've explicitly disabled it :D | 13:40 |
| *** PollyZ has quit IRC | 13:41 | |
| krotscheck | clarkb: Got it. So, wait another 24 hours? | 13:41 |
| pabelanger | fifieldt: but, with that said, if we are doing patrolled edits moving forward, I'd rather see use review.o.o to as our review tool for the wiki edits, but then we are moving away from a wiki into sphinx / rst docs | 13:41 |
| *** pradk has joined #openstack-infra | 13:41 | |
| clarkb | krotscheck: ya | 13:41 |
| krotscheck | clarkb: Okies | 13:41 |
| fifieldt | pabelanger: my understanging is the review of wiki content concluded that was a nonoption | 13:41 |
| clarkb | fifieldt: at least for some of the use cases yes. ttx is a better source than me though | 13:42 |
| fifieldt | regardless, we do have to fix the spam problem asap :) | 13:42 |
| clarkb | I don't think every change to the wiki needs to be reviewed | 13:42 |
| fifieldt | seems like patrolling is a tool that we can use | 13:42 |
| clarkb | we just need better ways to nuke spam | 13:42 |
| fifieldt | exactly, it's not reviewing every change | 13:42 |
| pabelanger | krotscheck: clarkb: DIBs failed yesterday, they should be good for today | 13:42 |
| fifieldt | it's a way of filtering out stuff durin the spamfight | 13:43 |
| fifieldt | it feeds into many other plugins and tools, as I understand it | 13:43 |
| krotscheck | pabelanger: Alright, I'll let the horizon team know. | 13:43 |
| fifieldt | including some of the autopatrol bots you find on wikipedia | 13:43 |
| *** bdemers has joined #openstack-infra | 13:43 | |
| clarkb | krotscheck: what do people need in new images? | 13:43 |
| *** amoralej|lunch is now known as amoralej | 13:43 | |
| pabelanger | clarkb: better spam filtering needs a newer OS, which is on the list for me (basically last) | 13:43 |
| clarkb | pabelanger: ya I get that but that is only one of the options right? | 13:43 |
| krotscheck | clarkb: ulimit -Sn increase | 13:43 |
| clarkb | its not our only option | 13:43 |
| clarkb | krotscheck: you can do it in the jobs if you need ot fwiw | 13:44 |
| *** cloudtrainme has joined #openstack-infra | 13:44 | |
| *** bhavik has quit IRC | 13:44 | |
| pabelanger | clarkb: another option would be to migrate away from mediawiki to something more puppet friendly, and port our data to the new server. Something more and more I am thinking we should do if this server needs to be long lived | 13:44 |
| *** kzaitsev_mb has quit IRC | 13:44 | |
| clarkb | yes we could reconsider the wiki platform itself. We did use moin moin once upon a time | 13:45 |
| clarkb | (thought it suffered from other problems prompting the move) | 13:45 |
| pabelanger | I am a fan of moinmoin, was pretty easy to use puppet on that | 13:45 |
| *** nwkarsten has joined #openstack-infra | 13:45 | |
| *** nwkarsten has quit IRC | 13:46 | |
| *** kzaitsev_mb has joined #openstack-infra | 13:46 | |
| pabelanger | There are newer wikis that have git backends, so we could leverage that against review.o.o too | 13:46 |
| *** nwkarsten has joined #openstack-infra | 13:46 | |
| *** pblaho has quit IRC | 13:47 | |
| *** jlanoux_ has quit IRC | 13:47 | |
| pabelanger | krotscheck: I noticed an error in stackviz: http://nodepool.openstack.org/dib.fedora-23.log: 2016-06-08 10:57:04,163 INFO nodepool.image.build.fedora-23: OSError: [Errno 38] Function not implemented | 13:48 |
| *** hongbin has joined #openstack-infra | 13:48 | |
| *** asselin_ has joined #openstack-infra | 13:48 | |
| *** jamesmcarthur has quit IRC | 13:50 | |
| *** PollyZ has joined #openstack-infra | 13:50 | |
| *** jamesmcarthur has joined #openstack-infra | 13:50 | |
| *** jamesmcarthur has quit IRC | 13:52 | |
| *** watanabe_isao has quit IRC | 13:54 | |
| *** burgerk has joined #openstack-infra | 13:55 | |
| *** Guest29471 has quit IRC | 13:55 | |
| *** Hal has joined #openstack-infra | 13:55 | |
| openstackgerrit | Antoine Cabot proposed openstack-infra/project-config: Add python-watcherclient doc on docs.openstack.org https://review.openstack.org/327107 | 13:55 |
| *** Hal has quit IRC | 13:55 | |
| *** Hal has joined #openstack-infra | 13:56 | |
| *** Hal is now known as Guest99800 | 13:56 | |
| *** jamesmcarthur has joined #openstack-infra | 13:57 | |
| *** thorst has quit IRC | 13:57 | |
| *** Guest99800 has quit IRC | 13:58 | |
| *** jamesmcarthur has quit IRC | 13:58 | |
| *** cloudtrainme has quit IRC | 13:59 | |
| *** rbrndt has joined #openstack-infra | 14:00 | |
| *** Swami has joined #openstack-infra | 14:01 | |
| *** cloudtrainme has joined #openstack-infra | 14:01 | |
| *** annegentle has joined #openstack-infra | 14:02 | |
| *** Julien-zte has joined #openstack-infra | 14:02 | |
| *** dimtruck is now known as zz_dimtruck | 14:03 | |
| *** jamesmcarthur has joined #openstack-infra | 14:03 | |
| krotscheck | clarkb: That requires root. | 14:04 |
| krotscheck | clarkb: you can only increase the file descriptor limit as root. Regular users can only decrease it. That means i'd have to write a brand new macro builder, go through review, wait for it to merge, etc etc. | 14:04 |
| *** zz_dimtruck is now known as dimtruck | 14:05 | |
| *** Swami_ has joined #openstack-infra | 14:05 | |
| *** watanabe_isao has joined #openstack-infra | 14:06 | |
| *** rossella_s has quit IRC | 14:07 | |
| *** jamesmcarthur has quit IRC | 14:08 | |
| *** yolanda_ has joined #openstack-infra | 14:08 | |
| *** Swami has quit IRC | 14:09 | |
| *** yolanda has quit IRC | 14:09 | |
| *** yolanda_ has quit IRC | 14:09 | |
| *** yolanda has joined #openstack-infra | 14:09 | |
| *** signed8bit_Zzz is now known as signed8b_ | 14:10 | |
| *** vsaienko1 has quit IRC | 14:12 | |
| *** jamesmcarthur has joined #openstack-infra | 14:12 | |
| AJaeger | clarkb: the repo was created yesterday, it should have been available. Is puppet not creating the repo on the nodepool server? | 14:13 |
| *** zeih has quit IRC | 14:13 | |
| *** tkelsey has joined #openstack-infra | 14:13 | |
| *** rossella_s has joined #openstack-infra | 14:13 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Create the ARA project https://review.openstack.org/321226 | 14:14 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Add a new purge_env command to Test env hosts https://review.openstack.org/327116 | 14:14 |
| *** Swami__ has joined #openstack-infra | 14:14 | |
| fungi | AJaeger: pabelanger: i think the requirements project there's already an other-requirements.txt which is supposed to contain all the requirements needed to be able to install the global requirements list (and then bindep is called by the job which generates the upper constraints list i think). would make sense to use that | 14:14 |
| *** infra-red has joined #openstack-infra | 14:14 | |
| mriedem | i'm surprised the tempest liberty compat job doesn't specify max_microversion in tempest.conf? http://logs.openstack.org/69/325569/7/check/gate-tempest-dsvm-neutron-full-liberty/cba3b22/logs/tempest_conf.txt.gz | 14:15 |
| AJaeger | fungi: indeed, we can add that to other-requirements. Let me do that quickly... | 14:15 |
| fungi | pleia2: pabelanger: clarkb: jamesmcarthur: looking at the file list, i think we need to clean those up too... https://wiki.openstack.org/wiki/Special:ListFiles?limit=500&ilsearch=&user= | 14:16 |
| AJaeger | fungi, I screwed up in reviewing: os-http was created in openstack-dev namespace, I missed that during review ;( | 14:16 |
| *** dimtruck is now known as zz_dimtruck | 14:16 | |
| fungi | fifieldt: ^ | 14:16 |
| *** jamesmcarthur has quit IRC | 14:16 | |
| clarkb | krotscheck: we give every job root | 14:16 |
| clarkb | krotscheck: but yes there is some work involved somewhere | 14:17 |
| *** Swami has joined #openstack-infra | 14:17 | |
| *** Swami_ has quit IRC | 14:18 | |
| fungi | pleia2: pabelanger: clarkb: jamesmcarthur: fifieldt: scroll down to uploads from the end of march and you'll see what i mean | 14:18 |
| *** EricGonczer_ has joined #openstack-infra | 14:19 | |
| *** rbrndt has quit IRC | 14:19 | |
| AJaeger | dims, dirk: https://review.openstack.org/327120 should fix the kerberos problem you noticed | 14:19 |
| *** vsaienko has joined #openstack-infra | 14:19 | |
| *** Swami__ has quit IRC | 14:19 | |
| *** ramishra_ has quit IRC | 14:19 | |
| *** fifieldt has quit IRC | 14:20 | |
| *** ramishra has joined #openstack-infra | 14:20 | |
| vsaienko | please help to merge https://review.openstack.org/#/c/316662 1 line patch that allows to pass provision/unprovision timeout for ironic grenade jobs | 14:20 |
| fungi | pleia2: pabelanger: clarkb: jamesmcarthur: fifieldt: actually, looks like there were a handful as late as march 18 ("Avast Technical Support" png image after pdf uploads were disabled but before new account creation was disabled) | 14:20 |
| *** erlon has joined #openstack-infra | 14:21 | |
| fungi | turning off new accounts and deleting all the ones which seemed to be spamming did at least put an end to that | 14:21 |
| dims | AJaeger : so we let the projects deal with it if they see it in their own repo? (since this was needed possibly on the client side) | 14:21 |
| dims | AJaeger : am ok with that | 14:22 |
| AJaeger | dims: that is our general approach forward: Projects should add their own other-requirements.txt files. | 14:22 |
| AJaeger | dims, see http://docs.openstack.org/infra/manual/drivers.html#package-requirements | 14:22 |
| fungi | dims: yeah, the up side to this is that it also serves as documentation to devs who want to try running the same tests locally | 14:23 |
| dims | fungi : works for me. thanks AJaeger | 14:23 |
| fungi | so they don't have to guess/trial-and-error to suss out the distro-packaged requirements | 14:23 |
| *** psachin has quit IRC | 14:23 | |
| *** jamesmcarthur has joined #openstack-infra | 14:24 | |
| fungi | as a convenience, you can also add a "bindep" env in tox that will list the specific packages they're missing on their platform | 14:24 |
| AJaeger | fungi, dims: Documented in the manual - see my link | 14:24 |
| dims | AJaeger : thanks! | 14:25 |
| *** jcoufal has quit IRC | 14:25 | |
| *** jaosorior has quit IRC | 14:26 | |
| *** mriedem1 has joined #openstack-infra | 14:27 | |
| *** mriedem has quit IRC | 14:27 | |
| *** pt_15 has joined #openstack-infra | 14:28 | |
| *** savihou has joined #openstack-infra | 14:28 | |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/infra-ansible: Reuse the puppet-openstackci zuul scheduler class https://review.openstack.org/327129 | 14:29 |
| pabelanger | fungi: Ya, for the most part there is a issue in bulk delete for spamming. Sometime not all attachments are removed. In this case, it appears to be a manual process | 14:29 |
| *** psilvad has joined #openstack-infra | 14:29 | |
| pabelanger | rcarrillocruz: yolanda: I am ready if you are | 14:29 |
| openstackgerrit | Thomas Goirand proposed openstack-dev/pbr: Add --sysconfdir option to install command https://review.openstack.org/327132 | 14:29 |
| *** links has quit IRC | 14:29 | |
| *** esikachev has quit IRC | 14:31 | |
| *** yamahata has joined #openstack-infra | 14:31 | |
| *** infra-red has quit IRC | 14:32 | |
| openstackgerrit | Julien Danjou proposed openstack-dev/pbr: tests: fix data_files testing https://review.openstack.org/278360 | 14:32 |
| *** psilvad has quit IRC | 14:32 | |
| fungi | pabelanger: clarkb pointed out that the wiki search (now that it's updating correctly again) provides a better check of what patterns are still matched than the mass delete does, and also searches in page contents | 14:33 |
| *** jamesmca_ has joined #openstack-infra | 14:33 | |
| *** jamesmcarthur has quit IRC | 14:33 | |
| *** psilvad has joined #openstack-infra | 14:34 | |
| rcarrillocruz | pabelanger: sure | 14:34 |
| rcarrillocruz | here, or other channel? or maybe an etherpad? | 14:34 |
| mtreinish | clarkb: it is generating subunit with generate-subunit (which I primarily wrote because I was impatient about: https://github.com/testing-cabal/subunit/pull/17 ) | 14:34 |
| *** tonytan4ever has joined #openstack-infra | 14:34 | |
| fungi | pabelanger: interestingly, the mass delete special page doesn't seem to match even on page titles for spam pages the wiki search finds (which i've confirmed still exist). i have a feeling the fact that mass delete says it's for "pages recently added" makes me think those are past its timeframe limit | 14:36 |
| pabelanger | rcarrillocruz: lets start here | 14:36 |
| rcarrillocruz | so | 14:36 |
| jroll | this seems to be the last patch outstanding to get grenade passing for ironic, just a one-liner if someone has a moment https://review.openstack.org/#/c/316662/ | 14:36 |
| rcarrillocruz | we have between 129 to 150 servers , yet to see how many depending on HD issues and other faulty hw | 14:37 |
| *** julim has quit IRC | 14:37 | |
| rcarrillocruz | we moved all those servers to Houston datacenter, something they call AM1 location | 14:37 |
| rcarrillocruz | i've been working via tickets the issue of not being able to access them via iLO | 14:37 |
| *** zeih has joined #openstack-infra | 14:37 | |
| *** oanson has joined #openstack-infra | 14:37 | |
| pabelanger | fungi: Ya, the mass delete doesn't always pick things up. I am not sure why right now | 14:37 |
| *** piet has joined #openstack-infra | 14:37 | |
| rcarrillocruz | they put the servers in a wrong VLAN | 14:37 |
| jroll | there's also a project-config patch that should unbreak ironic CI on stable/liberty here, that already has a +2 https://review.openstack.org/#/c/326137/ | 14:37 |
| rcarrillocruz | now, that's sorted | 14:37 |
| *** psilvad has quit IRC | 14:37 | |
| rcarrillocruz | i can access the servers via iLO | 14:37 |
| *** psilvad has joined #openstack-infra | 14:38 | |
| pabelanger | rcarrillocruz: regarding tickets, can non HPE employees access them? | 14:38 |
| AJaeger | jroll: I'll review 326137. Could you review https://review.openstack.org/326203 , please? | 14:38 |
| rcarrillocruz | one of the issues we didn't foresee is to keep a more thorough info of the servers, in particular serial numbers and iLO ips ( we just keep the provisioning NIC mac on system-config hiera) | 14:38 |
| fungi | pabelanger: okay, i guess i'll dive into this myself. it seems like everyone else is too busy to deal with the wiki right now | 14:38 |
| jroll | AJaeger: yep, looking | 14:38 |
| rcarrillocruz | pabelanger: no, only HPE employees can, you need to VPN in to HPE to access Jira | 14:38 |
| openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate: DNM: experimenting with local.conf https://review.openstack.org/326585 | 14:38 |
| rcarrillocruz | i found by guessing the baremetal00 machine | 14:39 |
| rcarrillocruz | the baremetal00 machine is our 'jumphost' and the bifrost server itself | 14:39 |
| rcarrillocruz | pabelanger: you familiar with bifrost? | 14:39 |
| rcarrillocruz | i can give a super short overview of what it is | 14:40 |
| pabelanger | rcarrillocruz: yes, I haven't run it yet | 14:40 |
| pabelanger | but looked at the code | 14:40 |
| rcarrillocruz | ok, so baremetal00=bifrost | 14:40 |
| openstackgerrit | Merged openstack-infra/nodepool: Add nodepool.manager.MANAGER.nodes.STATE also https://review.openstack.org/326774 | 14:40 |
| *** julim has joined #openstack-infra | 14:40 | |
| rcarrillocruz | which acts as the DHCP/PXE server to boot up the controller and computes from the 'infracloud' itself | 14:40 |
| *** julim has quit IRC | 14:40 | |
| rcarrillocruz | now issue is: | 14:40 |
| openstackgerrit | Merged openstack-infra/storyboard: Filter non-public fields when searching for users https://review.openstack.org/324725 | 14:40 |
| rcarrillocruz | i can get in to the baremetal00 via iLO textcons interface | 14:41 |
| rcarrillocruz | howerver, there is a networking issue . The server can't ping other servers on the mgmt network neither the internet on the public network | 14:41 |
| *** jamesmca_ has quit IRC | 14:41 | |
| yolanda | sorry, not available now, pabelanger | 14:41 |
| rcarrillocruz | as soon as I sort out that with DC folks (ticket opened as blocker) | 14:41 |
| rcarrillocruz | i'll open up access via internet | 14:41 |
| rcarrillocruz | so you and other core folks can just ssh to it | 14:42 |
| *** jamesmcarthur has joined #openstack-infra | 14:42 | |
| *** rbrndt has joined #openstack-infra | 14:42 | |
| rcarrillocruz | we will then: | 14:42 |
| rcarrillocruz | 1. change hiera for infracloud with new data (new IPs, serial numbers, iLO ips, etc) | 14:42 |
| rcarrillocruz | 2. redeploy | 14:42 |
| rcarrillocruz | 3. puppet openstack mitaka (or whatever) | 14:42 |
| pabelanger | yolanda: np, ping me again when free | 14:43 |
| pabelanger | rcarrillocruz: okay, how long do you expect that take? | 14:43 |
| rcarrillocruz | by having the serial numbers and ilo ips for all machines on our hiera, i expect it will be WAY EASIER to find out what server is what | 14:43 |
| pabelanger | rcarrillocruz: mostly curious of the turn around times on tickets | 14:43 |
| *** esikachev has joined #openstack-infra | 14:43 | |
| rcarrillocruz | cos right now we just keep the old ip plus the pxe mac | 14:43 |
| pabelanger | ++ | 14:43 |
| rcarrillocruz | but DC folks they just know about iLOs and serial numbers | 14:44 |
| rcarrillocruz | pabelanger: well, i've spent last night entirely to get the baremetal00 access via iLO | 14:44 |
| rcarrillocruz | today i spent whole day with a NET engineer, per him the switch and switch ports are all good, vlan configs etc | 14:44 |
| rcarrillocruz | so most likely a cabling issue | 14:44 |
| pabelanger | okay | 14:44 |
| rcarrillocruz | the DC guy *should* be in shortly | 14:44 |
| rcarrillocruz | the ticket is blocker, which is the highest sev, so i expect they respond today | 14:45 |
| AJaeger | rcarrillocruz: thanks for moving this forward! | 14:45 |
| rcarrillocruz | sure thing | 14:45 |
| pabelanger | rcarrillocruz: so for #1, I assume the serial numbers are located in each iLo? | 14:46 |
| rcarrillocruz | i'm hoping when crinkle is back to US we can move this forward faster, mainly because of TZ difference. DC folks in Houston, me in Spain | 14:46 |
| rcarrillocruz | don't play well :/ | 14:46 |
| *** anteaya has joined #openstack-infra | 14:46 | |
| rcarrillocruz | pabelanger: i have all serial numbers in a spreadsheet | 14:46 |
| rcarrillocruz | however i can't just put on our hiera | 14:46 |
| rcarrillocruz | cos in our hiera | 14:46 |
| *** mriedem1 is now known as mriedem | 14:46 | |
| rcarrillocruz | we have: | 14:46 |
| rcarrillocruz | pxe mac | 14:46 |
| pabelanger | okay | 14:47 |
| rcarrillocruz | old pxe IP | 14:47 |
| rcarrillocruz | old public IP | 14:47 |
| openstackgerrit | Yaroslav Lobankov proposed openstack-infra/project-config: [Rally] Enable Ceilometer for one of the 'rally verify' jobs https://review.openstack.org/327157 | 14:47 |
| rcarrillocruz | but on the spreadsheet i have | 14:47 |
| rcarrillocruz | serial number | 14:47 |
| rcarrillocruz | iLO ip | 14:47 |
| *** cargonza has joined #openstack-infra | 14:47 | |
| rcarrillocruz | so it's fun fun | 14:47 |
| pabelanger | right | 14:47 |
| rcarrillocruz | cos I have to get into the ilO with textcons | 14:48 |
| rcarrillocruz | to figure out what is what | 14:48 |
| rcarrillocruz | plus because we didn't have a user/pass in common with all servers | 14:48 |
| rcarrillocruz | i have to: | 14:48 |
| rcarrillocruz | 1. reboot the server | 14:48 |
| rcarrillocruz | 2. wait 5-10min with the left shift key pressed | 14:48 |
| rcarrillocruz | 3. enter on recovery mode via grub | 14:48 |
| rcarrillocruz | that gives me the keys to the server in question | 14:48 |
| rcarrillocruz | so yeah, it's taking time | 14:48 |
| rcarrillocruz | however, if i can give public access to the baremetal00 machine | 14:49 |
| rcarrillocruz | that should give us the keys to the kingdom | 14:49 |
| *** mtanino has joined #openstack-infra | 14:49 | |
| fungi | pabelanger: found it. the nuke extension queries the recent changes table, which as of 1.25 (the version we're running) gets purged of entries older than 90 days by default. it's basically useless to us now because all the spam pages are from >90 days ago | 14:49 |
| rcarrillocruz | because bifrost bakes in a key on the root user of all controllers and computes it provisions | 14:49 |
| *** zeih has quit IRC | 14:49 | |
| rcarrillocruz | so priority for me is get baremetal00 good and dandy | 14:49 |
| fungi | pabelanger: so at this point, i guess cleanup is entirely manual and page-at-a-time | 14:49 |
| *** dizquierdo has joined #openstack-infra | 14:50 | |
| pabelanger | fungi: Oooh, nice find. Any way we can hack nuke to remove the 90day limit? | 14:50 |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Add a subscription button to project detail view https://review.openstack.org/323351 | 14:50 |
| *** Hal has joined #openstack-infra | 14:50 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient: Add a subscription button to project group detail view https://review.openstack.org/323367 | 14:50 |
| jeblair | clarkb, fungi, pabelanger, AJaeger: this change just failed because we ran a projects.yaml test on it and it referenced a github repo that doesn't exist: http://logs.openstack.org/42/326742/1/gate/gate-project-config-projects-yaml/d7eb791/console.html the change does not use projects.yaml | 14:50 |
| pabelanger | rcarrillocruz: sure, let me know if I can help with anything. | 14:50 |
| *** Hal is now known as Guest93364 | 14:50 | |
| fungi | pabelanger: no, we need to configure https://www.mediawiki.org/wiki/Manual:$wgRCMaxAge if we want it to not delete recent changes entries so quickly in the future, but ones from the past are already gone | 14:50 |
| rcarrillocruz | as soon as i get that online i will let you and other folks | 14:51 |
| *** watanabe_isao has quit IRC | 14:51 | |
| rcarrillocruz | know | 14:52 |
| pabelanger | jeblair: Hmm, I know that review. Let me see if I can find it | 14:53 |
| fungi | jeblair: we should probably disable that check until it can be reengineered to only complain about entries added by the change being tested | 14:53 |
| jeblair | fungi: well, i think it's a really great check, though, it used to only run when we changed that file | 14:54 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/infra-ansible: Wait for server to be ssh reachable before bootstrap/configuration https://review.openstack.org/326957 | 14:54 |
| pabelanger | jeblair: fungi: 326720 added puppet-congress to openstack | 14:54 |
| fungi | because the only problem a nonexistent import url presents is for repo import of teh change being approved. once that repo is imported, we don't want subsequent deletion of the remote repo to result in spontaneous test failure | 14:55 |
| jeblair | fungi: i'm hoping to illustrate that we have been silently benefiting from having the files sections. | 14:55 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Add a new purge_env command to Test env hosts https://review.openstack.org/327116 | 14:55 |
| pabelanger | rcarrillocruz: works for me | 14:55 |
| jeblair | fungi: that's why i wrote the cleanup proposal job | 14:55 |
| openstackgerrit | Ricardo Carrillo Cruz proposed openstack-infra/infra-ansible: Set resolv.conf on nodes before bootstrap https://review.openstack.org/326959 | 14:55 |
| *** jcoufal has joined #openstack-infra | 14:55 | |
| pabelanger | jeblair: fungi: I am happy to revert the files changes, what ever the team thinks is best | 14:56 |
| rcarrillocruz | jeblair: have a sec to talk about ansible-puppet? or fungi ? is there a list of issues for the role or can you tell me what are the pain points? | 14:56 |
| fungi | jeblair: makes sense, just pointing out that between when the remote repo is imported and then deleted by its prior owner and when the cleanup proposal merges, there's a window where that job will fail for other changes adding unrelated new projects | 14:56 |
| *** salv-orl_ has quit IRC | 14:56 | |
| fungi | even if we reinstitute the file filtyering | 14:56 |
| jeblair | fungi: agreed, it could certainly be smarter | 14:56 |
| jeblair | there must be an outstanding proposal bot change to remove that url, but i can't find it | 14:57 |
| *** salv-orlando has joined #openstack-infra | 14:57 | |
| *** links has joined #openstack-infra | 14:57 | |
| *** nadya has joined #openstack-infra | 14:58 | |
| *** abregman_ has joined #openstack-infra | 14:58 | |
| *** thiagop has joined #openstack-infra | 14:59 | |
| *** abregman has quit IRC | 14:59 | |
| *** yujunz has joined #openstack-infra | 14:59 | |
| openstackgerrit | Vasyl Saienko proposed openstack-infra/project-config: Make grenade jobs for Ironic non voting https://review.openstack.org/319336 | 15:00 |
| *** fawadkhaliq has joined #openstack-infra | 15:00 | |
| *** nadya_ has joined #openstack-infra | 15:01 | |
| jroll | AJaeger: approve away on 326203 | 15:01 |
| pabelanger | jeblair: fungi: better to check if repo exists in git.o.o first, before hitting github.com? | 15:01 |
| jeblair | oh, it only recently merged, so it would get cleaned up tomorrow | 15:01 |
| *** amitgandhinz has quit IRC | 15:02 | |
| fungi | pabelanger: that's an interesting idea. have the job only check import urls for repos it confirms aren't imported yet | 15:02 |
| jeblair | pabelanger: well, i think the logic should actually be: if track-upstream is true, always check the upstream. if it is false, ensure it exists either in gerrit or the upstream works. | 15:02 |
| *** Julien-z_ has joined #openstack-infra | 15:02 | |
| fungi | yep, that | 15:02 |
| *** zhurong has quit IRC | 15:02 | |
| fungi | i keep forgetting about the track-upstream option | 15:02 |
| pabelanger | ya, I am not sure how that job works actually. Going to poke at it to see | 15:03 |
| *** tonytan4ever has quit IRC | 15:03 | |
| *** zz_dimtruck is now known as dimtruck | 15:03 | |
| *** nadya has quit IRC | 15:03 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Remove puppet-congress upstream url https://review.openstack.org/327172 | 15:04 |
| jeblair | fungi, pabelanger: ^ | 15:04 |
| pabelanger | +2 | 15:04 |
| fungi | pleia2: pabelanger: clarkb: jamesmcarthur: fifieldt: i'm going through now and deleting obviously spam files from the wiki files special page after first checking whether they are linked from another article (so that i can potentially delete it too) | 15:04 |
| *** zul_ has joined #openstack-infra | 15:05 | |
| anteaya | fungi: yay | 15:05 |
| anteaya | fungi: thank you | 15:05 |
| fungi | we'll see how far i get. it's a manual process now that the mass delete extension is no longer usable | 15:05 |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config: Remove jenkins01 target from nodepool https://review.openstack.org/326742 | 15:05 |
| *** zul has quit IRC | 15:05 | |
| *** frickler has quit IRC | 15:06 | |
| *** Julien-zte has quit IRC | 15:06 | |
| *** frickler has joined #openstack-infra | 15:06 | |
| *** zul_ has quit IRC | 15:06 | |
| fungi | i've cleaned up all the non-pdf spam file uploads from march, but there were only a few. there are numerous pdf uploads from february which are spam | 15:06 |
| fungi | working my way back through that list now | 15:07 |
| *** zul has joined #openstack-infra | 15:07 | |
| anteaya | fungi: yay! | 15:07 |
| *** tonytan4ever has joined #openstack-infra | 15:09 | |
| jamesmcarthur | fungi: thanks!!! | 15:10 |
| *** faizy has joined #openstack-infra | 15:10 | |
| *** e0ne_ is now known as e0ne | 15:11 | |
| *** nadya_ has quit IRC | 15:12 | |
| *** amitgandhinz has joined #openstack-infra | 15:13 | |
| *** dprince has quit IRC | 15:13 | |
| *** piet has quit IRC | 15:14 | |
| *** mriedem1 has joined #openstack-infra | 15:15 | |
| openstackgerrit | Dan Radez proposed openstack-infra/project-config: Move puppet-tacker to Openstack https://review.openstack.org/327178 | 15:16 |
| *** mriedem has quit IRC | 15:17 | |
| *** samuelBartel has joined #openstack-infra | 15:17 | |
| *** ggnel_t has quit IRC | 15:17 | |
| *** sorantis has quit IRC | 15:18 | |
| *** thingee has quit IRC | 15:18 | |
| *** david-lyle has joined #openstack-infra | 15:19 | |
| *** jamesmcarthur has quit IRC | 15:19 | |
| *** davidlenwell has quit IRC | 15:20 | |
| *** julim has joined #openstack-infra | 15:20 | |
| *** annegentle has quit IRC | 15:20 | |
| *** julim has quit IRC | 15:20 | |
| *** julim has joined #openstack-infra | 15:21 | |
| *** akshai has quit IRC | 15:21 | |
| *** akshai has joined #openstack-infra | 15:21 | |
| *** julim has quit IRC | 15:22 | |
| *** julim has joined #openstack-infra | 15:22 | |
| dulek | Is anyone able to point me to resources on what plumbing is needed to enable multinode Tempest/Grenade for another service? I'm specifically talking about Cinder now. | 15:23 |
| dulek | Maybe at least a raw TODO list? ;) | 15:23 |
| *** woodster_ has joined #openstack-infra | 15:24 | |
| *** mriedem1 is now known as mriedem | 15:24 | |
| *** faizy has quit IRC | 15:24 | |
| *** xarses has joined #openstack-infra | 15:25 | |
| *** samuelBartel has quit IRC | 15:25 | |
| *** cloudtrainme has quit IRC | 15:25 | |
| *** cloudtrainme has joined #openstack-infra | 15:27 | |
| *** yamamoto has joined #openstack-infra | 15:28 | |
| yolanda | infra cores, can i get reviews for https://review.openstack.org/326605 and https://review.openstack.org/326885 ? | 15:28 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Skip upstream check if we have already imported upstream https://review.openstack.org/327189 | 15:28 |
| pabelanger | fungi: jeblair: I believe that should work ^ | 15:28 |
| *** dizquierdo has quit IRC | 15:28 | |
| fungi | i should see if i can figure out how to speed up these deletions. at my current pace i'm on track to be able to delete the ~1300 spam images from late february in just under 9 hours of continuous clicking | 15:29 |
| *** dmk0202 has quit IRC | 15:29 | |
| fungi | if my fingers don't fall off first | 15:29 |
| anteaya | fungi: that sounds like fun | 15:29 |
| *** ajmiller has joined #openstack-infra | 15:29 | |
| *** pblaho has joined #openstack-infra | 15:30 | |
| anteaya | if we look at another wiki option ever can we evaluate it for ease of mass deletion | 15:30 |
| *** dizquierdo has joined #openstack-infra | 15:30 | |
| anteaya | I mean actual mass deletion | 15:30 |
| openstackgerrit | Vitaly Gridnev proposed openstack-infra/project-config: make all sahara-dashboard jobs voting https://review.openstack.org/326474 | 15:30 |
| anteaya | not just something that looks like mass deletion but isn't | 15:30 |
| mnaser | i could swear there was a link in gerrit that allowed cloning the repo for a specific review | 15:30 |
| mnaser | or somewhere to access it | 15:30 |
| fungi | mnaser: the "download" button at the top does that now | 15:31 |
| mnaser | oooo | 15:31 |
| mnaser | thank you! | 15:31 |
| openstackgerrit | Vitaly Gridnev proposed openstack-infra/project-config: make all sahara-dashboard jobs voting https://review.openstack.org/326474 | 15:31 |
| *** ajmiller__ has quit IRC | 15:31 | |
| fungi | mnaser: note that "http" means "authenticated http" so you may want to select "anonymous http" instead of that | 15:32 |
| mnaser | i only see ssh and anonymous http options anyways | 15:32 |
| *** julim has quit IRC | 15:32 | |
| anteaya | mnaser: are you signed into gerrit? | 15:32 |
| *** yamamoto has quit IRC | 15:33 | |
| mnaser | yes | 15:33 |
| *** ams__ has joined #openstack-infra | 15:34 | |
| timothyb89 | pabelanger: it looks like that stackviz error is in an optional dependency, probably safe to ignore? | 15:34 |
| pabelanger | anteaya: moinmoin is great for that, I believe wiki pages are stored on the local file system. I believe we'd simply delete the file to remove the page, but need to confirm that | 15:35 |
| mriedem | umm, jenkins is randomly telling me that i have a change which failed to merge, but doesn't show merge conflict, and i've already manually rebased the series once | 15:35 |
| mriedem | is this a known issue? | 15:35 |
| mriedem | and no depends-on | 15:35 |
| pabelanger | timothyb89: Ya, I think it is ignored right now. Not sure what affect it has on stackviz | 15:35 |
| anteaya | pabelanger: I think we moved from moinmoin if I remember my early history | 15:35 |
| anteaya | mriedem: have you a link? | 15:36 |
| *** mikelk has quit IRC | 15:36 | |
| mriedem | anteaya: https://review.openstack.org/#/c/297906/ | 15:36 |
| mriedem | xenproject ci says the same | 15:36 |
| mriedem | can't figure it out | 15:36 |
| anteaya | mnaser: my dropdown menus also contains on two options ssh and anonymous http | 15:36 |
| *** pblaho has quit IRC | 15:37 | |
| mriedem | anteaya: that's change 3 in the series, the first 2 changes are going through check queue now | 15:37 |
| anteaya | mriedem: yes, I'm looking at the parents, they last passed check yesterday | 15:38 |
| anteaya | I'm wondering if one of them currently has a merge conflict | 15:38 |
| *** Swami_ has joined #openstack-infra | 15:38 | |
| timothyb89 | pabelanger: it seems to have built fine, hopefully no negative effect at all | 15:39 |
| *** reed has quit IRC | 15:40 | |
| *** nadya has joined #openstack-infra | 15:40 | |
| mriedem | anteaya: don't think so. i rebased the series around an hour ago | 15:40 |
| *** Swami__ has joined #openstack-infra | 15:40 | |
| anteaya | hmmmm | 15:40 |
| *** samuelBartel has joined #openstack-infra | 15:40 | |
| *** nadya has quit IRC | 15:40 | |
| pabelanger | timothyb89: yup, uploads should be starting shortly | 15:41 |
| *** abregman_ has quit IRC | 15:41 | |
| anteaya | none of the first three patches have Depends-On in the commit message | 15:41 |
| mriedem | anteaya: and i have a tempest change that's dependent on the top nova change in the series, and the tempest change is running through check now https://review.openstack.org/#/c/327191/ | 15:41 |
| *** hdaniel has quit IRC | 15:41 | |
| *** Swami has quit IRC | 15:42 | |
| anteaya | mriedem: oh I went the wrong way, I always think the parent is at the bottom of the stack not the top | 15:42 |
| anteaya | let me go through the stack again | 15:42 |
| timrc | I feel strongly that openstack-infra should port this to storyboard https://gist.github.com/penk/8415099 | 15:42 |
| *** julim has joined #openstack-infra | 15:42 | |
| *** zeih has joined #openstack-infra | 15:43 | |
| *** vincentll has quit IRC | 15:43 | |
| *** julim has quit IRC | 15:43 | |
| *** Swami_ has quit IRC | 15:43 | |
| *** zeih has quit IRC | 15:43 | |
| anteaya | timrc: make a story about it | 15:43 |
| pabelanger | timrc: epic | 15:43 |
| *** zeih has joined #openstack-infra | 15:44 | |
| *** julim has joined #openstack-infra | 15:44 | |
| *** julim has quit IRC | 15:44 | |
| *** Swami__ has quit IRC | 15:45 | |
| anteaya | epics are a different feature | 15:45 |
| *** julim has joined #openstack-infra | 15:46 | |
| *** julim has quit IRC | 15:46 | |
| anteaya | mriedem: when you say the top nova change you mean the tip of the series? | 15:46 |
| *** vhosakot has joined #openstack-infra | 15:46 | |
| *** yamamoto has joined #openstack-infra | 15:46 | |
| *** julim has joined #openstack-infra | 15:47 | |
| mriedem | anteaya: yeah, HEAD | 15:47 |
| *** amitgandhinz has quit IRC | 15:47 | |
| anteaya | okay thanks | 15:48 |
| *** amitgandhinz has joined #openstack-infra | 15:48 | |
| *** zeih has quit IRC | 15:48 | |
| *** sulo has left #openstack-infra | 15:49 | |
| jeblair | mriedem, anteaya: i'm checking zuul logs | 15:50 |
| anteaya | jeblair: thank you | 15:50 |
| *** Apoorva has joined #openstack-infra | 15:51 | |
| anteaya | jesusaur: I'm wondering if logstash is logstashing to the extent that it could be logstashing, might you have an opportunity to take a peek? | 15:54 |
| *** yamamoto has quit IRC | 15:56 | |
| *** davidlenwell has joined #openstack-infra | 15:56 | |
| *** yamamoto has joined #openstack-infra | 15:56 | |
| *** tzn has quit IRC | 15:57 | |
| *** julim has quit IRC | 15:57 | |
| *** chlong has quit IRC | 15:58 | |
| *** tzn has joined #openstack-infra | 15:58 | |
| *** samuelBartel has quit IRC | 15:59 | |
| *** Jeffrey4l has quit IRC | 15:59 | |
| *** yamamoto has quit IRC | 16:00 | |
| *** yamamoto has joined #openstack-infra | 16:00 | |
| *** markusry has quit IRC | 16:00 | |
| *** timello has joined #openstack-infra | 16:01 | |
| openstackgerrit | Vasyl Saienko proposed openstack-infra/project-config: Make grenade jobs for Ironic non voting https://review.openstack.org/319336 | 16:01 |
| *** cody-somerville has joined #openstack-infra | 16:02 | |
| *** mhickey has quit IRC | 16:02 | |
| *** markmcd has quit IRC | 16:03 | |
| *** yujunz has quit IRC | 16:03 | |
| *** Rockyg has joined #openstack-infra | 16:04 | |
| *** benny_ has joined #openstack-infra | 16:06 | |
| *** fawadkhaliq has quit IRC | 16:07 | |
| *** xarses has quit IRC | 16:07 | |
| *** markmcd has joined #openstack-infra | 16:07 | |
| *** reed has joined #openstack-infra | 16:07 | |
| *** cody-somerville has quit IRC | 16:08 | |
| *** Sukhdev_ has joined #openstack-infra | 16:08 | |
| *** nadya has joined #openstack-infra | 16:10 | |
| *** markmcd has quit IRC | 16:10 | |
| *** julim has joined #openstack-infra | 16:11 | |
| *** arxcruz has quit IRC | 16:11 | |
| *** shamail has joined #openstack-infra | 16:12 | |
| SpamapS | cinerama: did we ever get a glean release out for you? | 16:12 |
| *** markmcd has joined #openstack-infra | 16:12 | |
| *** nmagnezi has quit IRC | 16:12 | |
| *** julim has quit IRC | 16:12 | |
| *** xarses has joined #openstack-infra | 16:12 | |
| *** xarses has quit IRC | 16:13 | |
| *** julim has joined #openstack-infra | 16:13 | |
| *** xarses has joined #openstack-infra | 16:13 | |
| *** Sukhdev_ has quit IRC | 16:14 | |
| *** sarob has joined #openstack-infra | 16:14 | |
| cinerama | SpamapS, lemme check | 16:14 |
| greghaynes | SpamapS: cinerama looks like no | 16:14 |
| *** gyee has joined #openstack-infra | 16:15 | |
| greghaynes | (sorry, had it up already ;) | 16:15 |
| *** links has quit IRC | 16:15 | |
| cinerama | lol | 16:15 |
| *** vhosakot has quit IRC | 16:15 | |
| cinerama | yeah so no | 16:15 |
| *** piet has joined #openstack-infra | 16:16 | |
| cinerama | i'm trying to clear out bifrost's backlog of changes | 16:16 |
| *** salv-orlando has quit IRC | 16:18 | |
| anteaya | mriedem: the two parents passing is hopeful, I have my fingers crossed for the recheck passing | 16:18 |
| *** salv-orlando has joined #openstack-infra | 16:18 | |
| mriedem | anteaya: the recheck already failed | 16:18 |
| *** _shaps_ has quit IRC | 16:19 | |
| *** rossella_s has quit IRC | 16:19 | |
| *** sarob has quit IRC | 16:19 | |
| mriedem | oh i guess not | 16:19 |
| mriedem | just queued | 16:19 |
| fungi | okay, i have skimmed all the file uploads and assembled a list of 178 unique wiki users who uploaded at least one of them. now to see what sort of interesting things i can do with this data | 16:19 |
| anteaya | mriedem: oh boo | 16:19 |
| fungi | users who uploaded at least one spam file, that is | 16:19 |
| anteaya | mriedem: oh okay | 16:19 |
| *** _shaps_ has joined #openstack-infra | 16:20 | |
| *** xarses has quit IRC | 16:20 | |
| *** jtomasek has quit IRC | 16:20 | |
| anteaya | fungi: I'm interested in what interesting things you can find out about them | 16:20 |
| fungi | anteaya: more like what interesting things i can do with them, for example https://www.mediawiki.org/wiki/API:Delete | 16:20 |
| anteaya | I had been following them one at a time back to their edits page and the branching off from there | 16:20 |
| fungi | browser interaction for this stuff is just waaaay too slow | 16:21 |
| anteaya | I hope it works | 16:21 |
| anteaya | agreed | 16:21 |
| fungi | e.g., query all pages created by and images uploaded by users in this list, then perform corresponding delete actions on them | 16:21 |
| *** thingee has joined #openstack-infra | 16:22 | |
| *** nadya has quit IRC | 16:23 | |
| *** _shaps_ has quit IRC | 16:24 | |
| *** salv-orl_ has joined #openstack-infra | 16:24 | |
| anteaya | fungi: oh I hope it does it | 16:24 |
| fungi | unfortunately, for example the file list special page which claims to be able to search by username returns no results for the ones i've tried so far, even though there are file pages claiming they were the uploaders | 16:25 |
| openstackgerrit | David Lyle proposed openstack-infra/project-config: horizon job to use tempest plugin in gate https://review.openstack.org/327226 | 16:26 |
| *** Sukhdev_ has joined #openstack-infra | 16:26 | |
| zaro | jeblair: i've finally gotten around to setting up cgit server. was wondering whether we wanted to test performance with all protocals (git, smart http, and dumb http) or was the smart http enough? | 16:26 |
| anteaya | fungi: :( | 16:26 |
| *** Julien-z_ has quit IRC | 16:26 | |
| jeblair | mriedem, anteaya: it looks like at the time zuul queried gerrit about 297906,15, gerrit was not reporting 309574,12 as a dependency. later, when something else changed on 297906,15, zuul re-queried it, got the correct data, and invalidated its tests since the preconditions didn't make sense anymore. | 16:27 |
| krotscheck | pabelanger: Just to clarify- the logs that were uploaded this morning, from the dib build, appear to be broken. From your previous comment, I assume that these were actually the logs from yesterday's build? | 16:27 |
| krotscheck | pabelanger: I'm asking because a new one was just uploaded in the last 30 minutes, and it's also broken. | 16:27 |
| *** daemontool has joined #openstack-infra | 16:27 | |
| jeblair | mriedem, anteaya: we actually have (naive) protection in zuul against this sort of thing, but it appears 5 seconds was not sufficient this time: http://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/connection/gerrit.py#n51 | 16:27 |
| *** salv-orlando has quit IRC | 16:27 | |
| *** banix has quit IRC | 16:28 | |
| *** mixos has joined #openstack-infra | 16:28 | |
| *** Sukhdev_ has quit IRC | 16:29 | |
| jeblair | mriedem, anteaya: a recheck should certainly work in this instance (since gerrit's data are now consistent) | 16:29 |
| anteaya | jeblair: I did not know about this, how interesting | 16:29 |
| *** mixos_ has joined #openstack-infra | 16:29 | |
| jeblair | if this happens more often, it may indicate that gerrit is getting slower, and we either need to speed up gerrit, or increase the zuul delay | 16:30 |
| *** e0ne has quit IRC | 16:30 | |
| *** yamahata has quit IRC | 16:30 | |
| jeblair | zaro: all are used quite heavily, so we should test all of them i think | 16:30 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: WIP: Rework timeline events into a timeline https://review.openstack.org/327231 | 16:30 |
| *** yamamoto has quit IRC | 16:30 | |
| mriedem | jeblair: ok, i'll keep an eye on it. this is the 2nd time it's happened in this series. that's why i rebased this morning. | 16:31 |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Expose postci trace https://review.openstack.org/326759 | 16:31 |
| *** trown is now known as trown|lunch | 16:31 | |
| jeblair | mriedem: i also wonder if it varies by project -- perhaps because the nova repo is so big (zaro is working on that!) gerrit operations on nova may be slower. [just speculation] | 16:31 |
| *** tesseract has quit IRC | 16:31 | |
| *** dtantsur is now known as dtantsur|afk | 16:32 | |
| anteaya | I wonder if the size of the patch series also has an effect | 16:32 |
| anteaya | nova + 8 patch series | 16:32 |
| jeblair | anteaya: quite possible | 16:32 |
| fungi | anteaya: scratch that, it is working. i must have had an extra broken query parameter in there or something... https://wiki.openstack.org/wiki/Special:ListFiles?limit=50&ilsearch=&user=Fdgfdhge | 16:32 |
| jeblair | at any rate, it is very easy for us to further delay zuul if it's becoming a problem | 16:33 |
| anteaya | fungi: yay | 16:33 |
| *** d0ugal has quit IRC | 16:33 | |
| *** mixos has quit IRC | 16:33 | |
| fungi | so i should be able to leverage an api version of this to make the list of spam uploads and pages created by these accounts | 16:33 |
| anteaya | fungi: go you! | 16:34 |
| *** yamamoto has joined #openstack-infra | 16:34 | |
| *** weshay is now known as weshay_brb | 16:34 | |
| dhellmann | jeblair : I see zuul has quite long queue lengths reports, and I know you've explained to me before what's happening when I see that, but I can't remember. Is it rebuilding something as the result of a gate reset? | 16:36 |
| dhellmann | s/reports/reported/ | 16:36 |
| pabelanger | krotscheck: Right, we start DIBs each morning at 11:34 UTC. That usually takes about 1 hour per image, and we have 6 images. After that, its about another 5 hours to upload all images to clouds, which just started about 15mins ago. All that to say, we are still using the images from 2+ days ago in the gate | 16:36 |
| *** yolanda has quit IRC | 16:37 | |
| *** nadya has joined #openstack-infra | 16:37 | |
| krotscheck | pabelanger: 11:34 UTC< or 23:34? | 16:37 |
| *** vgridnev has quit IRC | 16:38 | |
| anteaya | dhellmann: yes it is recalculating the gate queue | 16:38 |
| *** vgridnev has joined #openstack-infra | 16:38 | |
| *** vgridnev has quit IRC | 16:38 | |
| *** sorantis has joined #openstack-infra | 16:38 | |
| dhellmann | anteaya : thanks | 16:38 |
| anteaya | dhellmann: when it is recalculating the queue grow because it isn't process incoming or outgoing events | 16:38 |
| anteaya | dhellmann: welcome | 16:38 |
| pabelanger | krotscheck: 10:34UTC actually: http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/nodepool.yaml#n8 | 16:38 |
| *** yamamoto has quit IRC | 16:39 | |
| anteaya | queues, processing | 16:39 |
| dhellmann | anteaya : that makes sense. I'll keep an eye out for it to finish | 16:39 |
| jeblair | dhellmann: contributing factors may include: 1) a zuul reconfiguration causing zuul to drop its cache and having to query gerrit a lot; 2) a lot of changes merging and causing zuul to query gerrit for open changes to populate the merge-check pipelines [combined with 1, this can get really bad]; 3) gerrit being slow to merge changes or replicate | 16:39 |
| anteaya | dhellmann: awesome, thank you | 16:39 |
| pabelanger | krotscheck: I want to expose the data better to grafana.o.o, so users have a better way to track the builds / uploads | 16:39 |
| fungi | anteaya: woo! step 1... https://wiki.openstack.org/w/api.php?action=query&list=allimages&format=json&aiprop=user=Fdgfdhge | 16:39 |
| *** markusry has joined #openstack-infra | 16:39 | |
| anteaya | fungi: that is impressive | 16:40 |
| * fungi is trying to make this look exciting, just like whitewashing a picket fence | 16:40 | |
| *** yolanda has joined #openstack-infra | 16:40 | |
| anteaya | let's hope it deletes stuff | 16:40 |
| *** Guest47242 is now known as med_ | 16:41 | |
| *** med_ has joined #openstack-infra | 16:41 | |
| *** dprince has joined #openstack-infra | 16:41 | |
| anteaya | fungi: that is a great looking picket fence | 16:41 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: WIP: Rework timeline events into a timeline https://review.openstack.org/327231 | 16:41 |
| dhellmann | jeblair : the current numbers are about as high as I remember ever seeing, but I really only watch it when I'm tagging releases so I don't know if that's a good sample. :-) | 16:41 |
| fungi | anteaya: well, this is just the "gimme a list of the files uploaded by this user" part | 16:41 |
| *** mdrabe has quit IRC | 16:41 | |
| * anteaya looks in her pocket for an apple | 16:41 | |
| anteaya | fungi: yup | 16:41 |
| fungi | anteaya: still need to feed it into a set of delete calls | 16:41 |
| krotscheck | pabelanger: I know this is probably coming at the problem form a completely different angle, but couldn't the image builds themselves be periodic jobs? | 16:41 |
| *** bhavik has joined #openstack-infra | 16:41 | |
| fungi | so i'm working on how to delete a file through the api next | 16:41 |
| *** amoralej is now known as amoralej|off | 16:41 | |
| jeblair | dhellmann: ~20k is i think the record | 16:41 |
| anteaya | fungi: go you! | 16:41 |
| * krotscheck expects there'd be some chicken-egg issues there. | 16:42 | |
| *** rcernin has quit IRC | 16:42 | |
| *** fawadkhaliq has joined #openstack-infra | 16:42 | |
| anteaya | jeblair: is it really? I must have missed that day | 16:42 |
| *** fawadkhaliq has quit IRC | 16:42 | |
| *** esikachev has quit IRC | 16:42 | |
| dhellmann | jeblair : I'll settle for 2.5k | 16:42 |
| jeblair | dhellmann: we'll work on improving the cache in v3 | 16:42 |
| *** fawadkhaliq has joined #openstack-infra | 16:42 | |
| pabelanger | krotscheck: You mean use jenkins to do the builds? | 16:42 |
| *** oanson has quit IRC | 16:42 | |
| pabelanger | krotscheck: trying to understand the issue you are wanting to address | 16:42 |
| dhellmann | jeblair: yeah, no worries, I have other things to do but was just trying to refresh my memory of what was actually going on | 16:42 |
| krotscheck | pabelanger: I'm bikeshedding, ignore me. | 16:43 |
| anteaya | ah the cinder meeting is over, I'll share this fine page of review stats that smcginnis linked to in the cinder meeting: http://cinderstats-dellstorage.rhcloud.com/cinder-openreviews.html | 16:43 |
| anteaya | I figured you'd all want to see | 16:43 |
| krotscheck | pabelanger: I just figure that you have a thing, that needs to be done periodically, and that needs to report some progress, and it seems like we've got all those tools in zuul/jenkins already. | 16:43 |
| anteaya | smcginnis: nice stats page there | 16:43 |
| anteaya | smcginnis: I like it | 16:43 |
| pabelanger | krotscheck: there is an issue about how long it takes to bring our images online. Right now that's about 11 hours. Something that happens today, which I want to fix, is we only upload images after all DIBs are built. VS, start uploads a DIB as soon as it has finished building. | 16:44 |
| pabelanger | krotscheck: interesting idea | 16:44 |
| krotscheck | pabelanger: Not unlike the periodic mirror build. | 16:45 |
| *** hrybacki is now known as hrybacki|food | 16:45 | |
| pabelanger | krotscheck: the man issue for that, is we do cache a fair bit of things on nodepool.o.o, so moving things to a periodic job means we lose that case (since the node is destroyed) | 16:45 |
| krotscheck | pabelanger: So it'd be slower. | 16:46 |
| pabelanger | krotscheck: I am sure there is some optimization that could be made to the progress moving forward however | 16:46 |
| krotscheck | pabelanger: Like adding particular git repositories to our mirrors? ;) | 16:46 |
| * krotscheck wonders if that might be a little overkill | 16:46 | |
| pabelanger | krotscheck: I'd have to look into it more. Today we don't actually use any of our mirror infra for DIBs, we rely completely on upstream repos. | 16:47 |
| pabelanger | for centos-7 already, we could switch to our mirros, since they are still signed packages | 16:47 |
| *** sfinucan has quit IRC | 16:47 | |
| pabelanger | put again, everything is mostly cached to filesystem | 16:48 |
| pabelanger | and reused each day | 16:48 |
| *** lucasagomes is now known as lucas-dinner | 16:48 | |
| krotscheck | pabelanger: There are persistent slaves in infra already, like the release slave. Creating one to persist that cache is possible. | 16:48 |
| *** d0ugal has joined #openstack-infra | 16:49 | |
| *** tphummel has joined #openstack-infra | 16:50 | |
| pabelanger | krotscheck: agreed. At one time I thought about running a nodepool-builder per cloud region, in an effort to reduce our upload times. clarkb pointed out, rightfully, each of our images would potentially be different since build times would not be atomic across regions. | 16:51 |
| *** nwkarsten has quit IRC | 16:52 | |
| pabelanger | krotscheck: But back to persistent slaves, I can't think of a reason not to do it, aside from the fact we already do it on nodepool.o.o :) But I do want to create nodepool-builder.o.o to create some isolation from nodepool.o.o, so could be an interesting discussion | 16:52 |
| *** javeriak has joined #openstack-infra | 16:52 | |
| JayF | https://review.openstack.org/#/c/316662 (devstack-gate) is the only patch left for Ironic to get grenade passing. It already has 8x+1s from Ironic cores | 16:53 |
| JayF | reviews appreciated for obvious reasons :D | 16:53 |
| fungi | okay, here's a slightly better query... https://wiki.openstack.org/w/api.php?action=query&list=allimages&ailimit=max&aiprop=user|timestamp|url&aisort=timestamp&aiuser=Fdgfdhge | 16:54 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/project-config: Skip puppet tempest jobs on tempest if no functional change https://review.openstack.org/327243 | 16:54 |
| jeblair | anteaya, dhellmann: it looks like it takes about 1 minute for a merged nova change to show up in the replicated repositories; or at least that's how long zuul is waiting for it (inefficiencies processing a 5MB git refs advertisement may be contributing as well) | 16:54 |
| *** Swami has joined #openstack-infra | 16:54 | |
| *** mdrabe has joined #openstack-infra | 16:54 | |
| mtreinish | anteaya, AJaeger, fungi: ^^^ that should be pretty quick (assuming iI did the regex correctly) | 16:54 |
| fungi | jeblair: i wonder if the ever increasing size of the nova repo on disk plays a role in that slowness at all | 16:54 |
| jeblair | anteaya, dhellmann: zuul completely pauses during that, so merging X nova changes means a complete pause of X minutes. | 16:55 |
| jeblair | fungi: quite possibly | 16:55 |
| jeblair | i am very eager to have the garbage collection | 16:55 |
| dhellmann | jeblair : a minute doesn't seem long, but it sounds like it adds up | 16:55 |
| *** asettle has quit IRC | 16:56 | |
| *** PollyZ has quit IRC | 16:56 | |
| dhellmann | this release patch I'm waiting for says it has been in the gate queue for 1 hr 11 min, with both tests passed after probably 15 minutes between waiting to be started and then running the jobs | 16:56 |
| anteaya | jeblair: thank you | 16:56 |
| *** weshay_brb is now known as weshay | 16:57 | |
| *** daemontool has quit IRC | 16:57 | |
| anteaya | mtreinish: I always assume you did the regex correctly | 16:57 |
| pleia2 | whee, first call of the day behind me | 16:57 |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Add timeout to deploy.sh https://review.openstack.org/326634 | 16:57 |
| mtreinish | anteaya: haha, I don't. I always get the regex wrong :) | 16:57 |
| anteaya | pleia2: woooo | 16:57 |
| pleia2 | anteaya: re: SB liason for infra, happy to do this | 16:58 |
| jeblair | it only takes about 10 seconds to query for open changes | 16:58 |
| anteaya | mtreinish: oh, yes, that's what I meant | 16:58 |
| fungi | pleia2: thanks! | 16:58 |
| anteaya | pleia2: wonderful, thank you | 16:58 |
| *** thorst_ has joined #openstack-infra | 16:58 | |
| anteaya | pleia2: this is what you are agreeing to: "Feature stakeholders identify remaining needed features" | 16:58 |
| anteaya | pleia2: so keen to have all your input in what you notice | 16:59 |
| *** psilvad has quit IRC | 16:59 | |
| anteaya | but would like to have a very small list of we need this to migrate | 16:59 |
| *** sorantis has quit IRC | 16:59 | |
| *** PollyZ has joined #openstack-infra | 16:59 | |
| *** shamail has left #openstack-infra | 17:00 | |
| fungi | worth noting, the list for infra could be vanishingly small as we are already migrated to sb | 17:00 |
| pleia2 | anteaya: yep, I read up earlier this morning, I'll start collecting our previous feedback from our last sprint in a more organized way asap | 17:01 |
| fungi | but we still might be aware of gaps which would prevent other teams from migrating which they might not themselves think to identify | 17:01 |
| pleia2 | anteaya: thanks :) | 17:01 |
| *** ayoung has quit IRC | 17:01 | |
| anteaya | pleia2: awesome thank you | 17:01 |
| pleia2 | fungi: yeah, we did find a few pain points when we did our bug day, so I'll make sure those rise to the top of our list | 17:01 |
| anteaya | pleia2: I added you to the notes for the task "identify target users for this process" on https://storyboard.openstack.org/#!/story/2000610 | 17:01 |
| pleia2 | anteaya: thanks :) | 17:02 |
| anteaya | fungi: agreed | 17:02 |
| anteaya | pleia2: awesome thank you | 17:02 |
| anteaya | also did you see storyboard is having a bug sprint | 17:02 |
| anteaya | https://wiki.openstack.org/wiki/VirtualSprints#StoryBoard_Bug_Sprint | 17:03 |
| anteaya | if you have some time in that window would love to have your help | 17:03 |
| *** javeriak_ has joined #openstack-infra | 17:04 | |
| *** javeriak has quit IRC | 17:04 | |
| *** PollyZ has quit IRC | 17:04 | |
| *** cody-somerville has joined #openstack-infra | 17:04 | |
| *** tonytan4ever has quit IRC | 17:04 | |
| *** deadnull_ has joined #openstack-infra | 17:05 | |
| *** thorst_ has quit IRC | 17:05 | |
| openstackgerrit | Jay Faulkner proposed openstack-infra/project-config: Upgrade Ironic grenade jobs to -nv in check queue https://review.openstack.org/327246 | 17:05 |
| openstackgerrit | Anita Kuno proposed openstack-infra/storyboard-webclient: Add a cancel button to the Add Tag text box https://review.openstack.org/325904 | 17:07 |
| *** xarses has joined #openstack-infra | 17:07 | |
| *** derekh has quit IRC | 17:08 | |
| odyssey4me | pabelanger Are the UCA packages mirrored in OpenStack-Infra? | 17:08 |
| pleia2 | anteaya: I had not, added to my calendar | 17:09 |
| anteaya | pleia2: awesome thank you | 17:09 |
| pabelanger | odyssey4me: currently no | 17:09 |
| *** shashank_hegde has joined #openstack-infra | 17:09 | |
| *** hashar has quit IRC | 17:09 | |
| *** jerryz has joined #openstack-infra | 17:10 | |
| *** sdake has joined #openstack-infra | 17:10 | |
| *** yamahata has joined #openstack-infra | 17:11 | |
| *** Rockyg has quit IRC | 17:12 | |
| *** bpokorny has joined #openstack-infra | 17:12 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/tripleo-ci: Add 15mins to OVERCLOUD_DEPLOY timeout https://review.openstack.org/327250 | 17:12 |
| *** cody-somerville has quit IRC | 17:12 | |
| odyssey4me | pabelanger bummer, that'd be useful | 17:12 |
| *** johnny_ has joined #openstack-infra | 17:12 | |
| * odyssey4me adds it to the ever-growing TODO list | 17:12 | |
| pabelanger | odyssey4me: I don't see a reason not to do it | 17:12 |
| anteaya | I never realized before how lovely it is to view all the stories that have patches ready to review in this view: https://storyboard.openstack.org/#!/dashboard/stories | 17:13 |
| *** javeriak has joined #openstack-infra | 17:15 | |
| *** rockstar has joined #openstack-infra | 17:17 | |
| *** thorst_ has joined #openstack-infra | 17:17 | |
| *** javeriak_ has quit IRC | 17:18 | |
| dhellmann | fungi : I have a test for reno that demonstrates that merging tags from a stable branch into master confuses its sense of what version applies to a release note. git treats the merged version number as "current" for "git describe" output, too. So before I make reno do something special, I wanted to discuss why we're merging tags from branches into master again. We talked about it at the summit, but my notes don't include that info. Do you | 17:18 |
| dhellmann | have some time for that today? | 17:18 |
| *** nadya has quit IRC | 17:18 | |
| fungi | dhellmann: sure, i can rediscuss it now | 17:18 |
| dhellmann | fungi : great, thanks for letting me interrupt your day. So, I know we only merge some tags, and I think it had something to do with ensuring that the final release version from a pre-release branch made it into master so that master would "have" that same version. Is that the only reason? | 17:19 |
| *** vhosakot has joined #openstack-infra | 17:20 | |
| *** trown|lunch is now known as trown | 17:20 | |
| fungi | dhellmann: it should _only_ be done for projects which tag mainline releases on non-master branches, e.g. we branch stable/newton at rc1 and then tag the first newton release there, we merge that tag back into the master branch so that ocata development shows a version higher than newton | 17:20 |
| dhellmann | ok, I think we're saying the same thing | 17:20 |
| fungi | dhellmann: that should be the _only_ situation where we perform that tag merge. if we're doing it elsewhere we need to figure out why | 17:20 |
| dhellmann | so, why do we need to do that? do you remember if there was a real failure, or was this a preemptive choice? | 17:21 |
| dhellmann | fungi : well, it turns out it doesn't matter -- doing it at all makes reno produce the "wrong" results | 17:21 |
| rockstar | Hi infra- can someone please add me to nova-lxd-core and nova-lxd-release groups? The import is complete. | 17:21 |
| fungi | dhellmann: because for example tarballs generated on master after a major release were still claiming to be a lower version | 17:21 |
| dhellmann | so I'm leaning towards proposing we stop doing it at all, but before I go that far I want to make sure I understand why we're doing it | 17:22 |
| *** dprince has quit IRC | 17:22 | |
| *** thorst_ has quit IRC | 17:22 | |
| *** shashank_hegde has quit IRC | 17:22 | |
| *** sorantis has joined #openstack-infra | 17:22 | |
| *** dprince has joined #openstack-infra | 17:23 | |
| fungi | dhellmann: i think you would need to immediate tag a "start of cycle" development milestone on master when you release from the stable branch as an alternative | 17:23 |
| dhellmann | so those are tarballs created after a merge into master, and the version *inside* the tarball is lower than the stable branch? | 17:23 |
| *** nadya has joined #openstack-infra | 17:23 | |
| *** tonytan4ever has joined #openstack-infra | 17:23 | |
| *** kien-ha has joined #openstack-infra | 17:23 | |
| fungi | dhellmann: correct | 17:23 |
| fungi | dhellmann: because the master branch does not have the most recent release tag in its history | 17:23 |
| dhellmann | what breaks because that version is "wrong"? | 17:23 |
| fungi | dhellmann: mostly people's perceptions | 17:24 |
| *** electrofelix has quit IRC | 17:24 | |
| dhellmann | ok | 17:24 |
| dhellmann | so we either have a lotta splainin to do, or we need to do something else to raise that version after a branch is created | 17:25 |
| dhellmann | "it's ok, really, your version will be fixed at milestone 1" or we tag an X.0.0.0a0 or something right after the branch is created | 17:25 |
| fungi | dhellmann: so if we hadn't done that tag merge for mitaka, nova would claim to be 13.0.0.rc1.devXXXX until 14.0.0.0b1 was tagged 3 months later | 17:26 |
| dhellmann | yeah, that's a long time to go with the apparently wrong version | 17:26 |
| *** itisha has joined #openstack-infra | 17:26 | |
| fungi | dhellmann: basically half a cycle where your master branch looked like it was "older" than your last release | 17:26 |
| *** asettle has joined #openstack-infra | 17:26 | |
| *** PollyZ has joined #openstack-infra | 17:26 | |
| dhellmann | I'll spend some time on it, but I'm not sure I'm going to be able to teach reno to ignore those tags. git doesn't -- git describe incorporates them -- so reno will act differently from git | 17:27 |
| dhellmann | how do you feel about changing the release process to add the extra alpha/beta tag right after a branch? | 17:27 |
| openstackgerrit | Joe Talerico proposed openstack-infra/project-config: Adding Browbeat https://review.openstack.org/327255 | 17:28 |
| fungi | dhellmann: right, we want git describe, pbr.version, et cetera to see that tag so they report a version newer than the release | 17:28 |
| dhellmann | basically, change our process and stop "fooling around" with git | 17:28 |
| dhellmann | well, we want those tools to see *a* tag. I'm not sure we want them to see the one we applied to the branch | 17:28 |
| dhellmann | now that we're using post versioning, we may just need to add an extra tag to our process | 17:29 |
| fungi | dhellmann: ideally the first divergent commit from stable/newton for nova would have 15.0.0.0a1 tagged, for example | 17:29 |
| dhellmann | right, that's what I think we ought to do | 17:29 |
| smcginnis | anteaya: Thanks! | 17:29 |
| *** thorst_ has joined #openstack-infra | 17:29 | |
| *** thorst_ has quit IRC | 17:29 | |
| dhellmann | fungi : on master | 17:30 |
| fungi | dhellmann: but you'd need to be careful not to tag a commit which also appears in stable. it would need to be a divergent commit | 17:30 |
| *** dimtruck is now known as zz_dimtruck | 17:30 | |
| fungi | yeah, on master | 17:30 |
| openstackgerrit | Augustina Ragwitz proposed openstack-infra/project-config: WIP: Add grenade test for Nova simple-cells-setup migration https://review.openstack.org/326105 | 17:30 |
| dhellmann | so we tag X.0.0.0rc1, create the stable/$SERIES branch from that, commit something on master, tag that commit something like X+1.0.0.0a1 | 17:30 |
| fungi | a commit on the master branch which is _not_ in the history of any stable branch | 17:30 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Create the ARA project https://review.openstack.org/321226 | 17:30 |
| dhellmann | fungi : we don't merge commits between branches, right? and it would be safe to tag something that had been cherry-picked? | 17:31 |
| fungi | dhellmann: right, it's the "commit something on master" part which worries me | 17:31 |
| *** PollyZ has quit IRC | 17:31 | |
| *** IlyaG has joined #openstack-infra | 17:31 | |
| fungi | if changes from master are backported as cherry-picks it would be safe. if changes from master are backported as a fast-forward then it would be a problem | 17:32 |
| anteaya | smcginnis: can I peek at the script? | 17:32 |
| *** e0ne has joined #openstack-infra | 17:32 | |
| *** IlyaG has quit IRC | 17:32 | |
| dmsimard | Weird error around puppet-congress on a (totally unrelated to congress) review http://logs.openstack.org/26/321226/8/check/gate-project-config-projects-yaml/ad368c5/console.html#_2016-06-08_14_49_10_309 | 17:32 |
| dhellmann | is it even possible, given the way we're set up, to backport using fast-forward? | 17:32 |
| dhellmann | I have only even seen us use cherry-picks | 17:32 |
| smcginnis | anteaya: I actually adapted that from somewhere. Let me see if I can find the actual source. | 17:32 |
| openstackgerrit | Augustina Ragwitz proposed openstack-infra/project-config: WIP: Add grenade test for Nova simple-cells-setup migration https://review.openstack.org/326105 | 17:32 |
| smcginnis | anteaya: I don't want to take all the credit for someone elses work. ;) | 17:32 |
| *** degorenko is now known as _degorenko|afk | 17:32 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config: Create the ARA project https://review.openstack.org/321226 | 17:32 |
| anteaya | smcginnis: that's what I like about you | 17:32 |
| *** yamahata has quit IRC | 17:33 | |
| fungi | dhellmann: i think you may be able to propose a change to another branch. i know at least that merge commits depending on commits in another branch will drag them in | 17:33 |
| *** zz_dimtruck is now known as dimtruck | 17:33 | |
| dhellmann | fungi : I think you're right. I think I've actually seen someone do that accidentally, in fact. | 17:33 |
| *** yamahata has joined #openstack-infra | 17:33 | |
| *** rbrndt has quit IRC | 17:33 | |
| fungi | dhellmann: maybe an update should be made to the release notes as the first new commit on master? though making sure to commit something on master takes us back to the old pain of remembering to merge a commit updating pre-versioning | 17:33 |
| openstackgerrit | Augustina Ragwitz proposed openstack-infra/project-config: WIP: Add grenade test for Nova simple-cells-setup migration https://review.openstack.org/326105 | 17:34 |
| dhellmann | yeah, and master may already be farther ahead than the point where the branch should be created | 17:34 |
| AJaeger | jeblair: I'm happy to add some files section back again! | 17:34 |
| smcginnis | anteaya: OK, been a while since I set that up so I had to shake out some cobwebs. :) | 17:34 |
| anteaya | smcginnis: sounds like my house | 17:35 |
| smcginnis | anteaya: I'm pulling together stuff from here: https://github.com/openstack-infra/reviewstats | 17:35 |
| smcginnis | anteaya: And here https://github.com/openstack-infra/bugdaystats | 17:35 |
| anteaya | awesome | 17:35 |
| smcginnis | anteaya: Hah, same. :) | 17:35 |
| anteaya | smcginnis: feel like offering a patch to either repo with your work? | 17:35 |
| anteaya | I like your choices | 17:35 |
| dhellmann | fungi : we have the problem now, don't we, of needing to not merge changes from master back to stable to avoid having a newer version introduced on the stable branch? If I did that today to nova, I would break versioning in their stable branch. | 17:36 |
| smcginnis | anteaya: Yeah, I should look. I remember needing to hack some things, but not sure how far off I am from the source. | 17:36 |
| smcginnis | anteaya: I'll get that in my backlog to take a look at. | 17:36 |
| pabelanger | AJaeger: mind helping 327189 land? | 17:36 |
| fungi | dhellmann: yeah, after the first milestone tag, correct | 17:36 |
| smcginnis | anteaya: I agree, if I did anything useful it would be good to get up there. | 17:36 |
| anteaya | smcginnis: I think you did something useful | 17:37 |
| dhellmann | fungi : so technically the change in process doesn't introduce any new danger, though it does reduce the safe window period | 17:37 |
| smcginnis | anteaya: Thanks, I'll try to remember to ping you if/when I post anything. :) | 17:38 |
| fungi | dhellmann: anyway, if you want to do away with and replace the current solution, i'd recommend a quick sync up with mordred and lifeless_ about it as well in case they can think of other edge cases this is protecting us from which the proposed alternative does not | 17:38 |
| anteaya | smcginnis: awesome thank you | 17:38 |
| dhellmann | fungi : yeah, my next step was going to be a ML post. I'll do that this week. thanks for talking through things with me. | 17:38 |
| fungi | dhellmann: any time! | 17:38 |
| *** sputnik13 has joined #openstack-infra | 17:39 | |
| JayF | fungi: mtreinish: jeblair: if one of you have a minute, https://review.openstack.org/#/c/316662/ is a one-line patch that is all that's needed to get grenade passing on Ironic. We'd love to get it merged today as this has been our primary focus this cycle so far. | 17:40 |
| pabelanger | dmsimard: does 321226 need sudo permissions for your gate jobs? | 17:40 |
| dmsimard | pabelanger: it didn't but now it does | 17:40 |
| pabelanger | okay | 17:40 |
| dmsimard | pabelanger: ansible integration tests will do things like install and uninstall packages | 17:40 |
| pabelanger | Yup, makes sense | 17:41 |
| dmsimard | thanks for looking, hope we can merge it soon :) | 17:41 |
| openstackgerrit | Joe Talerico proposed openstack-infra/project-config: Adding Browbeat https://review.openstack.org/327255 | 17:41 |
| *** nwkarsten has joined #openstack-infra | 17:42 | |
| *** nwkarsten has quit IRC | 17:43 | |
| *** e0ne has quit IRC | 17:43 | |
| *** nwkarsten has joined #openstack-infra | 17:43 | |
| *** hashar has joined #openstack-infra | 17:43 | |
| *** hashar is now known as hasharAway | 17:44 | |
| *** kzaitsev_mb has quit IRC | 17:44 | |
| *** nadya has quit IRC | 17:45 | |
| AJaeger | pabelanger: +2A | 17:46 |
| AJaeger | pabelanger: +2A on 327189 I mean... | 17:46 |
| *** sorantis has quit IRC | 17:46 | |
| *** abregman has joined #openstack-infra | 17:47 | |
| pabelanger | AJaeger: danke | 17:47 |
| *** ihrachys has quit IRC | 17:47 | |
| *** asettle has quit IRC | 17:49 | |
| *** tkelsey has quit IRC | 17:49 | |
| jeblair | fungi, pabelanger, notmorgan, Shrews: i think a contributing factor to the zuul slowness is that geard is running about at capacity now. zl01 does not register a set_description function, which means that zuul is constantly asking geard for the full function list to see if it has shown up. (we cache the function list; cache misses trigger a refresh every 5 seconds). that is exacerbating the load on geard and pausing zuul for long ... | 17:49 |
| jeblair | ... periods of time waiting for a response. | 17:49 |
| notmorgan | jeblair: oh fun | 17:49 |
| jeblair | we may *also* have issues with either too many functions, or sending too much statsd data, but i think that's probably being masked by the constant status calls | 17:49 |
| notmorgan | jeblair: i can see that | 17:50 |
| jeblair | so i think we need to fix that first, then look at the other | 17:50 |
| *** tzn has quit IRC | 17:50 | |
| *** shashank_hegde has joined #openstack-infra | 17:50 | |
| *** hrybacki|food is now known as hrybacki | 17:50 | |
| *** e0ne has joined #openstack-infra | 17:50 | |
| jeblair | we could either make a zuul config option to stop using 'describe_job', or we could have zuul-launcher register that function and then noop | 17:50 |
| jeblair | the second is probably easier | 17:51 |
| AJaeger | team, I saw earler failing jobs in the post queue - but no log files. Links are http://logs.openstack.org/f2/f20de1013003414a35abd22a31516921a9d8d202/post/openstack-manuals-releasenotes/ff790b9/ and http://logs.openstack.org/39/392bdecc981ce1f07afb91f4927938d570692340/post/openstack-manuals-releasenotes/7e4bb6d/ . Any ideas? | 17:51 |
| *** sdake has quit IRC | 17:51 | |
| notmorgan | jeblair: i would be inclined to noop immediately | 17:52 |
| notmorgan | jeblair: since it seems like a shorter path to "less problems" | 17:52 |
| anteaya | AJaeger: I confirm I cannot see any files at those urls | 17:52 |
| pabelanger | jeblair: notmorgan: I too like the noop | 17:52 |
| *** DevBox has joined #openstack-infra | 17:52 | |
| anteaya | AJaeger: I have no ideas | 17:52 |
| pabelanger | would moving geard off zuul.o.o help? | 17:53 |
| *** sambetts is now known as sambetts|afk | 17:53 | |
| pabelanger | not that it is easier | 17:53 |
| *** deadnull_ is now known as _deadnull | 17:53 | |
| *** salv-orl_ has quit IRC | 17:53 | |
| *** banix has joined #openstack-infra | 17:53 | |
| jeblair | pabelanger: no -- it's GIL bound, we're underutilizing the cpu on zuul.o.o | 17:53 |
| *** salv-orlando has joined #openstack-infra | 17:53 | |
| jeblair | hrm, also, geard seems to be having an issue with nodepool right now | 17:53 |
| *** e0ne has quit IRC | 17:53 | |
| jeblair | [pid 3360] sendto(423, "\t0\t0\t423\n", 9, 0, NULL, 0) = -1 EAGAIN (Resource temporarily unavailable) | 17:53 |
| jeblair | zuul-serv 3359 zuul 423u IPv4 62208500 0t0 TCP zuul.openstack.org:4730->nodepool.openstack.org:55043 (CLOSE_WAIT) | 17:53 |
| Shrews | jeblair: what's the risk of the noop? | 17:54 |
| jeblair | it's looping over that | 17:54 |
| *** PollyZ has joined #openstack-infra | 17:54 | |
| jeblair | Shrews: none i can think of. | 17:54 |
| *** sputnik13 has quit IRC | 17:54 | |
| notmorgan | jeblair: this, i think, is where we might benefit from asyncio+uvloop, but that is a big change(tm), and likely a lot less relevant in zuulv3 | 17:54 |
| jeblair | notmorgan: i very seriously doubt it, but i'd love to have that conversation some other time when systems are not on fire. :) (plus, you are right it is less relevant in v3) | 17:56 |
| notmorgan | jeblair: like i said :) | 17:56 |
| *** _deadnull is now known as deadnull_ | 17:57 | |
| jeblair | i think we need to restart zuul to get gearman running | 17:57 |
| openstackgerrit | Andrey Nikitin proposed openstack-infra/puppet-gerrit: Order of the classes parameters is refactored https://review.openstack.org/294728 | 17:57 |
| jeblair | and then i think i need to not send any extra status commands to it which might push it over the edge | 17:57 |
| *** sputnik13 has joined #openstack-infra | 17:58 | |
| jeblair | which may be what killed the nodepool connection this time | 17:58 |
| jeblair | (regardless, there is some kind of a bug with geard not handling that disconnection gracefully) | 17:58 |
| pabelanger | looking at zuul to see where we'd add the noop for set_description | 17:58 |
| *** PollyZ has quit IRC | 17:59 | |
| jeblair | stopping zuul | 17:59 |
| notmorgan | jeblair: ok. | 17:59 |
| notmorgan | i'll take a closer look at geard, i've been spending a bunch of time in it's code lately, if something stands out as an issue with disconnection, will let you know | 18:00 |
| jeblair | notmorgan: from strace, it looks like it was in the middle of trying to send a response to a 'status' command | 18:00 |
| mordred | fungi, dhellmann: heya - just got back online - quick tl;dr of what I missed? or should I just read the backscroll? | 18:01 |
| jeblair | notmorgan: oh, hrm. you can see from the line i pasted above, we were just getting EAGAIN over and over | 18:01 |
| notmorgan | jeblair: i'm wondering if this is just related to the oddness in sockets i've been seeing. | 18:01 |
| jeblair | notmorgan: on a connection that lsof says was in CLOSE_WAIT | 18:01 |
| jeblair | notmorgan: that doesn't make sense to me | 18:01 |
| *** signed8b_ is now known as signed8bit_Zzz | 18:02 | |
| fungi | mordred: the tag merge job confuses reno, and dhellmann is looking for alternative solutions instead of merging the first release on a stable branch back into master | 18:02 |
| dhellmann | mordred, fungi : https://etherpad.openstack.org/p/TzVUvEeYjB | 18:02 |
| notmorgan | jeblair: weird. | 18:02 |
| dhellmann | that's my email draft | 18:02 |
| openstackgerrit | Sean McGinnis proposed openstack-infra/project-config: Skip release notes jobs for pre-Mitaka os-brick and python-cinderclient https://review.openstack.org/327264 | 18:02 |
| jeblair | also, this is the first time we've hard-stopped zuul with zl01 running. it did not magically stop all of its jobs like i expected, but then again, there was weird geard stuff going on | 18:02 |
| jeblair | so i am stopping zl01 as well, and we should test zl with zuul hard stops | 18:03 |
| notmorgan | jeblair: we might need to handle a SIGPIPE | 18:03 |
| mordred | fungi, dhellmann: the merge is actually only needed for pre-versioning | 18:03 |
| notmorgan | jeblair: i'll poke at this a bit more. | 18:03 |
| mordred | for post versioning, it shouldn't matter, iirc | 18:04 |
| clarkb | mordred: depends on where branchung happens more | 18:04 |
| jeblair | notmorgan, Shrews, pabelanger: i'll leave zl stopped so we don't incur the set_description penalty | 18:04 |
| *** pvaneck has joined #openstack-infra | 18:04 | |
| clarkb | mordred: uf we always branch before ragging thwn need it | 18:05 |
| clarkb | wow phone typing | 18:05 |
| notmorgan | jeblair: wgm. | 18:05 |
| dhellmann | clarkb : we always branch from tagged versions | 18:05 |
| notmorgan | jeblair: wfm. | 18:05 |
| AJaeger | dhellmann: how do you want to wait for the next change to merge? Some projects might not be that quick with merging - like django_openstack_auth. Would you try merging a dummy commit? | 18:05 |
| dhellmann | clarkb: at least for stable branches | 18:05 |
| fungi | dhellmann: reno is only _actually_ being used to update release notes in release jobs though, right? if so, those would never be triggered for the tag merge into master anyway | 18:05 |
| dhellmann | AJaeger : this only applies to projects using cycle-with-milestone style versions | 18:06 |
| fungi | dhellmann: or is the concern that additions in master before the tag is merged are assumed by the eventual next release tag to have been added in a prior release? | 18:06 |
| AJaeger | dhellmann: so my example was bad;) Still, do we have projects that might not merge something for a week? | 18:06 |
| dhellmann | fungi : reno runs on every merge so we can get the "unreleased" release notes, too. But it doesn't really matter, because if it runs at any point after a tag has merged, then it will miscategorize notes from before the merge. | 18:07 |
| *** dtardivel has quit IRC | 18:07 | |
| fungi | got it | 18:07 |
| dhellmann | AJaeger : that's fine. We can either introduce a little commit just for the tag, or we can leave the tag until they have a commit. | 18:07 |
| AJaeger | dhellmann: if you leave the tag, it's busy polling basically on your side.... | 18:07 |
| AJaeger | dhellmann: might not be an issue - and we could wait whether it really hits us... | 18:08 |
| jeblair | status notice Zuul has been restarted to correct an error condition. Events since 17:30 may have been missed; please 'recheck' your changes if they were uploaded since then, or have "NOT_REGISTERED" errors. | 18:08 |
| dhellmann | AJaeger : yeah, I'm inclined to introduce something at the point of the branch if it's needed, just so we don't forget | 18:09 |
| jeblair | how does that look? ^ | 18:09 |
| *** signed8bit_Zzz is now known as signed8b_ | 18:09 | |
| fungi | jeblair: lgtm | 18:09 |
| jeblair | #status notice Zuul has been restarted to correct an error condition. Events since 17:30 may have been missed; please 'recheck' your changes if they were uploaded since then, or have "NOT_REGISTERED" errors. | 18:09 |
| openstackstatus | jeblair: sending notice | 18:09 |
| dhellmann | mordred, fungi, AJaeger : I'm going to move this to the -dev list so we have a good record of it. I think I've incorporated everything you've said into issues/alternatives, but please follow up if I've missed something | 18:10 |
| *** asselin has quit IRC | 18:10 | |
| fungi | dhellmann: yep, draft looks good | 18:10 |
| -openstackstatus- NOTICE: Zuul has been restarted to correct an error condition. Events since 17:30 may have been missed; please 'recheck' your changes if they were uploaded since then, or have "NOT_REGISTERED" errors. | 18:10 | |
| AJaeger | dhellmann: go ahead, thanks! | 18:11 |
| openstackstatus | jeblair: finished sending notice | 18:12 |
| *** oanson has joined #openstack-infra | 18:13 | |
| *** cloudtrainme has quit IRC | 18:13 | |
| dhellmann | mordred , fungi , AJaeger : http://lists.openstack.org/pipermail/openstack-dev/2016-June/096895.html | 18:14 |
| *** Sukhdev has joined #openstack-infra | 18:16 | |
| *** mriedem1 has joined #openstack-infra | 18:17 | |
| *** dizquierdo has quit IRC | 18:17 | |
| *** mriedem has quit IRC | 18:18 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Fix referenced before assignment for BuildCompletedEvent https://review.openstack.org/327274 | 18:18 |
| pabelanger | jeblair: noticed that while looking in debug.log ^ searching for the set_description problem | 18:19 |
| jeblair | pabelanger: wow, thanks. i love python. | 18:20 |
| *** javeriak_ has joined #openstack-infra | 18:21 | |
| pabelanger | jeblair: there are a few more exception in debug.log, but will loop back to them | 18:21 |
| *** PollyZ has joined #openstack-infra | 18:21 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove puppet-congress upstream url https://review.openstack.org/327172 | 18:21 |
| *** asselin has joined #openstack-infra | 18:21 | |
| pabelanger | jeblair: having troubling seeing how to add the noop for set_descriptions, not sure I'll be help to get a patch up | 18:22 |
| *** javeriak has quit IRC | 18:22 | |
| fungi | doing authenticated mediawiki api calls with openid credentials is exceedingly complex. anybody know how we set up the account for statusbot? i'm not finding details in our documentation | 18:25 |
| jeblair | fungi: i think ryan did something that i never completely understood | 18:25 |
| pabelanger | fungi: we use a username / password IRC | 18:25 |
| fungi | jeblair: i was worried that was the case. i'll keep looking | 18:26 |
| fungi | pabelanger: right, we do. trying to find out how to set a password and make password auth work for an account | 18:26 |
| *** mriedem1 is now known as mriedem | 18:26 | |
| *** tzn has joined #openstack-infra | 18:26 | |
| *** jerryz has quit IRC | 18:26 | |
| fungi | maybe if i just add my account to the bots group it will allow my newly reset password | 18:27 |
| *** tzn has quit IRC | 18:27 | |
| *** tzn has joined #openstack-infra | 18:27 | |
| fungi | nope | 18:27 |
| *** ayoung has joined #openstack-infra | 18:28 | |
| pabelanger | fungi: quick look at puppet-statusbot show we are using https://wiki.openstack.org/w/api.php URL | 18:28 |
| pabelanger | self.wiki = simplemediawiki.MediaWiki(self.url); self.wiki.login(user, pass) | 18:28 |
| fungi | pabelanger: yeah, i'm actually copying python directly from statusbot | 18:28 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: register a noop set_description job https://review.openstack.org/327277 | 18:28 |
| jeblair | pabelanger, fungi, notmorgan: ^ | 18:28 |
| fungi | problem is wiki.login() is returning False with my account name and freshly reset password | 18:29 |
| fungi | even if i'm in the bots group | 18:29 |
| pabelanger | fungi: let see check wiki logs | 18:29 |
| *** javeriak has joined #openstack-infra | 18:29 | |
| *** sarob has joined #openstack-infra | 18:30 | |
| openstackgerrit | Goutham Pacha Ravi proposed openstack-infra/project-config: Add releasenotes job for manila-ui https://review.openstack.org/320718 | 18:30 |
| *** annegentle has joined #openstack-infra | 18:30 | |
| *** tzn has quit IRC | 18:32 | |
| pabelanger | jeblair: +2, approve when ready | 18:32 |
| *** links has joined #openstack-infra | 18:32 | |
| *** deadnull_ has quit IRC | 18:33 | |
| *** javeriak_ has quit IRC | 18:33 | |
| *** bpokorny has quit IRC | 18:33 | |
| pabelanger | fungi: don't see anything specific. Did you try using the statusbot credentials? | 18:33 |
| *** bpokorny has joined #openstack-infra | 18:33 | |
| Shrews | jeblair: oh, that's simpler than i thought. i was thinking we had to implement setBuildDescription() | 18:34 |
| pabelanger | Shrews: I would have been all day doing that too | 18:34 |
| fungi | pabelanger: yeah, i'll give statusbot's creds a try with the login call just to make sure the code's right | 18:35 |
| *** maishsk has joined #openstack-infra | 18:35 | |
| *** zeih has joined #openstack-infra | 18:35 | |
| *** deadnull_ has joined #openstack-infra | 18:36 | |
| fungi | pabelanger: that works, so for some reason my password isn't being accepted | 18:36 |
| pabelanger | fungi: wonder if it has to do with the disabled password auth, maybe if disabled, resets don't work | 18:37 |
| *** sdake has joined #openstack-infra | 18:37 | |
| fungi | possible. i guess i can try setting it directly with a maintenance script or in the db | 18:37 |
| *** javeriak has quit IRC | 18:37 | |
| *** javeriak_ has joined #openstack-infra | 18:37 | |
| pabelanger | we could also enable it for 15mins in config and have puppet revert it | 18:37 |
| *** bpokorny has quit IRC | 18:38 | |
| openstackgerrit | Merged openstack-infra/project-config: Skip upstream check if we have already imported upstream https://review.openstack.org/327189 | 18:38 |
| fungi | pabelanger: i'm going to give https://www.mediawiki.org/wiki/Manual:ChangePassword.php a shot | 18:38 |
| pabelanger | fungi: ack | 18:38 |
| *** rbrndt has joined #openstack-infra | 18:39 | |
| fungi | pabelanger: jeblair: that was it! | 18:40 |
| pabelanger | jeblair: zl01 appears to still be getting nodes assigned to it by nodepool | 18:40 |
| fungi | so, yeah, the password reset special page is nonfunctional, probably actually as a result of us sticking with openid | 18:40 |
| jeblair | pabelanger: ah, yep. so we should try to land and start it quickly | 18:40 |
| openstackgerrit | Merged openstack-infra/project-config: Remove dib-buildimage-debian-ironic post job https://review.openstack.org/326203 | 18:40 |
| *** bpokorny has joined #openstack-infra | 18:40 | |
| *** deadnull_ has quit IRC | 18:41 | |
| *** jerryz has joined #openstack-infra | 18:41 | |
| AJaeger | dims, could you toggle your +A on https://review.openstack.org/#/c/327120/ - zuul didn't see that event | 18:41 |
| pabelanger | fungi: nice | 18:41 |
| *** javeriak has joined #openstack-infra | 18:42 | |
| dims | AJaeger : ack | 18:44 |
| *** javeriak_ has quit IRC | 18:44 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Move puppet-tacker to Openstack https://review.openstack.org/327178 | 18:45 |
| dims | AJaeger : done | 18:45 |
| *** gouthamr has quit IRC | 18:45 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Document how to reset statusbot wiki password https://review.openstack.org/327282 | 18:45 |
| fungi | pabelanger: ^ | 18:45 |
| AJaeger | thanks, dims | 18:45 |
| HenryG | Am I missing something? Why did PS6 of https://review.openstack.org/325865 not go to the check queue? | 18:46 |
| *** tkelsey has joined #openstack-infra | 18:46 | |
| AJaeger | argh, gate-grenade-dsvm-multinode is still NOT_REGISTERED ;( | 18:46 |
| AJaeger | HenryG: did you get the earlier status notice? | 18:46 |
| *** piet has quit IRC | 18:47 | |
| AJaeger | HenryG: See http://wiki.openstack.org/wiki/Infrastructure_Status | 18:47 |
| pabelanger | krotscheck: heads up, we only were able to upload new images to osic-cloud1 today. nodepool was reset to address an issue with the gate | 18:47 |
| krotscheck | pabelanger: Got it. | 18:47 |
| HenryG | AJaeger: Thanks, I missed it. I did not know about that wiki -- from now on I will check it first before asking. | 18:48 |
| openstackgerrit | Augustina Ragwitz proposed openstack-infra/project-config: WIP: Add grenade test for Nova simple-cells-setup migration https://review.openstack.org/326105 | 18:48 |
| *** yolanda has quit IRC | 18:48 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Fix referenced before assignment for BuildCompletedEvent https://review.openstack.org/327274 | 18:49 |
| *** vhosakot has quit IRC | 18:50 | |
| *** mixos_ has quit IRC | 18:51 | |
| fungi | woohoo! i deleted a spam page with the api (via the simplemediawiki python module like statusbot uses) | 18:51 |
| fungi | specifically an uploaded image file page, which also makes the image 404 | 18:52 |
| *** krtaylor has quit IRC | 18:52 | |
| *** Kennan has quit IRC | 18:52 | |
| AJaeger | congrats, fungi! | 18:52 |
| *** vhosakot has joined #openstack-infra | 18:52 | |
| *** Kennan has joined #openstack-infra | 18:52 | |
| fungi | from here all i need is some glue to iterate through our spam users, generate a list of the remaining files they've uploaded, and then delete them | 18:52 |
| *** LaT0rtue has joined #openstack-infra | 18:53 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: register a noop set_description job https://review.openstack.org/327277 | 18:53 |
| *** Sukhdev has quit IRC | 18:53 | |
| pabelanger | jeblair: I guess the issue is nodepool doesn't know there are no zuul-launchers online. And is continuing to add them to the zuul target. Would a solution be to add an target offline check to gearman to nodepool? | 18:53 |
| *** cody-somerville has joined #openstack-infra | 18:53 | |
| *** cody-somerville has quit IRC | 18:53 | |
| *** cody-somerville has joined #openstack-infra | 18:53 | |
| jeblair | pabelanger: there is no way to tell | 18:53 |
| jeblair | pabelanger: the way to fix it is to push forward quickly to move off of jenknis | 18:54 |
| jeblair | pabelanger: it only matters as long as we run jenkins | 18:54 |
| *** tkelsey has quit IRC | 18:54 | |
| pabelanger | jeblair: right | 18:54 |
| pabelanger | jeblair: okay, won't spend any more time on it | 18:54 |
| sdague | ok, it looks like the lots of NOT_REGISTERED is known issue | 18:55 |
| jeblair | sdague: yeah, expect resolution in maybe 10 mins | 18:56 |
| sdague | ok, cool. Thank you | 18:56 |
| *** flepied has quit IRC | 18:56 | |
| jeblair | pabelanger: i think we should drop the registration check from zuul | 18:56 |
| jeblair | pabelanger: or make it optional | 18:56 |
| anteaya | fungi: yay | 18:57 |
| jeblair | (we should also have a dequeue command in case something slips through, but i'm not sure we can wait any longer for that, registration checks are becoming a big problem) | 18:57 |
| *** piet has joined #openstack-infra | 18:59 | |
| pabelanger | jeblair: I guess moving to zuul-launcher doesn't mean we care too much about registered jobs, since we load that when the job launches right? | 18:59 |
| dmsimard | AJaeger: Replied to your comment about ARA. Thanks. | 19:00 |
| jeblair | pabelanger: the reason we don't care is that we have a gate check on project-config that prevents us from referencing jobs that don't exist | 19:00 |
| jeblair | the registration check predates that | 19:00 |
| pabelanger | Ah, right | 19:00 |
| *** markusry has quit IRC | 19:00 | |
| openstackgerrit | Merged openstack-infra/zuul: Ansible launcher: register a noop set_description job https://review.openstack.org/327277 | 19:00 |
| *** jtomasek has joined #openstack-infra | 19:01 | |
| *** Sukhdev has joined #openstack-infra | 19:01 | |
| pabelanger | jeblair: making it optional seems like the good thing to do, so we don't break other zuul users | 19:01 |
| pabelanger | Have to zip down to school to get my daughter, be back in 30 | 19:01 |
| *** benny_ has quit IRC | 19:01 | |
| *** gouthamr has joined #openstack-infra | 19:01 | |
| AJaeger | dmsimard: +2a | 19:01 |
| jeblair | starting zl01 | 19:02 |
| *** alex_xu has quit IRC | 19:02 | |
| *** psilvad has joined #openstack-infra | 19:03 | |
| *** psilvad has quit IRC | 19:04 | |
| *** hasharAway is now known as hashar | 19:04 | |
| *** psilvad has joined #openstack-infra | 19:05 | |
| *** Apoorva has quit IRC | 19:05 | |
| *** krtaylor has joined #openstack-infra | 19:05 | |
| *** alex_xu has joined #openstack-infra | 19:06 | |
| *** claudiub has quit IRC | 19:06 | |
| jeblair | i'm deleting the idle 2-node nodes to force relaunching/registration | 19:06 |
| dmsimard | AJaeger: woo! | 19:09 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/devstack-gate: DNM playing with tempest run https://review.openstack.org/327291 | 19:10 |
| *** harlowja has quit IRC | 19:12 | |
| *** esikachev has joined #openstack-infra | 19:13 | |
| *** e0ne has joined #openstack-infra | 19:13 | |
| *** salv-orlando has quit IRC | 19:14 | |
| *** salv-orlando has joined #openstack-infra | 19:14 | |
| *** harlowja has joined #openstack-infra | 19:15 | |
| fungi | and now i've successfully deleted a couple hundred file pages for uploads by one of our more prolific spam accounts... https://wiki.openstack.org/wiki/Special:ListFiles?limit=50&ilsearch=&user=Fdgfdhge | 19:15 |
| anteaya | woooooo | 19:16 |
| fungi | now it's just a matter of introducing a loop over the remaining spam uploaders | 19:16 |
| AJaeger | Yeah! | 19:16 |
| anteaya | yay | 19:16 |
| fungi | and then i'll recheck the uploaded images list and see if i missed any spammers when i skimmed earlier | 19:16 |
| fungi | a little wax on/wax off with this and we'll be spam file free shortly | 19:17 |
| *** esikachev has quit IRC | 19:17 | |
| *** kzaitsev_mb has joined #openstack-infra | 19:19 | |
| AJaeger | mtreinish: do we need both https://review.openstack.org/#/c/324158/ and https://review.openstack.org/#/c/327243/1 ? | 19:20 |
| mtreinish | AJaeger: nope, I didn't realize oomichi had beat me to it | 19:20 |
| mtreinish | I'll abandon mine | 19:20 |
| *** javeriak has quit IRC | 19:22 | |
| *** cloudtrainme has joined #openstack-infra | 19:22 | |
| *** dizquierdo has joined #openstack-infra | 19:22 | |
| *** javeriak has joined #openstack-infra | 19:22 | |
| *** piet has quit IRC | 19:24 | |
| *** rcernin has joined #openstack-infra | 19:25 | |
| jeblair | fungi, pabelanger, notmorgan, Shrews: aha -- we have 20,000 jobs registered, but i only expect ~10k | 19:27 |
| krotscheck | 20,000.... .... ...? | 19:28 |
| jeblair | that would certainly push geard over the edge... | 19:28 |
| pabelanger | back | 19:28 |
| AJaeger | 9736 is a number from earlier today - so 20k looks like everything twice? | 19:28 |
| jeblair | oh, that may be normal then, if we have both node-specific and non-node-specific versions of each job | 19:29 |
| *** flwang has quit IRC | 19:29 | |
| fungi | sounds right | 19:30 |
| fungi | btw, mass spam files deletion underway. i'm logging and will post stats shortly | 19:31 |
| *** ifarkas has quit IRC | 19:31 | |
| pleia2 | nice work, fungi | 19:31 |
| fungi | if the wiki gets slower... yeah | 19:31 |
| *** links has quit IRC | 19:31 | |
| fungi | i'll also post the script i used | 19:31 |
| fungi | we can tweak it for cleaning up other spam too | 19:31 |
| notmorgan | jeblair: wow 20k? | 19:33 |
| notmorgan | jeblair: thats a lot. | 19:33 |
| Shrews | so, are we always running geard over the edge? or did our edge just get redefined? | 19:33 |
| anteaya | fungi: yay! | 19:33 |
| jeblair | we've pretty much been at zuulv2's scalability limits for a while. thus the urgency for v3 | 19:33 |
| notmorgan | jeblair: also, the CLOSE_WAIT sockets might just be waiting for python to get around to closing things down fwiw, conn.close() will close down the socket when it gets a chance | 19:33 |
| jeblair | Shrews: i think we've been close to it for a while | 19:33 |
| notmorgan | jeblair: from what i've seen if you're under load conn.shutdown() then conn.close() ensures a more timeline finish | 19:34 |
| *** javeriak has quit IRC | 19:34 | |
| notmorgan | jeblair: this is based on some recent reading i've been doing. | 19:34 |
| *** flwang has joined #openstack-infra | 19:34 | |
| notmorgan | s/timeline/timely | 19:34 |
| rcarrillocruz | pabelanger: baremetal00 on infra cloud is reachable | 19:34 |
| notmorgan | doesn't mean it would solve the issue. | 19:34 |
| rcarrillocruz | crinkle: ^ | 19:34 |
| *** bpokorny has quit IRC | 19:35 | |
| rcarrillocruz | nibalizer: ^ | 19:35 |
| pabelanger | rcarrillocruz: excellent! | 19:35 |
| jeblair | notmorgan: it was trying to read from a socket in close wait, so it did not know it was supposed to close it | 19:35 |
| rcarrillocruz | 15.184.224.4 | 19:35 |
| rcarrillocruz | so whoever, should update rack dns | 19:35 |
| pabelanger | Didn't we decided to use a new subdomain for infracloud? | 19:36 |
| rcarrillocruz | tomorrow i'll start amending and adding more info to the infracloud inventory | 19:36 |
| pabelanger | since we didn't have space in openstack.org | 19:36 |
| fungi | pabelanger: same subdomain, just in a separate zone | 19:36 |
| notmorgan | jeblair: ah. then weird. | 19:36 |
| pabelanger | fungi: okay, let me peak into that | 19:36 |
| notmorgan | jeblair: it should plain error on reading from sockets in close_wait. | 19:36 |
| fungi | pabelanger: we were already doing ic.openstack.org as the subdomain, we just want to make sure we create an ic.openstack.org zone to put it in this time | 19:36 |
| notmorgan | jeblair: *should* is the key thing. | 19:36 |
| jeblair | notmorgan: i agree | 19:37 |
| jeblair | notmorgan: it handles about 20 thousand rude disconnections daily too, without incident | 19:37 |
| *** nadya has joined #openstack-infra | 19:37 | |
| notmorgan | hmm. | 19:37 |
| *** javeriak has joined #openstack-infra | 19:37 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/devstack-gate: DNM playing with tempest run https://review.openstack.org/327291 | 19:38 |
| jeblair | as edge cases go, it's pretty edgy | 19:38 |
| notmorgan | yeah | 19:38 |
| fungi | rcarrillocruz: awesome progress on that--thanks!!! | 19:38 |
| rcarrillocruz | fungi: sure thing | 19:38 |
| notmorgan | jeblair: might be a very low level bug in sockets under that kind of load we see -- and very edge-case-y | 19:39 |
| *** salv-orlando has quit IRC | 19:39 | |
| notmorgan | jeblair: i'll keep my eyes open for things that look like that, like a bad loop break in an edge case. | 19:40 |
| *** gyee has quit IRC | 19:40 | |
| *** salv-orlando has joined #openstack-infra | 19:40 | |
| notmorgan | readloop. | 19:40 |
| pabelanger | fungi: we need an email address for the new domain, so I use the one we have for openstack.org? | 19:41 |
| pabelanger | subdomain | 19:41 |
| *** bhavik has quit IRC | 19:42 | |
| fungi | pabelanger: what is it? hostmaster@ or something else? | 19:42 |
| pabelanger | ipadmin@ | 19:42 |
| *** bpokorny has joined #openstack-infra | 19:42 | |
| rcarrillocruz | folks, i just disabled password authentication on ssh in the baremetal00 | 19:42 |
| rcarrillocruz | it was enabled | 19:42 |
| rcarrillocruz | fyi | 19:43 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 Fixes: Bundled Fixes https://review.openstack.org/325661 | 19:46 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Python 3 Fix: Divide operator returns float https://review.openstack.org/327303 | 19:46 |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/zuul: Do not try and join() on booleans https://review.openstack.org/327304 | 19:46 |
| rcarrillocruz | if any of you can't get in cos you changed ssh keys or whatever, let me know and pm me your pub key | 19:46 |
| *** Apoorva has joined #openstack-infra | 19:47 | |
| *** sdake_ has joined #openstack-infra | 19:47 | |
| pabelanger | fungi: rcarrillocruz: baremetal00.ic.openstack.org ? or are we still doing east and west? | 19:48 |
| *** sdake has quit IRC | 19:48 | |
| rcarrillocruz | iirc, we wanted to keep two regions, despite being on the same DC ? | 19:48 |
| rcarrillocruz | jeblair , fungi ^ | 19:48 |
| fungi | we were doing vanilla and chocolate, but no idea what the plan is for this test deploy we know will get wiped soon and readdressed when it's relocated | 19:49 |
| AJaeger | so, https://review.openstack.org/283573 needs probably updating - it uses infracloud-east... | 19:50 |
| pabelanger | baremetal00.vanilla.ic.openstack.org? | 19:50 |
| fungi | wfm | 19:50 |
| *** tkelsey has joined #openstack-infra | 19:50 | |
| AJaeger | pabelanger, fungi, could either of you -1 that one, please? | 19:50 |
| *** yolanda has joined #openstack-infra | 19:50 | |
| AJaeger | pabelanger: it has your +2 :) | 19:51 |
| pabelanger | AJaeger: done | 19:51 |
| *** timello has quit IRC | 19:52 | |
| AJaeger | thanks | 19:52 |
| pabelanger | fungi: rcarrillocruz: baremetal00.vanilla.ic.openstack.org online | 19:52 |
| pabelanger | lets see what is needed to add it into our puppet wheel | 19:53 |
| fungi | okay, my script is up to ~150 spam file pages deleted so far, no new tracebacks | 19:53 |
| fungi | i'm about to leave it running and go get something to eat | 19:54 |
| rcarrillocruz | \o/ | 19:54 |
| anteaya | fungi: wooo, enjoy food | 19:54 |
| *** cody-somerville has quit IRC | 19:54 | |
| *** dimtruck is now known as zz_dimtruck | 19:54 | |
| fungi | here's what the current state of the script looks like (with my password redacted) http://paste.openstack.org/show/509006/ | 19:54 |
| *** tkelsey has quit IRC | 19:55 | |
| anteaya | nice | 19:55 |
| fungi | now 300 deletions and hasn't crashed out yet | 19:56 |
| rockstar | Can someone from infra add me to the nova-lxd-core team? The wiki page isn't clear on exactly HOW we go about getting the infra team to add us to the group created with the import. | 19:56 |
| rockstar | (thus, I'm here, hat in hand, hoping someone here can do it) | 19:56 |
| fungi | rockstar: asking in here, hat in hand, is how you do it ;) | 19:56 |
| fungi | (or on the infra ml) | 19:56 |
| *** zz_dimtruck is now known as dimtruck | 19:57 | |
| rockstar | fungi: great. My username in gerrit is "rockstar" and I just need to be added to the nova-lxd-core team. | 19:57 |
| rockstar | I can add everyone else. | 19:57 |
| fungi | rockstar: reminding us the url of the gerrit change which added that group/acl/project helps too | 19:57 |
| fungi | since we have to vet the request | 19:57 |
| rockstar | fungi: https://review.openstack.org/#/c/324667/ | 19:58 |
| fungi | generally we either add the person who submitted the new project creation change (for unofficial projects), or the recognized ptl (for official projects under the tc) | 19:58 |
| fungi | and yeah, looks like it's an unofficial project and you are the author of that change, so no problem | 19:59 |
| rockstar | I'm both the person who submitted it, and the lead (though not officially a ptl) | 19:59 |
| *** asettle has joined #openstack-infra | 19:59 | |
| *** asettle has quit IRC | 19:59 | |
| fungi | rockstar: done. you can edit it further at https://review.openstack.org/#/admin/groups/1418,members | 20:00 |
| fungi | up to 600 deletions now and still going | 20:00 |
| anteaya | woooo | 20:01 |
| *** fawadkhaliq has quit IRC | 20:01 | |
| AJaeger | great, fungi! Enjoy your well-deserved meal! | 20:01 |
| pabelanger | fungi: great work | 20:01 |
| * SotK wonders if anyone has ever given any thought to the idea of having "live" access to the logs from Zuul/Jenkins jobs? | 20:01 | |
| *** fawadkhaliq has joined #openstack-infra | 20:01 | |
| rockstar | fungi: cheers | 20:02 |
| fungi | SotK: we used to, but that depended on having jenkins publicly available. now that we've had to block access to jenkins masters for security reasons it's temporarily disabled | 20:02 |
| jeblair | it will be back soon | 20:02 |
| fungi | SotK: but the zuul v2.5 work nearing completion gets rid of jenkins for us, and jeblair has devised a new solution for streaming logs for jobs which are run directly by zuul-launcher | 20:02 |
| notmorgan | SotK: we are actively working on it. | 20:02 |
| SotK | that is exciting | 20:03 |
| SotK | how far off is zuul v2.5 ooi? | 20:03 |
| jeblair | it's 1/8 here | 20:03 |
| jeblair | maybe end of week? next week? | 20:03 |
| fungi | (counting by the number of jenkins masters we've shut down, anyway!) | 20:03 |
| SotK | (streaming logs from jobs run by zuul-launcher sounds like exactly what I want) | 20:04 |
| rcarrillocruz | it's amazing the current setup can handle both scenarios, awesome job folks | 20:04 |
| *** e0ne has quit IRC | 20:04 | |
| fungi | yeah, basically it's already in production, just phased implementation as we ramp up load on it and ramp down load on jenkins | 20:04 |
| openstackgerrit | Merged openstack-infra/project-config: Create the ARA project https://review.openstack.org/321226 | 20:04 |
| *** abregman has quit IRC | 20:04 | |
| *** nwkarsten has quit IRC | 20:04 | |
| *** fawadkhaliq has quit IRC | 20:05 | |
| rcarrillocruz | time to write zuul-launcher and whatever new things on puppet-openstackci i guess... | 20:05 |
| fungi | 900 deletions so far, and now i really am heading out to eat. bbiab | 20:05 |
| * SotK goes to hunt some documentation | 20:06 | |
| anteaya | see you | 20:06 |
| *** EricGonczer_ has quit IRC | 20:06 | |
| openstackgerrit | Morgan Fainberg proposed openstack-infra/gear: Do not encode the type (b) in the job name https://review.openstack.org/327310 | 20:06 |
| rcarrillocruz | tomorrow i'd like to test out ansible-puppet with ansible 2.1, at least that i know i should get an error straightaway | 20:06 |
| rcarrillocruz | looking stuff to work on | 20:06 |
| anteaya | SotK: I don't think there is any yet | 20:06 |
| fungi | rcarrillocruz: zuul v2.5 is not really intended for outside use as i understand it, since it's just a short-term solution while we work toward zuul v3 | 20:06 |
| notmorgan | jeblair ^: fix for gear and handleRequest() so we're not encoding the datatype in the string formatter. | 20:06 |
| *** bpokorny has quit IRC | 20:07 | |
| fungi | rcarrillocruz: jeblair can probably go into more detail when he's got a free moment | 20:07 |
| anteaya | rcarrillocruz: if you are looking for stuff to work on we could use a hand with storyboard gerrit integration, specifically seeing if the storyboard gerrit plugin works with gerrit 2.11 | 20:07 |
| rcarrillocruz | fungi: well , that's good for me, cos got back to infra-ansible recently and i was thinking 'doh, bad timing to work on jenkins/zuul when jenkins is going away!' | 20:07 |
| *** nwkarsten has joined #openstack-infra | 20:08 | |
| rcarrillocruz | anteaya: i'm still waiting for nibalizer list of ansible-puppet issues... depending on how that goes i'll ping you or zara about sb | 20:08 |
| *** nwkarsten has quit IRC | 20:08 | |
| *** Sukhdev has quit IRC | 20:08 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update EnvInject plugin https://review.openstack.org/324140 | 20:08 |
| *** nwkarsten has joined #openstack-infra | 20:08 | |
| anteaya | rcarrillocruz: okay thank you | 20:09 |
| *** amrith is now known as _amrith_ | 20:09 | |
| notmorgan | fungi: if you think of another project that needs some py3 love, please let me know. i'll put it on my backlog when i need a break from the current things. | 20:09 |
| *** cloudtrainme has quit IRC | 20:09 | |
| anteaya | rcarrillocruz: just wanted to help answer your call for looking for stuff to work on | 20:09 |
| *** timello has joined #openstack-infra | 20:09 | |
| *** jeblair has quit IRC | 20:09 | |
| anteaya | notmorgan: we could use your py3 help on this story: https://storyboard.openstack.org/#!/story/2000576 | 20:10 |
| *** jeblair has joined #openstack-infra | 20:10 | |
| notmorgan | anteaya: noted. | 20:10 |
| rcarrillocruz | although | 20:10 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Bring baremetal00.vanilla.ic.openstack.org online https://review.openstack.org/327314 | 20:10 |
| rcarrillocruz | wait | 20:10 |
| rcarrillocruz | anteaya: | 20:10 |
| rcarrillocruz | what's that work about | 20:10 |
| rcarrillocruz | writing a plugin | 20:10 |
| notmorgan | anteaya: will keep that on my backlog. | 20:10 |
| pabelanger | rcarrillocruz: first pass at updating our configuration ^ | 20:10 |
| rcarrillocruz | or just testing an alreaady existing plugin with a gerrit | 20:10 |
| anteaya | rcarrillocruz: no, the plugin is written | 20:10 |
| *** asettle has joined #openstack-infra | 20:10 | |
| jeblair | "vanilla ic" | 20:11 |
| rcarrillocruz | oh! | 20:11 |
| rcarrillocruz | well | 20:11 |
| rcarrillocruz | i'll ping zara | 20:11 |
| anteaya | rcarrillocruz: yes, testing the existing plugin with gerrit | 20:11 |
| anteaya | rcarrillocruz: awesome thank you | 20:11 |
| notmorgan | jeblair: i saw that... i laughed :) | 20:11 |
| notmorgan | jeblair: i cried. | 20:11 |
| rcarrillocruz | cos infra-ansible brings up a gerrit with almost zero effort | 20:11 |
| notmorgan | jeblair: i'd read again. | 20:11 |
| anteaya | rcarrillocruz: that is a great quality infra-ansible has | 20:11 |
| *** cloudtrainme has joined #openstack-infra | 20:11 | |
| notmorgan | anteaya: it's probably very simple <var>.encode('utf-8') missing | 20:11 |
| *** esberglu has quit IRC | 20:11 | |
| anteaya | notmorgan: probably | 20:12 |
| nibalizer | rcarrillocruz: oh right | 20:12 |
| nibalizer | ill create these now | 20:12 |
| anteaya | notmorgan: and thank you | 20:12 |
| notmorgan | anteaya: before being handled (usually from some wire-protocol, or urllib directly) | 20:12 |
| anteaya | notmorgan: I pictured jeblair with platinum blond standy-up hair | 20:12 |
| notmorgan | jeblair: i might need to pick your brain on the zuul "settled" bits. | 20:12 |
| anteaya | notmorgan: and then quickly moved on | 20:12 |
| notmorgan | anteaya: you should totally photoshop/gimp-photo-edit this now :P | 20:13 |
| anteaya | not a chance | 20:13 |
| notmorgan | anteaya: hehehe awwwww :( | 20:13 |
| anteaya | I need to be able to sleep at night | 20:13 |
| *** bpokorny has joined #openstack-infra | 20:13 | |
| *** EricGonczer_ has joined #openstack-infra | 20:14 | |
| rcarrillocruz | pabelanger: commented on the patch, some ips must be amended | 20:15 |
| pabelanger | rcarrillocruz: I figured that | 20:15 |
| *** timello has quit IRC | 20:15 | |
| rcarrillocruz | pabelanger: so, i asked on ironic channel this morning and ironic nodes can have arbitrary metadata | 20:15 |
| rcarrillocruz | the idea will be to put serial_number as a property | 20:15 |
| rcarrillocruz | but that can be done by myself | 20:15 |
| rcarrillocruz | cos i have the spreadsheet | 20:15 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Bring baremetal00.vanilla.ic.openstack.org online https://review.openstack.org/327314 | 20:16 |
| * rcarrillocruz would like to have a proper CMDB to manage all this, but oh spreadsheets? pffffff | 20:16 | |
| rcarrillocruz | so | 20:17 |
| rcarrillocruz | i mean | 20:17 |
| pabelanger | rcarrillocruz: sure, I'll defer to you to drive that since you have a better understanding. | 20:17 |
| rcarrillocruz | anteaya: just gave me the perfect excuse to put storyboard deployment on infra-ansible | 20:17 |
| rcarrillocruz | \o/ | 20:17 |
| notmorgan | rcarrillocruz: i can get you an AccessDB /s :P | 20:17 |
| rcarrillocruz | i'll def. work on that | 20:17 |
| anteaya | rcarrillocruz: yay | 20:17 |
| pabelanger | rcarrillocruz: I'm still trying to understand how ansible inventory pulls in this info. | 20:17 |
| notmorgan | rcarrillocruz: ^_^ :P | 20:17 |
| rcarrillocruz | notmorgan: lol... i loved the keys icons :D | 20:17 |
| notmorgan | rcarrillocruz: i know, right?! :) | 20:17 |
| notmorgan | totally the reason use it! | 20:18 |
| * rcarrillocruz is old...started playing with 'databases' with dBase III | 20:18 | |
| nibalizer | rcarrillocruz: do we have a storyboard project for ansible-puppet? | 20:18 |
| rcarrillocruz | nibalizer: i haven't checked tbh | 20:18 |
| nibalizer | im not seeing it | 20:18 |
| anteaya | anisible-role-puppet right? https://storyboard.openstack.org/#!/project/807 | 20:19 |
| anteaya | it got renamed yes? | 20:19 |
| pabelanger | DB/400 here | 20:19 |
| rcarrillocruz | ah yeah | 20:19 |
| rcarrillocruz | mordred did | 20:19 |
| pabelanger | for the AS/400 | 20:19 |
| rcarrillocruz | there you go | 20:19 |
| nibalizer | oh | 20:19 |
| rcarrillocruz | pabelanger: haha, awesome | 20:19 |
| rcarrillocruz | as/400 | 20:19 |
| rcarrillocruz | geez | 20:19 |
| nibalizer | neat | 20:19 |
| nibalizer | thanks anteaya | 20:20 |
| anteaya | nibalizer: welcome | 20:20 |
| *** esberglu has joined #openstack-infra | 20:21 | |
| *** nadya has quit IRC | 20:22 | |
| rcarrillocruz | i started looking at https://tumblr.github.io/collins/ months ago to put the gozer stuff on it, but then moved on from Gozer | 20:22 |
| rcarrillocruz | looks cool | 20:22 |
| *** timello has joined #openstack-infra | 20:22 | |
| anteaya | interesting | 20:24 |
| *** mixos has joined #openstack-infra | 20:24 | |
| *** oanson has quit IRC | 20:24 | |
| openstackgerrit | Merged openstack-infra/project-config: Don't run puppet tests on rst/unit test changes https://review.openstack.org/324158 | 20:25 |
| *** maishsk_ has joined #openstack-infra | 20:25 | |
| *** maishsk has quit IRC | 20:26 | |
| *** maishsk_ is now known as maishsk | 20:26 | |
| anteaya | it would be interesting to have their feedback on what they found problematic with openstack | 20:26 |
| *** ihrachys has joined #openstack-infra | 20:28 | |
| *** timello has quit IRC | 20:29 | |
| jroll | anteaya: well, to begin with, openstack doesn't have a CMDB-like thing, which was their initial goal | 20:30 |
| jroll | this sounds like a CMDB + neutron + nova + ironic + heat | 20:30 |
| anteaya | that would do it | 20:30 |
| *** flwang1 has quit IRC | 20:30 | |
| *** bpokorny has quit IRC | 20:30 | |
| jroll | (with a paas on top) | 20:30 |
| *** cody-somerville has joined #openstack-infra | 20:30 | |
| *** piet has joined #openstack-infra | 20:31 | |
| anteaya | um hmmm | 20:31 |
| *** maishsk has quit IRC | 20:32 | |
| *** maishsk has joined #openstack-infra | 20:34 | |
| openstackgerrit | Simon Leinen proposed openstack/diskimage-builder: Use dib-python in pkg-map/bin/pkg-map https://review.openstack.org/311634 | 20:34 |
| rcarrillocruz | jroll: btw, i remember there was a session to have some sort of a 'mini-cmdb' in ironic in Austin? | 20:35 |
| *** devananda has quit IRC | 20:35 | |
| jroll | rcarrillocruz: erm, not that I know of? :) | 20:35 |
| jroll | we have an 'inspector' project that can get info about servers | 20:35 |
| jroll | but no, ironic is explicitly not a CMDB | 20:35 |
| *** devananda has joined #openstack-infra | 20:36 | |
| *** javeriak_ has joined #openstack-infra | 20:36 | |
| *** javeriak_ has quit IRC | 20:36 | |
| *** cody-somerville has quit IRC | 20:38 | |
| *** timello has joined #openstack-infra | 20:38 | |
| *** timello has quit IRC | 20:38 | |
| *** javeriak has quit IRC | 20:39 | |
| rcarrillocruz | hmm, yeah i think i refer to https://etherpad.openstack.org/p/ironic-newton-summit-hardware-pools . Not a cmdb, but some support managing new objects... | 20:40 |
| rcarrillocruz | anyway, food | 20:40 |
| rcarrillocruz | later folks | 20:40 |
| *** nadya has joined #openstack-infra | 20:40 | |
| anteaya | rcarrillocruz: enjoy food | 20:41 |
| *** timello has joined #openstack-infra | 20:42 | |
| *** gouthamr has quit IRC | 20:43 | |
| nibalizer | rcarrillocruz: your stories, they are storied | 20:44 |
| anteaya | nibalizer: link? | 20:44 |
| jeblair | notmorgan: happy to talking about settling | 20:45 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Make isJobRegistered for gearman optional https://review.openstack.org/327323 | 20:45 |
| *** nadya has quit IRC | 20:45 | |
| pabelanger | jeblair: ^ I believe that enables us to by pass NOT_REGISTERED jobs. | 20:45 |
| *** dimtruck is now known as zz_dimtruck | 20:46 | |
| pabelanger | open to suggestions on naming of variable | 20:46 |
| *** zeih has quit IRC | 20:47 | |
| jeblair | pabelanger: suggested a variable name, otherwise, lgtm, thanks! | 20:47 |
| pabelanger | jeblair: sure | 20:48 |
| *** openstackgerrit has quit IRC | 20:48 | |
| *** openstackgerrit has joined #openstack-infra | 20:48 | |
| *** devananda has quit IRC | 20:51 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update EnvInject plugin https://review.openstack.org/324140 | 20:52 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Make isJobRegistered for gearman optional https://review.openstack.org/327323 | 20:52 |
| *** devananda has joined #openstack-infra | 20:53 | |
| openstackgerrit | Sean Dague proposed openstack-infra/devstack-gate: DNM: experimenting with local.conf https://review.openstack.org/326585 | 20:53 |
| *** ffrank has joined #openstack-infra | 20:53 | |
| *** PollyZ has quit IRC | 20:54 | |
| *** gyee has joined #openstack-infra | 20:55 | |
| dmsimard | What's the process for bootstrapping the initial members of gerrit groups for a new project ? | 20:55 |
| *** roxanaghe has joined #openstack-infra | 20:56 | |
| *** jbernard has quit IRC | 20:56 | |
| *** erlon has quit IRC | 20:56 | |
| dmsimard | I'd need to add myself to the release and core group in https://review.openstack.org/#/admin/projects/openstack/ara,access | 20:56 |
| *** nadya has joined #openstack-infra | 20:58 | |
| jeblair | dmsimard: you just performed the process. :) [you're added] | 20:59 |
| dmsimard | jeblair: cool, thanks. | 20:59 |
| jeblair | np | 20:59 |
| rcarrillocruz | nibalizer: awesome, thx mate | 21:01 |
| *** kien-ha has quit IRC | 21:01 | |
| *** dizquierdo has quit IRC | 21:01 | |
| *** PollyZ has joined #openstack-infra | 21:01 | |
| *** _ari_ has quit IRC | 21:01 | |
| *** sdake_ has quit IRC | 21:01 | |
| *** jbernard has joined #openstack-infra | 21:03 | |
| *** nadya has quit IRC | 21:03 | |
| jeblair | i'm starting zl on zlstatic01 | 21:03 |
| *** asettle has quit IRC | 21:03 | |
| *** julim has quit IRC | 21:04 | |
| jeblair | and stopping it since it seems to be picking up nodepool nodes | 21:05 |
| dmsimard | anteaya: is the process for onboarding projects to storyboard documented yet ? | 21:05 |
| *** docaedo2 has joined #openstack-infra | 21:05 | |
| *** chlong has joined #openstack-infra | 21:06 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: WIP: Rework timeline events into a timeline https://review.openstack.org/327231 | 21:06 |
| docaedo | krotscheck: any time you have a minute, I'd love to get your feedback on this spec: https://review.openstack.org/#/c/319506/ | 21:06 |
| *** mixos has quit IRC | 21:07 | |
| krotscheck | docaedo: Checking | 21:07 |
| docaedo | krotscheck: I believe you know some js stuff? If so would be great to collaborate on polishing any rough edges with it | 21:07 |
| docaedo | krotscheck: thanks! | 21:07 |
| *** docaedo2 has quit IRC | 21:07 | |
| *** eharney has quit IRC | 21:09 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-infracloud: Rename hpuswest to vanilla https://review.openstack.org/327332 | 21:10 |
| pabelanger | jeblair: ack | 21:10 |
| pabelanger | jeblair: anything else needed on the decommission of jenkins01? | 21:11 |
| *** banix has quit IRC | 21:11 | |
| dmsimard | What's the formulae for logs.o.o links ? logs.o.o/<?>/<change number>/<patch set>/<?>/logs | 21:11 |
| dmsimard | I want to peek at logs of something that's still in zuul queue. | 21:12 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: use underscores where appropriate https://review.openstack.org/327333 | 21:12 |
| jeblair | pabelanger: i'm plodding ahead on it | 21:12 |
| *** docaedo2 has joined #openstack-infra | 21:12 | |
| *** docaedo2 has quit IRC | 21:13 | |
| *** docaedo2 has joined #openstack-infra | 21:13 | |
| pabelanger | dmsimard: https://review.openstack.org/#/c/321580/2/doc/source/developers.rst | 21:13 |
| *** krotscheck-loung has joined #openstack-infra | 21:14 | |
| krotscheck-loung | docaedo: Testing | 21:14 |
| dmsimard | pabelanger: cool ty | 21:14 |
| openstackgerrit | James E. Blair proposed openstack-infra/puppet-zuul: Add support for launcher accept_nodes parameter https://review.openstack.org/327335 | 21:14 |
| *** krotscheck-loung has quit IRC | 21:14 | |
| docaedo2 | krotscheck-loung: testing here as well :) try to keep a long-running instance on a machine at home and it's been solid | 21:14 |
| pabelanger | jeblair: cool, I think the next time we restart nodepool we'll pick up the new statsd stats for zuul-launcher | 21:15 |
| dmsimard | pabelanger: ah, that explicitely mentions the POST queue, not the check queue | 21:15 |
| krotscheck | docaedo2: I just refreshed the browser page, lost the settings and the connection. Do you know of a setting that permits those to be saved? | 21:15 |
| *** ldnunes has quit IRC | 21:15 | |
| *** krotscheck-loung has joined #openstack-infra | 21:16 | |
| krotscheck-loung | Otherwise, it's pretty slick | 21:16 |
| docaedo | krotscheck: ah yes, you change public to false in default/config.js, and then you need to create a user account | 21:16 |
| *** krotscheck-loung has quit IRC | 21:16 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder: Move pkg-map to dib-python https://review.openstack.org/326228 | 21:17 |
| *** nadya has joined #openstack-infra | 21:17 | |
| docaedo | krotscheck-loung: when you do that, you get multiple accounts running from same place, they auth in and their session is persistent | 21:17 |
| *** xwizard has quit IRC | 21:18 | |
| docaedo | krotscheck: perfect world would be letting people use their openstack ID to log in - but I was hoping someone who knows the JS stuff might be able to help or at least evaluate how much work it would be to improve that | 21:18 |
| *** trown is now known as trown|outtypewww | 21:18 | |
| *** xwizard has joined #openstack-infra | 21:18 | |
| openstackgerrit | Merged openstack-infra/project-config: Remove jenkins01 target from nodepool https://review.openstack.org/326742 | 21:18 |
| krotscheck | docaedo: Well, it's a node app. I'm sure an OAuth2 or OpenID Connect thing exists somewhere. | 21:18 |
| docaedo | krotscheck: the folks at #thelounge are very receptive to changes and such | 21:18 |
| *** EricGonczer_ has quit IRC | 21:18 | |
| krotscheck | The question is: How easy would the integration be. | 21:18 |
| krotscheck | I'm also concerned about scale; maintaining persistent connections is going to hit hard system limits. | 21:19 |
| pabelanger | dmsimard: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/openstack_functions.py#n18 is out the log URL is built | 21:19 |
| docaedo | krotscheck: yes, my expectation is that there would only be a few (less than 40?) users, but that number comes from my fertile imagination | 21:20 |
| dmsimard | pabelanger: I figured it out | 21:20 |
| *** krotscheck-loung has joined #openstack-infra | 21:20 | |
| dmsimard | pabelanger: it's logs.o.o/<last 2 digits of change number>/<change number> | 21:20 |
| krotscheck | testing persistence. | 21:20 |
| docaedo | krotscheck: the hope is that more working group people would by using this become more comfortable with IRC, but no telling really if the number of users would be 5 or 500 | 21:20 |
| dmsimard | and then I can browse from that point | 21:20 |
| pabelanger | dmsimard: basically | 21:21 |
| pabelanger | dmsimard: logs only show up once the job has finished, and status.o.o/zuul will update the link once that happens | 21:21 |
| jeblair | pabelanger: oh, 2 more things on 327323 | 21:21 |
| *** nadya has quit IRC | 21:22 | |
| pabelanger | dmsimard: if, you know the job is running under zl01.0.0, you can use nc <node IP> 8088 to stream the output live | 21:22 |
| pabelanger | jeblair: np | 21:22 |
| krotscheck-loung | docaedo: I'd err on the larger size. Hosting what amounts to a free bouncer also may result in abuse. I know given the opportunity, I'd turn off my own bouncer in favor of this. | 21:23 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: use underscores where appropriate https://review.openstack.org/327333 | 21:23 |
| krotscheck-loung | docaedo: Not saying that we should scuttle this altogether. I'd have to dig into the codebase to see what options we have for limiting the usefulness- for instance, if we could restrict channels and servers to openstack only, that'd go a long way towards keeping it with its original mission. | 21:24 |
| krotscheck-loung | I have to say, this is REALLY slick. | 21:24 |
| docaedo | krotscheck-loung: glad you are liking it - I'm a big fan and am pretty impressed with it | 21:24 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul: Make isJobRegistered for gearman optional https://review.openstack.org/327323 | 21:24 |
| pabelanger | jeblair: okay, fixed. I'll update out puppet manifests to set check_job_registration=False for us | 21:25 |
| docaedo | krotscheck-loung: I agree abuse is a concern - my initial expectation was that a new account would have to be handled through a gerrit review, which I'd be happy to manage for the sake of working group people | 21:25 |
| krotscheck-loung | Pretty responsive, too. | 21:25 |
| *** _shaps_ has joined #openstack-infra | 21:26 | |
| docaedo | krotscheck-loung: at least in that case, we wouldn't have to worry about random people finding it and abusing it | 21:26 |
| *** rhallisey has quit IRC | 21:26 | |
| krotscheck-loung | True dat | 21:26 |
| docaedo | krotscheck-loung: the other hope with this is that we find a balance where it's easy enough to get up and running for people to where they don't keep saying "lets just use slack" | 21:26 |
| *** krotscheck-loung has quit IRC | 21:27 | |
| *** krotscheck-loung has joined #openstack-infra | 21:27 | |
| krotscheck | Connections are lost on reboot. | 21:27 |
| krotscheck | But channels are not. | 21:27 |
| krotscheck | Nice | 21:27 |
| krotscheck | docaedo: Ok, lemme dig into their code a little and see whether there's easy extension hooks. | 21:28 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Do not accept nodes on zlstatic01 https://review.openstack.org/327346 | 21:28 |
| docaedo | krotscheck: very much appreciated | 21:28 |
| dmsimard | pabelanger: where can I find the bindep binary ? Gate job is complaining about other-requirements.txt parsing failure and I'd like to test locally before submitting a fix., | 21:28 |
| dmsimard | nevermind | 21:29 |
| dmsimard | found it. | 21:29 |
| *** matt-borland has quit IRC | 21:29 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: remove deprecated hyphens https://review.openstack.org/327347 | 21:29 |
| openstackgerrit | James E. Blair proposed openstack-infra/puppet-zuul: Remove deprecated hyphens from zuul launcher https://review.openstack.org/327349 | 21:30 |
| lifeless_ | fungi: dhellmann: context? | 21:30 |
| *** lifeless_ is now known as lifeless | 21:30 | |
| lifeless | dhellmann: if you're around, the releases regex in mailman needs one more tweak | 21:31 |
| jeblair | pabelanger: can you take a quick look at https://review.openstack.org/327333 https://review.openstack.org/327335 https://review.openstack.org/327347 https://review.openstack.org/327349 ? | 21:31 |
| *** tonytan4ever has quit IRC | 21:31 | |
| lifeless | dhellmann: to use mailmans 'give me everything except pattern X' you actually have to do 'give me everything that does not match a selected pattern + select at least one pattern' - so we need a second pattern (even if its a useless one). | 21:31 |
| jeblair | pabelanger: also https://review.openstack.org/327346 | 21:31 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-zuul: Expose gearman_check_job_registration setting https://review.openstack.org/327351 | 21:32 |
| pabelanger | jeblair: checking | 21:32 |
| *** amitgandhinz has quit IRC | 21:33 | |
| krotscheck | docaedo: Alright, I'll comment on the review. | 21:33 |
| docaedo | krotscheck: thanks! | 21:34 |
| pabelanger | jeblair: +2 for everybody | 21:35 |
| *** nadya has joined #openstack-infra | 21:35 | |
| *** yamahata has quit IRC | 21:35 | |
| *** maishsk has quit IRC | 21:35 | |
| pabelanger | I have to step away from the computer for a bit, will be back online in a few hours | 21:35 |
| jeblair | a round of +2's on pabelanger! | 21:35 |
| jeblair | pabelanger: cheers! | 21:35 |
| *** ihrachys has quit IRC | 21:36 | |
| *** ihrachys has joined #openstack-infra | 21:37 | |
| dhellmann | lifeless: I have to sign off for the evening. If you have a pattern, you can DM it to me and I can give it a try tomorrow. | 21:37 |
| *** baoli_ has quit IRC | 21:38 | |
| *** piet has quit IRC | 21:38 | |
| dhellmann | lifeless : regarding the other thing, you might have some input on this ML thread: http://lists.openstack.org/pipermail/openstack-dev/2016-June/096895.html | 21:38 |
| lifeless | dhellmann: I'll see if someone else around can do it today and failing that DM you | 21:38 |
| dhellmann | lifeless : sounds good. If you're no longer on the -dev list and have input on the thread, you can DM me about that, too | 21:38 |
| dhellmann | gotta run, family | 21:39 |
| lifeless | dhellmann: ciao | 21:39 |
| *** nadya has quit IRC | 21:39 | |
| lifeless | dhellmann: I am not on the -dev list but will send mail | 21:39 |
| dhellmann | lifeless : ok, cool, thanks | 21:39 |
| *** DevBox has quit IRC | 21:40 | |
| *** cody-somerville has joined #openstack-infra | 21:42 | |
| *** nadya has joined #openstack-infra | 21:44 | |
| *** cloudtrainme has quit IRC | 21:45 | |
| *** thorst has joined #openstack-infra | 21:47 | |
| *** tpsilva has quit IRC | 21:47 | |
| mriedem | i'm guessing a bunch of recent multinode NOT_REGISTERED job failures is a known issue? | 21:48 |
| lifeless | ok so who here can edit openstack-annouce list filters? | 21:48 |
| *** timello has quit IRC | 21:49 | |
| *** nadya has quit IRC | 21:49 | |
| *** sdake has joined #openstack-infra | 21:49 | |
| *** jcoufal has quit IRC | 21:51 | |
| dmsimard | Wow zuul's status.json is 2.2MB and 118k lines long | 21:53 |
| *** ihrachys_ has joined #openstack-infra | 21:53 | |
| *** rcernin is now known as rcernin|off | 21:54 | |
| *** sdake has quit IRC | 21:54 | |
| *** r-daneel has joined #openstack-infra | 21:55 | |
| anteaya | dmsimard: no, you add a line to project-config/gerrit/projects.yaml that says use-storyboard: true | 21:55 |
| jeblair | mriedem: how recent? | 21:55 |
| *** sdake has joined #openstack-infra | 21:55 | |
| anteaya | dmsimard: you should see it elsewhere in that file for an example | 21:55 |
| *** ihrachys has quit IRC | 21:56 | |
| *** hashar has quit IRC | 21:57 | |
| mriedem | jeblair: last hour? | 21:57 |
| dmsimard | anteaya: it'll automagically create the project ? | 21:57 |
| jeblair | mriedem: that's surprising. can you point me at one? | 21:57 |
| mriedem | https://review.openstack.org/#/c/325684/4 | 21:58 |
| anteaya | dmsimard: I do believe so | 21:58 |
| mriedem | jeblair: seen in at least 3 different patches in the last ~hour, only on multinode jobs | 21:58 |
| dmsimard | anteaya: okay, I'll try tomorrow and see what happens. | 21:58 |
| anteaya | okay | 21:59 |
| jeblair | mriedem: ah, that's been corrected. | 21:59 |
| *** shashank_hegde has quit IRC | 22:00 | |
| *** dprince has quit IRC | 22:01 | |
| anteaya | jeblair: if you have time to take a peek at SotK's latest patch on storyboard event timeline and comment work, login to storyboard-dev and open http://docs-draft.openstack.org/31/327231/3/check/gate-storyboard-webclient-js-draft/64c54a0//dist/#!/story/11 | 22:02 |
| anteaya | to get a preview | 22:02 |
| anteaya | I'm interested in your thoughts | 22:02 |
| *** nadya has joined #openstack-infra | 22:02 | |
| lifeless | fungi: ok so who here can edit openstack-annouce list filters? | 22:03 |
| jeblair | i've restarted nodepool because i think the jenkins01 removal made it sad | 22:03 |
| *** cody-somerville has quit IRC | 22:03 | |
| *** vhosakot has quit IRC | 22:03 | |
| fungi | lifeless: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-announce says you want ttx | 22:04 |
| anteaya | jeblair: ack | 22:04 |
| pleia2 | lifeless: I think ttx is the one who handles that | 22:04 |
| pleia2 | ohhifungi | 22:04 |
| jeblair | anteaya: i can't seem to get past the front pagewhen i go to that link | 22:04 |
| *** asettle has joined #openstack-infra | 22:04 | |
| lifeless | fungi: ah, I know that dhellmann also has the passwd, so figured there might be more | 22:04 |
| fungi | "OpenStack-announce list run by thierry at openstack.org" | 22:04 |
| anteaya | jeblair: you have to login | 22:04 |
| lifeless | fungi: yes, run by != set of folk that have the admin passwd :) | 22:04 |
| fungi | lifeless: if someone else knows ttx's login to lists.o.o, i don't want to know that ;) | 22:04 |
| *** krotscheck-loung has quit IRC | 22:04 | |
| anteaya | jeblair: I had the same problem, logging in fixed it for me | 22:04 |
| *** timello has joined #openstack-infra | 22:05 | |
| lifeless | fungi: hehk :) | 22:05 |
| fungi | lifeless: right, maybe mailman 4.x fixes this | 22:05 |
| dmsimard | anteaya: oh those comments are much, much better | 22:05 |
| lifeless | anyhwo fungi pleia2 thanks | 22:06 |
| anteaya | dmsimard: great thank you, can you perhaps review https://review.openstack.org/#/c/327231/ | 22:06 |
| fungi | pleia2: i'm back, checking scrollback now. did i break something and then go out drinking? | 22:06 |
| dmsimard | anteaya: I'd like to see what the status transition looks like | 22:06 |
| fungi | i hope so^H^Hnot | 22:06 |
| anteaya | dmsimard: what status transition? | 22:06 |
| *** gouthamr has joined #openstack-infra | 22:06 | |
| dmsimard | anteaya: like when you set a task from todo to review or something like that | 22:06 |
| pleia2 | fungi: only jeblair is breaking things | 22:07 |
| dmsimard | on storyboard.o.o it's just like "status changed from todo ==> review" | 22:07 |
| anteaya | well that view is the -dev server and is meant as the sandbox, play away | 22:07 |
| fungi | pleia2: i seriously doubt only jeblair is breaking things. he's probably the only one who will cop to it | 22:07 |
| anteaya | change the status and see what it does, or can you change the status on that story? | 22:07 |
| anteaya | ha ha ha | 22:08 |
| pleia2 | hehe | 22:08 |
| *** nadya has quit IRC | 22:08 | |
| *** shashank_hegde has joined #openstack-infra | 22:08 | |
| dmsimard | anteaya: oh, nothing special for the status changes (yet) | 22:08 |
| anteaya | dmsimard: okay | 22:08 |
| fungi | so my log says i my script deleted 1060 uploads from the wiki. i still see a bunch that are terribad spammy, so i'm going to add the accounts who uploaded them to my list | 22:09 |
| *** asettle has quit IRC | 22:09 | |
| anteaya | awesome | 22:09 |
| anteaya | and your script saved your clicking finger some work there | 22:10 |
| fungi | in positive news, "a bunch" is an order of magnitude fewer files than when i started | 22:10 |
| anteaya | your script adds to your spammers list? | 22:10 |
| anteaya | fungi: indeed | 22:10 |
| fungi | anteaya: i'll add account names to the list in my script and rerun it | 22:10 |
| anteaya | ah I see | 22:10 |
| jeblair | anteaya: i see 10 out of 11 options, so i suspect we're still paginating or something.. "Pedro Alvarez updated "THE FLOWER OF UTAH" on 2016-06-03 at 13:33:30" means nothing to me and may as well not be there. the non-comment items are still quite large and take up more screen real-estate than they warrant (which devalues the comments). i do like that the comments are easier to see now. | 22:10 |
| jeblair | anteaya: also "on 5 minutes ago" could maybe use some more anglicization. | 22:11 |
| *** ihrachys_ has quit IRC | 22:11 | |
| anteaya | jeblair: thank you for looking, and for your feedback, I appreciate it | 22:11 |
| *** nadya has joined #openstack-infra | 22:11 | |
| *** flepied has joined #openstack-infra | 22:11 | |
| jeblair | anteaya: (i never want to see "x minutes ago" i only ever want to see UTC timestamps. anything else means i have to do math, which i am bad at, and leaves me less time for things i am good at) | 22:12 |
| *** ihrachys has joined #openstack-infra | 22:12 | |
| fungi | i'm actually rerunning my script as-is first to make sure i didn't miss any due to pagination or other issues | 22:12 |
| fungi | answer: no | 22:12 |
| jeblair | anteaya: i do not know what timezone those times are in, which means i have to do *a lot* of math, and then ask you if you remember what time it was when you created the story | 22:12 |
| jeblair | anteaya: and then do more math | 22:12 |
| *** ihrachys has quit IRC | 22:12 | |
| anteaya | jeblair: I agree, utc timestamps for everyone | 22:12 |
| anteaya | jeblair: yup, I hear that | 22:13 |
| fungi | anteaya: i'm lacking context, but will agree with jeblair that any time i see events in an interface logged as "n <units> ago" i want to find the person responsible and explain timestamps to them in excruciating detail | 22:14 |
| openstackgerrit | Merged openstack-infra/infra-manual: Add info about the location of post job logs https://review.openstack.org/321580 | 22:14 |
| jeblair | anteaya: i said "10 out of 11 options" but i meant "10 out of 11 entries". you probably got that though. | 22:14 |
| anteaya | fungi: understood | 22:14 |
| anteaya | jeblair: i got that, thank you for the clarification though | 22:14 |
| jeblair | it's now 10 out of 13, and i can't seem to get to page 2 | 22:14 |
| jeblair | clicking the right arrow results in an http 400 | 22:15 |
| *** nadya has quit IRC | 22:16 | |
| fungi | also, for the benefit of hooverphonic fans everywhere, i named this mass deletion script "sh10151" ("...this is the serial number of our orbital gun..." it was playing on my jukebox earlier when i was hacking) | 22:16 |
| anteaya | jeblair: I see NaN to 13 of 13 in my view, I wonder why they are different | 22:17 |
| *** thiagop has quit IRC | 22:17 | |
| anteaya | fungi: ha ha ha | 22:17 |
| jeblair | anteaya: those are the characters i see, but when i count, i only see 10 things. | 22:17 |
| anteaya | ah sorry | 22:17 |
| * anteaya also counts | 22:18 | |
| jeblair | i ran out of fingers at the same time i ran out of timeline entries | 22:18 |
| fungi | if only we'd evolved with 16 fingers | 22:18 |
| openstackgerrit | Merged openstack-infra/puppet-zuul: Use zuul-launcher stop for service stop https://review.openstack.org/326741 | 22:18 |
| anteaya | ha ha ha | 22:19 |
| *** bpokorny has joined #openstack-infra | 22:19 | |
| anteaya | jeblair: I confirm I count 10 things, I also ran out of fingers but have my cat in my lap so could use a few of hers to go higher if I saw more things to count | 22:19 |
| anteaya | I also confirm a 400 error when clicking the red arrow | 22:19 |
| pleia2 | fungi: when you have a chance, care to dig into the secret imap trick for our rackspace email? | 22:20 |
| *** nadya has joined #openstack-infra | 22:20 | |
| *** rcernin|off has quit IRC | 22:21 | |
| *** signed8b_ has quit IRC | 22:21 | |
| fungi | pleia2: yep, let me finish adding overlooked wiki users to my blaster here and then i'll enlighten | 22:21 |
| jeblair | fungi: their fan will be pleased! | 22:21 |
| fungi | jeblair: their fan _was_ pleased | 22:21 |
| persia | Those who do finger counting: using fingers as bits is a useful way to expand one's counting max. | 22:23 |
| *** nwkarsten has quit IRC | 22:23 | |
| * fungi has been known to use that trick. 0-1023 on the fingers alone! | 22:23 | |
| jeblair | i must be doing it wrong. i still can only get up to 10 'on' bits. | 22:24 |
| fungi | still, wishing it could have been 0-65535 | 22:24 |
| persia | Indeed | 22:24 |
| lifeless | fungi: you have joints... | 22:24 |
| fungi | damn you, charles darwin! | 22:24 |
| persia | jeblair: 00000 00001, 00000 00010, 00000 00011, 00000 00100, ... | 22:25 |
| lifeless | you should be able to get at least trinary out of each digit | 22:25 |
| persia | (just be careful where you pause at 132) | 22:25 |
| *** salv-orl_ has joined #openstack-infra | 22:25 | |
| *** nadya has quit IRC | 22:25 | |
| fungi | 5-bit bytes little endian? | 22:25 |
| jeblair | fungi: or -32768 to 32767? | 22:25 |
| *** tosky has quit IRC | 22:25 | |
| fungi | jeblair: touché | 22:25 |
| persia | fungi: Yes. | 22:25 |
| jeblair | fungi: nice job on that accent | 22:26 |
| fungi | jeblair: it's compose,',e | 22:26 |
| lifeless | fungi: like I tell my daughter, 'no byting' | 22:26 |
| fungi | having a compose key is wonderful. no idea why it has an operating system logo on it | 22:27 |
| fungi | lifeless: ohhh, that's terrible | 22:27 |
| jeblair | lifeless: i assume she listens to that advice as well as you did :) | 22:27 |
| lifeless | jeblair: she has very good auditory filtering. | 22:27 |
| fungi | (actually, the operating system logo dead keys on my ducky shine are ducks instead of any popular operating system logo. that's nice) | 22:28 |
| lifeless | jeblair: words she hears: treats, tv, games, playball, ... | 22:28 |
| lifeless | jeblair: words she doesn't hear: jobs, bedtime, don't do that | 22:28 |
| fungi | jobs | 22:28 |
| *** salv-orlando has quit IRC | 22:28 | |
| jeblair | lifeless: your last line showed up blank for me | 22:28 |
| fungi | give her a few more years | 22:28 |
| lifeless | jeblair: :) | 22:28 |
| lifeless | fungi: jobs: cleaning her room, putting clean clothes away - small things that she owns :) | 22:29 |
| fungi | got it. i didn't have paying jobs until i neared double-digit years age | 22:29 |
| *** nadya has joined #openstack-infra | 22:29 | |
| anteaya | https://storyboard.openstack.org/#!/story/2000639 | 22:29 |
| jeblair | fungi: slacker | 22:30 |
| fungi | as far as the state of north carolina knows, i didn't have a paying job until i was legally allowed to at 14 | 22:30 |
| *** banix has joined #openstack-infra | 22:30 | |
| fungi | until then, computer repair was a "hobby" | 22:30 |
| lifeless | hobby with benefits? | 22:31 |
| fungi | currency benefits, right | 22:31 |
| openstackgerrit | James E. Blair proposed openstack-infra/system-config: Do not accept nodes on zlstatic01 https://review.openstack.org/327346 | 22:31 |
| *** tphummel has quit IRC | 22:31 | |
| *** mriedem has quit IRC | 22:31 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: use underscores where appropriate https://review.openstack.org/327333 | 22:32 |
| fungi | http://paste.openstack.org/show/509040 is the previous script with ~ a dozen previously missed users added | 22:32 |
| fungi | hopefully after this completes, https://wiki.openstack.org/wiki/Special:ListFiles?limit=500 will look reasonably spam-free | 22:33 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: remove deprecated hyphens https://review.openstack.org/327347 | 22:33 |
| *** tphummel has joined #openstack-infra | 22:33 | |
| anteaya | fungi: yay | 22:34 |
| *** timello has quit IRC | 22:34 | |
| *** nadya has quit IRC | 22:35 | |
| *** zz_dimtruck is now known as dimtruck | 22:35 | |
| jeblair | infra-root: anyone object to me deleting jenkins01? | 22:35 |
| jeblair | like, literally deleting the server | 22:35 |
| fungi | pleia2: so, if you're using mutt (you _are_ using mutt, right?) try set folder=imaps://infra-root\@openstack.org@secure.emailsrvr.com/inbox | 22:35 |
| pleia2 | fungi: perfect, thanks | 22:35 |
| fungi | jeblair: it's just a pile of insecure dead weight. go for it | 22:35 |
| * anteaya doesn't object on her way to find some dinner | 22:36 | |
| jeblair | (btw, i'm still trying to catch a openstackci-beaker-trusty job running from zl01) | 22:36 |
| fungi | okay, that last pass removed another ~130 file: pages | 22:37 |
| *** edmondsw has quit IRC | 22:37 | |
| *** claudiub has joined #openstack-infra | 22:37 | |
| fungi | if anybody sees any obvious spam uploads still sticking around at https://wiki.openstack.org/wiki/Special:ListFiles?limit=500 give me a heads up | 22:38 |
| *** nwkarsten has joined #openstack-infra | 22:38 | |
| *** nadya has joined #openstack-infra | 22:39 | |
| *** timello has joined #openstack-infra | 22:39 | |
| fungi | also, if you want to try tweaking the script at http://paste.openstack.org/show/509040 to remove non-file pages created by the same spammer users and need help, i'm happy to oblige | 22:39 |
| *** flepied has quit IRC | 22:41 | |
| anteaya | fungi: thank yiou | 22:41 |
| *** rlandy is now known as rlandy|bbl | 22:41 | |
| fungi | i came in well under the 9-hour continuous clicking estimate from this morning | 22:42 |
| *** nwkarsten has quit IRC | 22:43 | |
| JayF | TIL openstack.org was hosted on Rackspace Email; I worked on that for years | 22:43 |
| *** nadya has quit IRC | 22:43 | |
| anteaya | fungi: looks good to me | 22:43 |
| anteaya | fungi: plus your hands are in better shape too | 22:44 |
| *** fawadkhaliq has joined #openstack-infra | 22:44 | |
| *** kzaitsev_mb has quit IRC | 22:44 | |
| *** gordc has quit IRC | 22:44 | |
| *** kzaitsev_mb has joined #openstack-infra | 22:44 | |
| *** rbrndt has quit IRC | 22:45 | |
| *** salv-orl_ has quit IRC | 22:45 | |
| openstackgerrit | Merged openstack-infra/system-config: Decommission jenkins01 https://review.openstack.org/326749 | 22:45 |
| *** salv-orlando has joined #openstack-infra | 22:46 | |
| openstackgerrit | Merged openstack-infra/project-config: Add python-watcherclient doc on docs.openstack.org https://review.openstack.org/327107 | 22:47 |
| *** Jeffrey4l has joined #openstack-infra | 22:47 | |
| *** nadya has joined #openstack-infra | 22:48 | |
| *** mdrabe has quit IRC | 22:49 | |
| jhesketh | Morning | 22:49 |
| pleia2 | fungi: needed two lines: | 22:49 |
| pleia2 | set spoolfile="imaps://infra-root@openstack.org@secure.emailsrvr.com/inbox" | 22:49 |
| pleia2 | set folder="imaps://secure.emailsrvr.com/inbox" | 22:49 |
| pleia2 | (otherwise is looks for a spool file locally) | 22:50 |
| *** banix has quit IRC | 22:50 | |
| pleia2 | now I have lots of french email | 22:50 |
| anteaya | morning jhesketh | 22:50 |
| anteaya | pleia2: lucky you | 22:51 |
| *** tzn has joined #openstack-infra | 22:51 | |
| pleia2 | ovh email is in french :) | 22:51 |
| pleia2 | we'll need to tweak this or my filters some, I actually didn't keep inbox as the priority box, but filtered things into cloud/provider/ directories | 22:52 |
| *** nadya has quit IRC | 22:52 | |
| *** piet has joined #openstack-infra | 22:53 | |
| *** pradk has quit IRC | 22:54 | |
| *** kushal has quit IRC | 22:55 | |
| *** yolanda has quit IRC | 22:55 | |
| *** tzn has quit IRC | 22:56 | |
| *** amotoki has quit IRC | 22:56 | |
| *** yujunz has joined #openstack-infra | 22:57 | |
| *** nadya has joined #openstack-infra | 22:57 | |
| anteaya | pleia2: that would make sense | 22:58 |
| *** camunoz has joined #openstack-infra | 22:59 | |
| *** sdague has quit IRC | 22:59 | |
| *** yujunz has quit IRC | 23:00 | |
| *** banix has joined #openstack-infra | 23:00 | |
| *** amotoki has joined #openstack-infra | 23:02 | |
| *** PollyZ has quit IRC | 23:02 | |
| *** nadya has quit IRC | 23:02 | |
| *** hongbin has quit IRC | 23:03 | |
| *** timello has quit IRC | 23:03 | |
| fungi | pleia2: yeah, i was doing muttrc shorthand. i have that (and lots of other stuff) in my config | 23:04 |
| fungi | was mainly conveying the imaps:// url | 23:04 |
| fungi | and username format (i.e. full e-mail address) | 23:05 |
| *** yujunz has joined #openstack-infra | 23:05 | |
| pleia2 | yep, that was enough for me to figure it out :) | 23:05 |
| *** nadya has joined #openstack-infra | 23:06 | |
| *** annegentle has quit IRC | 23:06 | |
| *** vhosakot has joined #openstack-infra | 23:08 | |
| *** tkelsey has joined #openstack-infra | 23:08 | |
| auggy | hey there lovely infra folks! i'm trying to get the gate-wrap script in devstack-gate to run and i'm having some issues | 23:09 |
| auggy | https://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/README.rst#n203 | 23:10 |
| auggy | when the script runs it's failing on line 36 "source $WORKSPACE/devstack-gate/functions.sh" | 23:10 |
| auggy | but when i enter that in manually it sources that file just fine | 23:10 |
| *** nadya has quit IRC | 23:11 | |
| auggy | i'm logged in as the jenkins user per the instructions in the README and i've set all of my environment variables per the readme so I'm at a loss as to what to try next | 23:11 |
| pabelanger | jeblair: fungi: delete all the things | 23:11 |
| *** shashank_hegde has quit IRC | 23:11 | |
| *** bnemec has quit IRC | 23:11 | |
| fungi | auggy: no idea (yet) but what is the full error message? | 23:11 |
| pabelanger | pleia2: fungi suggested I update you about adding infra-root@openstack.org to the ic.openstack.org domain contact info, if you needed to make a filter | 23:12 |
| auggy | fungi: i'll paste the output of the gatewrap script into a pastie thing | 23:12 |
| fungi | auggy: perfect! | 23:12 |
| auggy | http://paste.openstack.org/show/509041/ | 23:12 |
| pleia2 | pabelanger: ic or ci? | 23:13 |
| *** tkelsey has quit IRC | 23:13 | |
| auggy | i'm guessing i'm missing a few things, this is a completely clean ubuntu install and i've just followed the readme instructions | 23:13 |
| *** markvoelker has quit IRC | 23:13 | |
| fungi | pleia2: ic ~= "infra-cloud" | 23:13 |
| pleia2 | fungi: aah | 23:14 |
| openstackgerrit | Merged openstack/diskimage-builder: Move pkg-map to dib-python https://review.openstack.org/326228 | 23:14 |
| *** vhosakot has quit IRC | 23:14 | |
| *** jgriffith is now known as jgriffith_away | 23:14 | |
| pleia2 | pabelanger: great, who/what would be sending mail to this? | 23:14 |
| *** shashank_hegde has joined #openstack-infra | 23:14 | |
| *** vhosakot has joined #openstack-infra | 23:14 | |
| fungi | pleia2: probably only people who decide to try and reach the responsible party for that domain... ~=nobody | 23:15 |
| *** gouthamr_ has joined #openstack-infra | 23:15 | |
| *** kzaitsev_mb has quit IRC | 23:15 | |
| pleia2 | alright, for now I'll be aware that it's another thing going to it, we'll consider if there's a filtering mechanism later | 23:16 |
| *** timello has joined #openstack-infra | 23:16 | |
| pabelanger | right, what fungi said | 23:17 |
| pabelanger | I don't expect much traffic to it | 23:17 |
| *** fawadkhaliq has quit IRC | 23:17 | |
| pabelanger | jeblair: http://grafana.openstack.org/dashboard/db/nodepool start to update with zl01 stats | 23:18 |
| pabelanger | still missing building nodes however | 23:18 |
| *** fawadkhaliq has joined #openstack-infra | 23:18 | |
| jeblair | pabelanger: there won't ever be any, we should probably remove that one | 23:18 |
| *** gouthamr has quit IRC | 23:18 | |
| jeblair | pabelanger: (nodes only get assigned to zuul launchers when ready) | 23:19 |
| fungi | auggy: i would start with skeptical questioning of that first line of output... "./safe-devstack-vm-gate-wrap.sh: line 36: /devstack-gate/functions.sh: No such file or directory" | 23:19 |
| *** bnemec has joined #openstack-infra | 23:19 | |
| anteaya | pleia2: you missed the vanilla ic joke earlier | 23:20 |
| jeblair | auggy: are you aware of reproduce.sh? | 23:20 |
| *** yujunz has quit IRC | 23:20 | |
| fungi | auggy: i think it does really need functions from that file | 23:20 |
| pabelanger | jeblair: Oh, interesting. Is there a (better) way to expose the building information? Moving forward? | 23:20 |
| auggy | fungi: ok, when i checked echo $WORKSPACE it shows the testing directory | 23:20 |
| jeblair | pabelanger: both building graphs above it will still be correct | 23:20 |
| fungi | auggy: and the leading / makes me suspect a questionable relative directory somewhere | 23:20 |
| auggy | but for some reason maybe the script isn't getting it and trying to do /devstack-gate.. | 23:20 |
| auggy | jeblair: no i haven't | 23:21 |
| pabelanger | jeblair: right, it would be based on target, not manager | 23:21 |
| fungi | auggy: might be sudo sanitizing the calling environment? | 23:21 |
| pabelanger | okay, I can update | 23:21 |
| auggy | fungi: hrm, i'm not running hte script as sudo, and it doesn't appear to be sudo'ing in the script | 23:21 |
| auggy | that was my thought too | 23:21 |
| jeblair | auggy: it's created by devstack-gate and published along with each build's log. you can theoretically use it to reproduce a given devstack-gate run | 23:21 |
| *** fawadkhaliq has quit IRC | 23:21 | |
| *** nwkarsten has joined #openstack-infra | 23:22 | |
| auggy | jeblair: ah ok, my goal here is to create a devstack environment just like what the gate job is using | 23:22 |
| auggy | so i don't need the test run, i need the devstack config | 23:22 |
| fungi | auggy: yeah, shot in the dark, but i agree with jeblair that comparing to what an arbitrary reproduce.sh runs might provide some additional insight | 23:23 |
| auggy | fungi: jeblair: ok, i'll look at reproduce.sh | 23:23 |
| jeblair | http://logs.openstack.org/70/283770/34/gate/gate-tempest-dsvm-full/0bb4ca8/logs/reproduce.sh | 23:23 |
| auggy | what repo is that in? | 23:23 |
| jeblair | there's a random one i pulled off the status page | 23:24 |
| fungi | auggy: it's dynamically generated from every devstack-gate execution | 23:24 |
| auggy | ahhh ok | 23:24 |
| *** nadya has joined #openstack-infra | 23:24 | |
| auggy | ok, yeah i just copied the gate yaml | 23:24 |
| *** timello has quit IRC | 23:24 | |
| auggy | i put that into a sh script to run the gate-wrap script, but then tried doing each line manually when it failed | 23:25 |
| *** vhosakot has quit IRC | 23:25 | |
| jeblair | auggy: did you export WORKSPACE ? | 23:25 |
| auggy | well not the yaml but that section of the yaml with the setup steps, per the readme | 23:25 |
| auggy | yes i did | 23:25 |
| auggy | jenkins@devstack-controller-node:~/workspace/testing/devstack-gate$ echo $WORKSPACE | 23:26 |
| *** vhosakot has joined #openstack-infra | 23:26 | |
| auggy | /home/jenkins/workspace/testing | 23:26 |
| auggy | i'll try the reproduce, i'm guessing some things are missing which is what's causin this | 23:27 |
| *** johnny_ has quit IRC | 23:27 | |
| fungi | but you did an actual export statement (so it'll be inherited by child processes) and not just a regular variable assignment? | 23:27 |
| *** pahuang has quit IRC | 23:28 | |
| fungi | also, ubuntu 14.04 lts? | 23:28 |
| auggy | yes i did | 23:28 |
| *** nadya has quit IRC | 23:29 | |
| auggy | let me check, i might be doing ubuntu 16.04 | 23:29 |
| auggy | i started on 14.04 and then upgraded when i was running into issues thinking something was out of date | 23:29 |
| fungi | if you're doing 16.04, fun you may want to check in with the qa team and make sure stuff is actually working there under devstack | 23:29 |
| fungi | we're not using it for most jobs ye | 23:30 |
| fungi | t | 23:30 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: WIP: Add option to include run_id test_runs by run meta call https://review.openstack.org/327411 | 23:30 |
| auggy | i was running into the same issues on 14.04 this was just a shot in the dark | 23:30 |
| auggy | thanks fungi | 23:31 |
| fungi | yeah, i doubt the distro version has anything to do with your current issue | 23:31 |
| fungi | just general cautionary response | 23:31 |
| auggy | yeah me either, i was just grasping at straws ;) | 23:31 |
| *** pahuang has joined #openstack-infra | 23:33 | |
| auggy | fungi: jeblair: reproduce.sh seems to be working | 23:33 |
| auggy | so some step was missing.. i just have no idea what ;) | 23:33 |
| auggy | i'm doing this one from an old patch i had lying around - http://logs.openstack.org/70/283770/34/gate/gate-tempest-dsvm-full/0bb4ca8/logs/reproduce.sh | 23:33 |
| fungi | auggy: if you end up spotting what's missing in the d-g readme, i'll happily update it | 23:34 |
| *** amitgandhinz has joined #openstack-infra | 23:34 | |
| auggy | fungi: yeah i'll look and see if there's anything obvious, otherwise i'm happy to submit a patch referencing reproduce.sh | 23:34 |
| fungi | auggy: also appreciated! | 23:34 |
| *** banix has quit IRC | 23:35 | |
| auggy | yeah i figure you all are doing so much already ;) | 23:35 |
| fungi | never enough, it seems | 23:35 |
| fungi | though at the moment, i have cookies, beer and cartoons | 23:35 |
| fungi | so i'm probably done with less serious work today | 23:36 |
| *** claudiub has quit IRC | 23:38 | |
| auggy | hahahah | 23:38 |
| auggy | nice | 23:38 |
| *** ffrank has quit IRC | 23:39 | |
| *** Goneri has quit IRC | 23:40 | |
| auggy | thanks for the help! | 23:40 |
| fungi | i'm still around if you need any other advice of questionable provenance | 23:41 |
| auggy | hahaha | 23:41 |
| ianw | auggy: xenial is reported working for devstack | 23:41 |
| auggy | ianw: oh cool, thanks | 23:41 |
| fungi | ianw: that's excellent news! | 23:41 |
| ianw | but obviously there's a lot of knobs, so parts might still be broken ... but in general ok | 23:41 |
| auggy | ianw: yeah i'm not doing anything too tricky, basically just trying to get the current devstack config for a grenade job | 23:42 |
| *** nadya has joined #openstack-infra | 23:42 | |
| ianw | auggy: ok ... as long as you're not getting mikata branch involved in grenade; that branch happened before xenial so it will probably error out there saying distro not supported | 23:43 |
| *** dingyichen has joined #openstack-infra | 23:43 | |
| auggy | ianw: nope this will all be newton | 23:43 |
| auggy | ianw: basically i need to create a new grenade job to test nova cells v2 simple-cells-setup | 23:44 |
| *** markvoelker has joined #openstack-infra | 23:44 | |
| auggy | ianw: so i'm trying to get a base devstack i can apply the change to, then once that's successful get the updated devstack configs so i can set that up as the grenade target | 23:44 |
| auggy | so then we can have a grenade test for the simple cells setup, from newton to newton | 23:45 |
| auggy | i'm hoping that makes sense, this is totally uncharted territory for me :) | 23:46 |
| fungi | auggy: you might also find help in the #openstack-qa channel. the north american crowd are dwindling at this time of day, but oomichi and the apac crowd are probably well-caffeinated by now | 23:46 |
| auggy | haha thanks fungi | 23:46 |
| *** timello has joined #openstack-infra | 23:47 | |
| *** Sukhdev has joined #openstack-infra | 23:47 | |
| *** amitgandhinz has quit IRC | 23:47 | |
| fungi | especially if you get deeper into devstack and grenade details | 23:47 |
| *** nadya has quit IRC | 23:47 | |
| oomichi | fungi: I am living in USA now, so my working time is almost end today ;) | 23:48 |
| auggy | oomichi: i think we're on the same timezone :) us/pacific? | 23:48 |
| *** gildub has joined #openstack-infra | 23:48 | |
| oomichi | auggy: oh, yeah. that is the same :) | 23:48 |
| *** markvoelker has quit IRC | 23:49 | |
| fungi | oomichi: oh, i didn't realize you'd moved! regardless, there are some apac qa contributors with understanding of devstack and grenade who are probably around now | 23:49 |
| auggy | good to know! i'll ask in there if/when i run into issues | 23:50 |
| auggy | i know i will have a pile of grenade questions when i get there ;) | 23:50 |
| oomichi | fungi: yeah, that is right. good morning for them now | 23:50 |
| oomichi | auggy: heh, ok. I need to be ready ;) | 23:51 |
| auggy | oomichi: ha :) | 23:51 |
| fungi | auggy: yep, i hesitate to say it, but devstack-gate is probably the "easy" part of your quest | 23:51 |
| *** nwkarsten has quit IRC | 23:51 | |
| auggy | fungi: it's fine, there's a reason no one had volunteered for this particular task :P | 23:51 |
| *** nadya has joined #openstack-infra | 23:51 | |
| * auggy chalks it up to gaining OpenStack XP points | 23:52 | |
| *** nwkarsten has joined #openstack-infra | 23:52 | |
| fungi | i always say i'm banking my karma for my next incarnation | 23:52 |
| auggy | i'm actually curious to see if the bugs team could use devstack-gate to set up devstack environments for reproducing bugs | 23:52 |
| auggy | the nova bugs team i mean | 23:52 |
| auggy | sometimes people have special setups and if we could just use devstack-gate to recreate an environment that's close enough, that would save people a lot of time and possibly make bug triage go a bit faster | 23:53 |
| *** rhallisey has joined #openstack-infra | 23:53 | |
| fungi | auggy: possibly. that readme in the d-g repo grew out of the vmt's need to test patches proposed in private for embargoed security vulnerabilities | 23:53 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Fix building nodes panel for zuul launchers https://review.openstack.org/327421 | 23:54 |
| auggy | oh interesting | 23:54 |
| fungi | rather, the reproduction section of that readme did | 23:54 |
| auggy | yeah | 23:54 |
| auggy | holy cow, it did a thing and it worked | 23:54 |
| auggy | ;D | 23:54 |
| auggy | PROGRESS! | 23:54 |
| * auggy does cartwheels | 23:55 | |
| fungi | deific bovines indeed! | 23:55 |
| auggy | :D | 23:55 |
| *** nadya has quit IRC | 23:56 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: add private gearman function https://review.openstack.org/327422 | 23:57 |
| jeblair | fungi, pabelanger, Shrews, notmorgan: ^ that may be a band-aid we can apply to zuulv2 to keep us going until zuulv3. however, we are running out of band-aids. :) | 23:58 |
| fungi | even our secret stash of snoopy band-aids? | 23:58 |
| *** banix has joined #openstack-infra | 23:59 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Ansible launcher: add private gearman function https://review.openstack.org/327422 | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!