| openstackgerrit | Paul Belanger proposed openstack/diskimage-builder master: [WIP] Add DIB_DISTRIBUTION_MIRROR support for centos-minimal https://review.openstack.org/455852 | 00:01 |
|---|---|---|
| pabelanger | unbound is running in your job | 00:01 |
| kfox1111 | yeah. | 00:01 |
| *** esikachev has joined #openstack-infra | 00:01 | |
| ianw | rm_work: ooohhhh ... Collecting diskimage-builder===2.2.0 (from -c /opt/stack/new/requirements/upper-constraints.txt (line 129)) | 00:02 |
| ianw | needs to be 2.3.0 | 00:02 |
| clarkb | do you collect any data from the subnode? | 00:03 |
| clarkb | all the logs seem to be for the primary node? | 00:03 |
| *** sflanigan has quit IRC | 00:04 | |
| pabelanger | http://logs.openstack.org/41/454841/8/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-multi-nv/50ae7d9/console.html#_2017-04-11_16_54_27_994332 | 00:04 |
| pabelanger | that expected? | 00:04 |
| *** pahuang has quit IRC | 00:05 | |
| pabelanger | and http://logs.openstack.org/41/454841/8/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-multi-nv/50ae7d9/console.html#_2017-04-11_16_54_36_132949 | 00:05 |
| pabelanger | nameservers are not localhost | 00:05 |
| *** esikachev has quit IRC | 00:05 | |
| pabelanger | also, need to work on that glean config file to disable dns updates | 00:06 |
| kfox1111 | pabelanger: yeah. I still need to fix that. but a known issue. | 00:06 |
| clarkb | the subnode appears to be heartbeating in nodes.yaml just fine until the end of the job | 00:07 |
| rm_work | ianw: aah so we just need to update g-r | 00:08 |
| rm_work | to fix their u-c | 00:08 |
| clarkb | http://logs.openstack.org/41/454841/8/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-multi-nv/50ae7d9/logs/pods/default-test-dns-centos-7-2-node-rax-iad-8375909-522920-qgv49.txt is the one that failed | 00:08 |
| *** thorst has quit IRC | 00:09 | |
| clarkb | kfox1111: what is 172.16.128.1:443 ? | 00:09 |
| kfox1111 | I believe thats kubernetes's api. | 00:10 |
| clarkb | ok that makes sense given the url, and thats running on the primary node? | 00:11 |
| kfox1111 | yeah. | 00:11 |
| clarkb | what provides the overlay l2 network for that /24? | 00:11 |
| kfox1111 | two pieces. kube-proxy deals with the /24 service network. and flannel deals with the /16 | 00:12 |
| kfox1111 | pod network. | 00:12 |
| clarkb | sorry let me reprhase how are you overlaying that l2 | 00:13 |
| clarkb | vxlan? gre? | 00:13 |
| clarkb | eg how do packets get from subnode to primary node in order to make that http connection to 172.16.128.1:443 succeed | 00:14 |
| ianw | rm_work: yep, after pip3 installing git it seems to work | 00:14 |
| kfox1111 | flannel is using vxlan's between the two hosts. | 00:14 |
| kfox1111 | let me double check that address... | 00:14 |
| kfox1111 | yeah. thats the service ip space. | 00:15 |
| kfox1111 | so, thing on slave contacts 172.16.128.1:443, | 00:15 |
| kfox1111 | it goes through iptables through the rules kube-proxy wrote for that endpointk, and it gets rewritten with the ip adress of the kube-apiserver running on the master node. | 00:15 |
| kfox1111 | in this case, the hosts own ip address, so no overlay network at all. | 00:16 |
| clarkb | so 172.16.128.1:443 is NAT'd to primary_node_realip:6443? | 00:17 |
| kfox1111 | yeah. | 00:17 |
| clarkb | woo its metadata server all over again. ok | 00:17 |
| kfox1111 | hehe. | 00:17 |
| kfox1111 | yeah. | 00:17 |
| *** sflanigan has joined #openstack-infra | 00:17 | |
| *** sflanigan has joined #openstack-infra | 00:17 | |
| clarkb | literally the worst design ever and everyone uses it apparently | 00:17 |
| clarkb | with that out of the way, I would double check the NAT rules first and then general firewall rules to make sure packets are allowed to all the addrs and ports in question | 00:18 |
| clarkb | 10.210.34.86:6443 looks like thats what it will be NATed to | 00:19 |
| kfox1111 | http://logs.openstack.org/41/454841/8/check/gate-kolla-kubernetes-deploy-centos-binary-3-ceph-multi-nv/0fa9c5c/logs/iptables.txt | 00:19 |
| clarkb | but I don't think we have any subnode routing table or iptables info | 00:19 |
| clarkb | all the info we have is for primary | 00:19 |
| kfox1111 | on the master. | 00:19 |
| kfox1111 | http://logs.openstack.org/41/454841/8/check/gate-kolla-kubernetes-deploy-centos-binary-3-ceph-multi-nv/0fa9c5c/logs/iptables-10.211.2.162.txt on the slave. | 00:19 |
| kfox1111 | I collect a lot of logs. :) | 00:20 |
| kfox1111 | I don't have slave routes, but the parent is: http://logs.openstack.org/41/454841/8/check/gate-kolla-kubernetes-deploy-centos-binary-3-ceph-multi-nv/0fa9c5c/logs/routes.txt | 00:20 |
| *** jkilpatr_ has quit IRC | 00:21 | |
| clarkb | reading the nat rule for the primary at least (which won't affect subnode nat) its 172.16.128.1/32 port 443 to 10.210.34.86/32 port 6443 so that side looks ok | 00:22 |
| kfox1111 | hmm.... eth1's routes are weird. | 00:22 |
| *** pahuang has joined #openstack-infra | 00:22 | |
| clarkb | kfox1111: according to the config drive http://logs.openstack.org/41/454841/8/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-multi-nv/50ae7d9/console.html#_2017-04-11_16_53_57_800613 are the routes you should have on eth1 | 00:24 |
| clarkb | http://logs.openstack.org/41/454841/8/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-multi-nv/50ae7d9/console.html#_2017-04-11_16_53_57_819996 and they are configured properly at job start | 00:25 |
| kfox1111 | yeah. | 00:25 |
| clarkb | so something is breaking your default gateway for those networks | 00:25 |
| kfox1111 | sec... | 00:25 |
| clarkb | oh wait thats a different job | 00:26 |
| clarkb | streams got crossed | 00:26 |
| kfox1111 | oh. sorry. | 00:26 |
| clarkb | http://logs.openstack.org/41/454841/8/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-multi-nv/50ae7d9/logs/routes.txt looks ok and lines up with the info I linked to above | 00:26 |
| kfox1111 | k. | 00:26 |
| clarkb | kfox1111: thats the private interface so rax supplied a set of routes to be able to hit the other hosts private interfaces | 00:27 |
| *** esikachev has joined #openstack-infra | 00:27 | |
| kfox1111 | ah. yeah. | 00:27 |
| clarkb | that is the interface that traffic to 10.210.34.86/32 would go over so possibly involved but routes look ok | 00:27 |
| kfox1111 | I didnt realize they had multiple subnets hanging off of it. | 00:27 |
| clarkb | I think a good next step would be getting the iptables, route, and interface info off the subnode(s) too | 00:28 |
| kfox1111 | hmm... | 00:28 |
| kfox1111 | finally got one to pass. | 00:28 |
| kfox1111 | osic. | 00:28 |
| kfox1111 | so it does seem tied to rax somehow. | 00:28 |
| kfox1111 | http://logs.openstack.org/41/454841/10/check/gate-kolla-kubernetes-deploy-centos-binary-2-ceph-multi-nv/a0f563a/console.html | 00:28 |
| *** armax has quit IRC | 00:29 | |
| clarkb | well networking is different in rax, so if this is a networking issue like ip conflict or route problem its possible it would be restricted to a specific cloud | 00:29 |
| kfox1111 | yeah. thats kind of what I was thinking. | 00:29 |
| kfox1111 | helps narrow down the possible causes. | 00:29 |
| *** rfolco has joined #openstack-infra | 00:30 | |
| *** john-davidge has joined #openstack-infra | 00:30 | |
| kfox1111 | and another fail... rax again http://logs.openstack.org/41/454841/10/check/gate-kolla-kubernetes-deploy-centos-binary-3-ceph-multi-nv/c3e8590/console.html | 00:30 |
| *** cody-somerville has joined #openstack-infra | 00:31 | |
| *** esikachev has quit IRC | 00:31 | |
| *** jkilpatr_ has joined #openstack-infra | 00:31 | |
| clarkb | reading through logs it appears a bunch of other containers do successfully start as well | 00:32 |
| kfox1111 | yeah. so far in the run, it starts up all of k8s on the primary, | 00:32 |
| kfox1111 | calico/flannel seems to maybe launch on both nodes. | 00:32 |
| kfox1111 | kube-proxy on both nodes, | 00:32 |
| kfox1111 | then the first real container we start is the test-dns pods. one per host. | 00:32 |
| kfox1111 | its working on the primary. failing to launch on the slave. | 00:32 |
| kfox1111 | the pod log files are in /logs/pods | 00:33 |
| clarkb | ya looking at those its easy to tell where the dns test pods launch because of their names, but how do you tell where the others launch? | 00:33 |
| *** csomerville has quit IRC | 00:34 | |
| kfox1111 | there are two types of log files in there. | 00:34 |
| kfox1111 | ones with an extra -containername.txt are the individulal logs in the pod. | 00:34 |
| kfox1111 | the ones without the container name are the 'kubectl describe pod' output. | 00:34 |
| *** john-davidge has quit IRC | 00:34 | |
| kfox1111 | with the lifecycle of the pod. | 00:34 |
| kfox1111 | so, like: | 00:34 |
| kfox1111 | http://logs.openstack.org/41/454841/8/check/gate-kolla-kubernetes-deploy-centos-binary-3-ceph-multi-nv/0fa9c5c/logs/pods/default-test-dns-centos-7-2-node-rax-ord-8375918-522923-8mc8w.txt | 00:34 |
| *** thorst has joined #openstack-infra | 00:35 | |
| kfox1111 | the error on it is very strange though. havent see it before. | 00:35 |
| kfox1111 | seems like its getting scheduled an ip, but TeardownNetworkError is unusual. | 00:35 |
| clarkb | I think the failed to setup network one is likely the underlying issue | 00:36 |
| kfox1111 | ah. but this is probably it:Get https://172.16.128.1:443/api/v1/namespaces/default/pods/test-dns-centos-7-2-node-rax-ord-8375918-522923-8mc8w: dial tcp 172.16.128.1:443: i/o timeout" | 00:36 |
| clarkb | and the i/o timeout. Basically its trying to configure networking and can't bceause https not working | 00:36 |
| kfox1111 | so its trying to reach across and failing. | 00:36 |
| kfox1111 | yeah. | 00:36 |
| clarkb | and it looks like it might be the only pod that actually starts on the subnode | 00:37 |
| clarkb | at least up to that point | 00:37 |
| kfox1111 | its the only pod using the sdn. | 00:38 |
| clarkb | so ya I think we need to see what networking is like on the subnode | 00:38 |
| kfox1111 | k. I'll add some extra dumping of slave state. | 00:38 |
| kfox1111 | gotta head out for a bit. thanks for the help. :) | 00:38 |
| *** kgiusti has quit IRC | 00:39 | |
| clarkb | no problem. Definitely look at the iptables, routes, and interfaces on subnode to see if anything is off there that would prevent traffic making it to k8s api service | 00:39 |
| *** zhurong has joined #openstack-infra | 00:39 | |
| *** armax has joined #openstack-infra | 00:41 | |
| *** Apoorva_ has quit IRC | 00:42 | |
| *** Apoorva has joined #openstack-infra | 00:43 | |
| openstackgerrit | Paul Belanger proposed openstack/diskimage-builder master: [WIP] Add DIB_DISTRIBUTION_MIRROR support for centos-minimal https://review.openstack.org/455852 | 00:43 |
| *** thorst has quit IRC | 00:46 | |
| *** adriant has joined #openstack-infra | 00:52 | |
| *** Goneri has quit IRC | 00:54 | |
| openstackgerrit | Paul Belanger proposed openstack/diskimage-builder master: [WIP] Add DIB_DISTRIBUTION_MIRROR support for centos-minimal https://review.openstack.org/455852 | 00:54 |
| *** mdrabe has joined #openstack-infra | 00:56 | |
| *** pvaneck has quit IRC | 00:58 | |
| openstackgerrit | Paul Belanger proposed openstack/diskimage-builder master: [WIP] Add DIB_DISTRIBUTION_MIRROR support for centos-minimal https://review.openstack.org/455852 | 00:59 |
| *** armax has quit IRC | 01:08 | |
| *** esikachev has joined #openstack-infra | 01:08 | |
| *** smarcet has joined #openstack-infra | 01:16 | |
| *** esikachev has quit IRC | 01:17 | |
| *** thorst has joined #openstack-infra | 01:17 | |
| *** liujiong has joined #openstack-infra | 01:18 | |
| *** esikachev has joined #openstack-infra | 01:20 | |
| *** thorst has quit IRC | 01:24 | |
| *** esikachev has quit IRC | 01:25 | |
| *** MasterOfBugs has quit IRC | 01:25 | |
| *** pramodrj07 has quit IRC | 01:25 | |
| *** smarcet has quit IRC | 01:26 | |
| *** ricardoas has quit IRC | 01:28 | |
| *** john-davidge has joined #openstack-infra | 01:31 | |
| *** gcb has joined #openstack-infra | 01:32 | |
| *** yanyanhu has joined #openstack-infra | 01:35 | |
| *** tphummel has quit IRC | 01:35 | |
| *** john-davidge has quit IRC | 01:35 | |
| mriedem | pbr 2.1.0 is breaking some docs and specs repos https://bugs.launchpad.net/pbr/+bug/1681983 | 01:36 |
| openstack | Launchpad bug 1681983 in PBR "2.1.0: sphinx.errors.SphinxWarning: WARNING: no "man_pages" config value found; no manual pages will be written" [Undecided,New] | 01:36 |
| *** smarcet has joined #openstack-infra | 01:41 | |
| *** rderose_ has quit IRC | 01:42 | |
| *** esikachev has joined #openstack-infra | 01:42 | |
| mordred | mriedem: and the behavior you thing is more correct is for it to not break repos, right? | 01:44 |
| mordred | mriedem: for now, if you comment out that man_pages = [] line, it should 'fix' the issue | 01:44 |
| *** smarcet has quit IRC | 01:45 | |
| mordred | mriedem: (I just approved the blacklist patch - just wanted to make sure you knew of the workaround in case that's useful) | 01:46 |
| *** esikachev has quit IRC | 01:47 | |
| *** Sukhdev has quit IRC | 01:47 | |
| *** ricardoas has joined #openstack-infra | 01:52 | |
| *** esikachev has joined #openstack-infra | 01:54 | |
| *** s-shiono_ has joined #openstack-infra | 01:54 | |
| *** sflanigan has quit IRC | 01:56 | |
| *** s-shiono has quit IRC | 01:57 | |
| *** esikachev has quit IRC | 02:00 | |
| *** esikachev has joined #openstack-infra | 02:00 | |
| *** wolverineav has quit IRC | 02:04 | |
| *** jamielennox is now known as jamielennox|away | 02:07 | |
| *** esikachev has quit IRC | 02:08 | |
| *** rfolco has quit IRC | 02:08 | |
| *** gongysh has joined #openstack-infra | 02:09 | |
| *** Apoorva_ has joined #openstack-infra | 02:09 | |
| *** esikachev has joined #openstack-infra | 02:09 | |
| *** fandi has joined #openstack-infra | 02:10 | |
| *** sflanigan has joined #openstack-infra | 02:10 | |
| *** sflanigan has joined #openstack-infra | 02:10 | |
| *** mdrabe has quit IRC | 02:12 | |
| *** Apoorva has quit IRC | 02:13 | |
| *** Apoorva_ has quit IRC | 02:14 | |
| *** esikachev has quit IRC | 02:14 | |
| *** cody-somerville has quit IRC | 02:14 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Skip python3-virtualenv on <= trusty https://review.openstack.org/455816 | 02:17 |
| *** thorst has joined #openstack-infra | 02:21 | |
| *** Julien-zte has joined #openstack-infra | 02:26 | |
| *** mriedem1 has joined #openstack-infra | 02:30 | |
| mriedem1 | mordred: thanks, i didn't know if there was a workaround | 02:30 |
| *** mriedem has quit IRC | 02:32 | |
| *** esikachev has joined #openstack-infra | 02:33 | |
| *** mriedem1 is now known as mriedem | 02:36 | |
| *** esikachev has quit IRC | 02:38 | |
| *** thorst has quit IRC | 02:41 | |
| *** Julien-zte has quit IRC | 02:43 | |
| *** Julien-zte has joined #openstack-infra | 02:44 | |
| *** gouthamr has quit IRC | 02:44 | |
| *** erikmwilson has joined #openstack-infra | 02:47 | |
| *** erikmwil_ has quit IRC | 02:47 | |
| *** chason has joined #openstack-infra | 02:47 | |
| *** Julien-zte has quit IRC | 02:47 | |
| *** Julien-zte has joined #openstack-infra | 02:47 | |
| chason | Hi, is there anybody know the IRC channel of "keystone-specs" ? | 02:48 |
| *** esikachev has joined #openstack-infra | 02:49 | |
| *** gildub has quit IRC | 02:49 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: TEST: DONT RECHECK or REVIEW: periodic jobs https://review.openstack.org/359215 | 02:53 |
| *** esikachev has quit IRC | 02:53 | |
| *** Kevin_Zheng has joined #openstack-infra | 02:54 | |
| *** Julien-zte has quit IRC | 02:56 | |
| *** wolverineav has joined #openstack-infra | 02:56 | |
| *** Julien-zte has joined #openstack-infra | 02:56 | |
| *** esikachev has joined #openstack-infra | 02:56 | |
| *** Apoorva has joined #openstack-infra | 02:58 | |
| *** vikrant has joined #openstack-infra | 02:59 | |
| *** smarcet has joined #openstack-infra | 03:00 | |
| *** smarcet has quit IRC | 03:00 | |
| *** esikachev has quit IRC | 03:00 | |
| *** chlong has joined #openstack-infra | 03:03 | |
| *** chlong_ has joined #openstack-infra | 03:03 | |
| *** nicolasbock has quit IRC | 03:03 | |
| *** wolverineav has quit IRC | 03:07 | |
| *** Julien-zte has quit IRC | 03:07 | |
| *** Julien-zte has joined #openstack-infra | 03:08 | |
| *** tphummel has joined #openstack-infra | 03:08 | |
| *** esikachev has joined #openstack-infra | 03:09 | |
| *** wolverineav has joined #openstack-infra | 03:11 | |
| *** vikrant is now known as vikrant|brb | 03:13 | |
| *** esikachev has quit IRC | 03:14 | |
| *** Julien-zte has quit IRC | 03:17 | |
| *** Julien-zte has joined #openstack-infra | 03:18 | |
| *** esikachev has joined #openstack-infra | 03:18 | |
| *** gongysh has quit IRC | 03:21 | |
| *** esikachev has quit IRC | 03:23 | |
| *** jamielennox|away is now known as jamielennox | 03:23 | |
| *** wolverineav has quit IRC | 03:23 | |
| *** wolverineav has joined #openstack-infra | 03:24 | |
| *** Apoorva has quit IRC | 03:24 | |
| *** vikrant|brb is now known as vikrant | 03:25 | |
| *** Julien-zte has quit IRC | 03:26 | |
| *** Julien-zte has joined #openstack-infra | 03:27 | |
| gmann | AJaeger: i did not get about removal from jenkin/jobs - https://review.openstack.org/#/c/455637/ | 03:28 |
| *** wolverineav has quit IRC | 03:28 | |
| *** Julien-zte has quit IRC | 03:31 | |
| *** Julien-zte has joined #openstack-infra | 03:32 | |
| *** esikachev has joined #openstack-infra | 03:33 | |
| *** gongysh has joined #openstack-infra | 03:35 | |
| *** sflanigan has quit IRC | 03:35 | |
| *** esikachev has quit IRC | 03:38 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid master: Updated claims on OpenId/OAuth https://review.openstack.org/455895 | 03:38 |
| *** armax has joined #openstack-infra | 03:41 | |
| *** esikachev has joined #openstack-infra | 03:41 | |
| *** Julien-zte has quit IRC | 03:44 | |
| *** wolverineav has joined #openstack-infra | 03:44 | |
| openstackgerrit | Merged openstack-infra/openstackid master: Updated claims on OpenId/OAuth https://review.openstack.org/455895 | 03:45 |
| *** wolverineav has quit IRC | 03:45 | |
| *** wolverin_ has joined #openstack-infra | 03:45 | |
| *** Julien-zte has joined #openstack-infra | 03:45 | |
| *** krtaylor has quit IRC | 03:45 | |
| *** esikachev has quit IRC | 03:46 | |
| *** sflanigan has joined #openstack-infra | 03:48 | |
| *** sflanigan has joined #openstack-infra | 03:48 | |
| *** mriedem has quit IRC | 03:48 | |
| ianw | pabelanger: 2.3.2 pushed, only contains the trusty venv fix | 03:51 |
| *** Julien-zte has quit IRC | 03:51 | |
| *** Julien-zte has joined #openstack-infra | 03:52 | |
| *** baoli has quit IRC | 03:53 | |
| *** dave-mccowan has joined #openstack-infra | 03:57 | |
| *** john-davidge has joined #openstack-infra | 03:58 | |
| *** Julien-zte has quit IRC | 03:58 | |
| *** Julien-zte has joined #openstack-infra | 03:59 | |
| *** rcernin has joined #openstack-infra | 03:59 | |
| *** rcernin is now known as rcernin|wfh|mtg | 03:59 | |
| *** armax has quit IRC | 04:01 | |
| *** armax has joined #openstack-infra | 04:02 | |
| *** john-davidge has quit IRC | 04:03 | |
| *** links has joined #openstack-infra | 04:03 | |
| *** esikachev has joined #openstack-infra | 04:05 | |
| *** zhurong has quit IRC | 04:06 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-dev/hacking master: Updated from global requirements https://review.openstack.org/440029 | 04:09 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/shade master: Updated from global requirements https://review.openstack.org/455898 | 04:09 |
| *** esikachev has quit IRC | 04:10 | |
| *** esikachev has joined #openstack-infra | 04:10 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/diskimage-builder master: Updated from global requirements https://review.openstack.org/455909 | 04:10 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Clean up dib-python symlink https://review.openstack.org/455486 | 04:12 |
| *** Julien-zte has quit IRC | 04:13 | |
| *** Julien-zte has joined #openstack-infra | 04:13 | |
| *** Julien-zte has joined #openstack-infra | 04:14 | |
| *** esikachev has quit IRC | 04:15 | |
| openstackgerrit | Tuan Luong-Anh proposed openstack/diskimage-builder master: Add support for building images capable of UEFI https://review.openstack.org/287784 | 04:15 |
| *** dave-mccowan has quit IRC | 04:16 | |
| *** esikachev has joined #openstack-infra | 04:16 | |
| *** gildub has joined #openstack-infra | 04:16 | |
| *** dave-mccowan has joined #openstack-infra | 04:16 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/os-testr master: Updated from global requirements https://review.openstack.org/454511 | 04:19 |
| *** esikachev has quit IRC | 04:21 | |
| *** esikachev has joined #openstack-infra | 04:23 | |
| *** wolverin_ has quit IRC | 04:26 | |
| *** wolverineav has joined #openstack-infra | 04:27 | |
| *** esikachev has quit IRC | 04:27 | |
| *** tphummel has quit IRC | 04:27 | |
| *** wolverineav has quit IRC | 04:31 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: Clean up dib-python symlink https://review.openstack.org/455486 | 04:34 |
| *** zhurong has joined #openstack-infra | 04:34 | |
| *** psachin has joined #openstack-infra | 04:35 | |
| *** Julien-zte has quit IRC | 04:36 | |
| *** thorst has joined #openstack-infra | 04:37 | |
| *** yamamoto has quit IRC | 04:38 | |
| *** Julien-zte has joined #openstack-infra | 04:38 | |
| *** yamamoto has joined #openstack-infra | 04:41 | |
| *** thorst has quit IRC | 04:42 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-manual master: Improve rebase section https://review.openstack.org/456044 | 04:46 |
| AJaeger | clarkb, this addresses most of your comments ^ | 04:46 |
| *** Julien-zte has quit IRC | 04:47 | |
| *** esmute has joined #openstack-infra | 04:49 | |
| *** dave-mccowan has quit IRC | 04:50 | |
| *** Julien-zte has joined #openstack-infra | 04:50 | |
| *** Julien-zte has quit IRC | 04:53 | |
| *** Julien-zte has joined #openstack-infra | 04:54 | |
| *** ihrachys has quit IRC | 04:57 | |
| *** wolverineav has joined #openstack-infra | 04:59 | |
| *** diltram has quit IRC | 05:01 | |
| AJaeger | ianw, pabelanger : Looking at https://review.openstack.org/#/c/455816/ - this is not in dib 2.3.1, correct? In that case we would need a 2.3.2 for https://review.openstack.org/#/c/455843, don't we? | 05:02 |
| ianw | AJaeger: bah, sorry i probably meant to type 2.3.2 | 05:02 |
| ianw | oh, i did | 05:02 |
| ianw | yeah, i made a 2.3.2 release | 05:03 |
| AJaeger | oh, you did - let me check again... | 05:03 |
| *** chlong_ has quit IRC | 05:03 | |
| *** chlong has quit IRC | 05:03 | |
| ianw | i did push it didn't i? | 05:03 |
| * ianw dobuts myself | 05:03 | |
| AJaeger | ianw: yes, everything fine. | 05:03 |
| ianw | heh, ok :) | 05:03 |
| *** wolverineav has quit IRC | 05:03 | |
| openstackgerrit | Merged openstack-infra/project-config master: Do not set permissions for jenkins/* https://review.openstack.org/455584 | 05:03 |
| AJaeger | ianw: Ah, I got as new tags: 2.3.2, 2.3.0, 2.3.1 - and checked the latest only, missed that those are not in order. And then only checked 2.3.1. Mea culpa | 05:04 |
| openstackgerrit | Merged openstack-infra/system-config master: Move exim management to openstack_project::server https://review.openstack.org/455011 | 05:04 |
| *** jtomasek_ has joined #openstack-infra | 05:04 | |
| *** esikachev has joined #openstack-infra | 05:04 | |
| AJaeger | ianw: so, shall we merge pabelanger's revert 455843? Or wait until he's awake? | 05:04 |
| ianw | AJaeger: i think i'd leave it for now ... it's probably worth doing it then manually watching builds | 05:05 |
| AJaeger | ok | 05:06 |
| openstackgerrit | Merged openstack-infra/system-config master: Stop including params class where it is not needed https://review.openstack.org/455021 | 05:06 |
| openstackgerrit | Merged openstack-infra/system-config master: Move automatic_upgrades management to o_p::server https://review.openstack.org/455012 | 05:07 |
| *** Hal1 has quit IRC | 05:07 | |
| openstackgerrit | Merged openstack-infra/system-config master: Move package and file purging to o_p::server https://review.openstack.org/455013 | 05:08 |
| *** wolverineav has joined #openstack-infra | 05:08 | |
| *** esikachev has quit IRC | 05:09 | |
| *** esikachev has joined #openstack-infra | 05:10 | |
| openstackgerrit | Merged openstack-infra/system-config master: Clean up $update_pkg_list_cmd param https://review.openstack.org/455020 | 05:10 |
| *** rcernin|wfh|mtg has quit IRC | 05:11 | |
| *** sflanigan has quit IRC | 05:12 | |
| *** Julien-zte has quit IRC | 05:12 | |
| *** wolverineav has quit IRC | 05:13 | |
| *** jaosorior_away is now known as jaosorior | 05:13 | |
| *** knangia has joined #openstack-infra | 05:14 | |
| *** Julien-zte has joined #openstack-infra | 05:16 | |
| *** esikachev has quit IRC | 05:16 | |
| *** rwsu has quit IRC | 05:17 | |
| *** prateek has joined #openstack-infra | 05:18 | |
| *** Sukhdev has joined #openstack-infra | 05:19 | |
| *** rcernin has joined #openstack-infra | 05:20 | |
| *** prateek is now known as prateek_ | 05:22 | |
| *** prateek_ is now known as prateek | 05:22 | |
| *** prateek is now known as prateek_ | 05:23 | |
| *** diltram has joined #openstack-infra | 05:25 | |
| *** sflanigan has joined #openstack-infra | 05:25 | |
| *** sflanigan has joined #openstack-infra | 05:25 | |
| openstackgerrit | Chhavi Agarwal proposed openstack/diskimage-builder master: Centos7 support for ppc64le https://review.openstack.org/456055 | 05:28 |
| *** Julien-zte has quit IRC | 05:30 | |
| *** Julien-zte has joined #openstack-infra | 05:31 | |
| *** MasterOfBugs has joined #openstack-infra | 05:32 | |
| *** pramodrj07 has joined #openstack-infra | 05:32 | |
| *** jkilpatr_ has quit IRC | 05:35 | |
| *** wolverineav has joined #openstack-infra | 05:38 | |
| *** thorst has joined #openstack-infra | 05:39 | |
| *** wolverineav has quit IRC | 05:42 | |
| *** thorst has quit IRC | 05:43 | |
| *** jtomasek_ has quit IRC | 05:44 | |
| *** shu-mutou-AWAY is now known as shu-mutou | 05:45 | |
| *** Julien-zte has quit IRC | 05:46 | |
| *** markus_z has joined #openstack-infra | 05:46 | |
| *** Julien-zte has joined #openstack-infra | 05:47 | |
| *** jkilpatr_ has joined #openstack-infra | 05:47 | |
| *** jtomasek has quit IRC | 05:49 | |
| *** cshastri has joined #openstack-infra | 05:49 | |
| *** esikachev has joined #openstack-infra | 05:54 | |
| *** Julien-zte has quit IRC | 05:57 | |
| *** Julien-zte has joined #openstack-infra | 05:57 | |
| *** Sukhdev has quit IRC | 05:58 | |
| *** claudiub has joined #openstack-infra | 05:59 | |
| *** Dinesh_Bhor has quit IRC | 06:04 | |
| *** Julien-zte has quit IRC | 06:06 | |
| *** Julien-zte has joined #openstack-infra | 06:07 | |
| *** ociuhandu has quit IRC | 06:08 | |
| *** gyee has quit IRC | 06:08 | |
| *** Julien-zte has quit IRC | 06:11 | |
| *** Julien-zte has joined #openstack-infra | 06:12 | |
| *** Julien-zte has quit IRC | 06:15 | |
| *** Julien-zte has joined #openstack-infra | 06:16 | |
| *** tphummel has joined #openstack-infra | 06:16 | |
| *** wolverineav has joined #openstack-infra | 06:17 | |
| *** Julien-zte has quit IRC | 06:20 | |
| *** Julien-zte has joined #openstack-infra | 06:21 | |
| *** wolverineav has quit IRC | 06:22 | |
| *** john-davidge has joined #openstack-infra | 06:25 | |
| *** aeng has quit IRC | 06:28 | |
| openstackgerrit | Andreas Florath proposed openstack-infra/bindep master: Refactor 'main.py' https://review.openstack.org/456071 | 06:29 |
| *** e0ne has joined #openstack-infra | 06:30 | |
| *** john-davidge has quit IRC | 06:30 | |
| *** pcaruana has joined #openstack-infra | 06:30 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config master: Clean up puppetmaster cruft https://review.openstack.org/452558 | 06:31 |
| *** Julien-zte has quit IRC | 06:31 | |
| *** pramodrj07 has quit IRC | 06:32 | |
| *** MasterOfBugs has quit IRC | 06:32 | |
| *** Julien-zte has joined #openstack-infra | 06:32 | |
| *** Julien-zte has quit IRC | 06:34 | |
| *** Julien-zte has joined #openstack-infra | 06:34 | |
| *** e0ne has quit IRC | 06:35 | |
| *** wolverineav has joined #openstack-infra | 06:37 | |
| *** Dinesh_Bhor has joined #openstack-infra | 06:38 | |
| *** wolverineav has quit IRC | 06:42 | |
| *** HeOS has joined #openstack-infra | 06:42 | |
| *** yuanying_ has joined #openstack-infra | 06:48 | |
| *** yuanying has quit IRC | 06:48 | |
| *** esmute has quit IRC | 06:49 | |
| *** pbourke has quit IRC | 06:49 | |
| *** tesseract has joined #openstack-infra | 06:49 | |
| *** pbourke has joined #openstack-infra | 06:51 | |
| *** cshastri_ has joined #openstack-infra | 06:52 | |
| *** Julien-zte has quit IRC | 06:54 | |
| *** andreykurilin has quit IRC | 06:54 | |
| *** andreykurilin has joined #openstack-infra | 06:54 | |
| *** cshastri has quit IRC | 06:54 | |
| *** esikache1 has joined #openstack-infra | 06:54 | |
| *** jistr has quit IRC | 06:54 | |
| *** jklare has quit IRC | 06:54 | |
| *** adreznec has quit IRC | 06:55 | |
| *** vtapia has quit IRC | 06:55 | |
| *** raissa has quit IRC | 06:55 | |
| *** SamYaple has quit IRC | 06:55 | |
| *** jzerebecki has quit IRC | 06:55 | |
| *** quite has quit IRC | 06:55 | |
| *** csaba has quit IRC | 06:55 | |
| *** dmsimard has quit IRC | 06:55 | |
| *** fabo has quit IRC | 06:55 | |
| *** degorenko has quit IRC | 06:55 | |
| *** jklare has joined #openstack-infra | 06:55 | |
| *** raissa has joined #openstack-infra | 06:55 | |
| *** SamYaple has joined #openstack-infra | 06:55 | |
| *** trown|outtypewww has quit IRC | 06:55 | |
| *** pcrews has quit IRC | 06:55 | |
| *** esikachev has quit IRC | 06:55 | |
| *** markus_z has quit IRC | 06:55 | |
| *** mika has quit IRC | 06:55 | |
| *** dmsimard has joined #openstack-infra | 06:55 | |
| *** mdurrant_ has joined #openstack-infra | 06:55 | |
| *** pfalleno1 has quit IRC | 06:56 | |
| *** corvus has quit IRC | 06:56 | |
| *** jistr has joined #openstack-infra | 06:56 | |
| *** dtardivel has joined #openstack-infra | 06:56 | |
| *** trown has joined #openstack-infra | 06:56 | |
| *** degorenko has joined #openstack-infra | 06:56 | |
| *** jzerebecki has joined #openstack-infra | 06:56 | |
| *** adreznec has joined #openstack-infra | 06:56 | |
| *** pfallenop has joined #openstack-infra | 06:56 | |
| *** fabo has joined #openstack-infra | 06:56 | |
| *** fabo has quit IRC | 06:56 | |
| *** fabo has joined #openstack-infra | 06:56 | |
| *** corvus has joined #openstack-infra | 06:56 | |
| *** csaba has joined #openstack-infra | 06:57 | |
| *** mika has joined #openstack-infra | 06:57 | |
| *** wolverineav has joined #openstack-infra | 06:57 | |
| *** Julien-zte has joined #openstack-infra | 06:57 | |
| *** pcrews has joined #openstack-infra | 06:58 | |
| *** quite has joined #openstack-infra | 06:58 | |
| *** mdurrant has quit IRC | 06:58 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Retire mitaka translation sync https://review.openstack.org/456081 | 06:58 |
| *** vtapia has joined #openstack-infra | 06:59 | |
| *** rcernin has quit IRC | 06:59 | |
| *** florianf has joined #openstack-infra | 07:00 | |
| *** markus_z has joined #openstack-infra | 07:01 | |
| *** wolverineav has quit IRC | 07:02 | |
| *** pgadiya has joined #openstack-infra | 07:02 | |
| *** prateek_ has quit IRC | 07:02 | |
| *** prateek has joined #openstack-infra | 07:02 | |
| *** d0ugal has joined #openstack-infra | 07:03 | |
| *** d0ugal has quit IRC | 07:03 | |
| *** d0ugal has joined #openstack-infra | 07:03 | |
| *** Julien-zte has quit IRC | 07:06 | |
| *** Julien-zte has joined #openstack-infra | 07:07 | |
| gmann | AJaeger: ping | 07:07 |
| *** cshastri_ is now known as cshastri | 07:07 | |
| *** Hal has joined #openstack-infra | 07:07 | |
| AJaeger | gmann: feel free to leave a message and ask, there are others in this channel that can help as well... | 07:08 |
| gmann | AJaeger: do we define separate job for mitaka - https://review.openstack.org/#/c/455637/1 | 07:09 |
| AJaeger | gmann: as I said: Yes | 07:09 |
| gmann | AJaeger: that should be using the standard definition which other stable branch use | 07:09 |
| *** bootsha[m] has left #openstack-infra | 07:10 | |
| gmann | like this - https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/devstack-gate.yaml#L42 | 07:10 |
| AJaeger | gmann: https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/devstack-gate.yaml#L2466 is the job-group and that is uses as periodic job in projects.yaml | 07:11 |
| gmann | AJaeger: yes so that is not specific to mitaka right | 07:12 |
| AJaeger | gmann: please coordinate with tonyb, I like his review on your change - we might want to make this larger. | 07:12 |
| AJaeger | gmann: it is, did you check the usage in projects.yaml? | 07:12 |
| *** jpich has joined #openstack-infra | 07:13 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul feature/zuulv3: Remove outdated TODO https://review.openstack.org/456089 | 07:13 |
| AJaeger | gmann: https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/projects.yaml#L3718 | 07:14 |
| gmann | AJaeger: i cannot find any mitaka specific defintion. that definition used for newton also for periodic pipeline. | 07:14 |
| gmann | AJaeger: oh, you there, | 07:15 |
| *** adriant has quit IRC | 07:15 | |
| AJaeger | for the gate jobs, there are other users still in zuul/layout, so those should not get removed | 07:15 |
| gmann | AJaeger: ok, let me update quickly, actually we cannot wait it till whole cleanup as tempest gate will be blocked if stable branches are gone | 07:15 |
| tonyb | AJaeger, gmann: Based on a 30second review I think we should also remove the mitaka "bitrot-jobs" also. I s'pose that could be a seperate but dependant change | 07:16 |
| AJaeger | gmann: removing stable branches takes time, no stress. | 07:16 |
| *** rcernin has joined #openstack-infra | 07:16 | |
| tonyb | gmann: the branches wont go until next week at the soonest so we have a few days to do it right | 07:16 |
| gmann | tonyb: AJaeger: but then we can merge current patch and remove definition along with complete cleanup | 07:16 |
| gmann | tonyb: AJaeger but does complete cleanup happen before stable branches are gone or after ? | 07:17 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul feature/zuulv3: Set the git sshkey for the connection https://review.openstack.org/456090 | 07:17 |
| tonyb | gmann: I need to think about it in more detail but right now I think I'm okay with that approach, but not okay enough to +1 it :/ | 07:17 |
| AJaeger | gmann, tonyb: Please check how it was done last time... | 07:17 |
| *** gildub has quit IRC | 07:17 | |
| tonyb | AJaeger: Yup will do | 07:17 |
| gmann | tonyb: AJaeger only difference here is tempest run mitaka jobs as voting in check pipeline | 07:18 |
| AJaeger | tonyb, gmann : Might be nice to use one topic for all of these... | 07:18 |
| tonyb | AJaeger: Yeah cleanup-mitaka would work for me | 07:18 |
| gmann | AJaeger: tonyb last time we did with complete cleanup but with blocking the tempest gate | 07:18 |
| *** Julien-zte has quit IRC | 07:18 | |
| AJaeger | gmann: Ah, yeah | 07:18 |
| *** andreas-f has quit IRC | 07:18 | |
| gmann | tonyb: AJaeger but that would nopt work in tempest case as stable job is voting in check pipeline | 07:19 |
| *** aviau has quit IRC | 07:19 | |
| *** Julien-zte has joined #openstack-infra | 07:19 | |
| *** aviau has joined #openstack-infra | 07:19 | |
| AJaeger | let's prepare changes, and merge them before we retire branches - but this can wait until next week. | 07:19 |
| tonyb | AJaeger: +1 | 07:19 |
| gmann | https://review.openstack.org/#/c/410928/ and https://review.openstack.org/#/c/411113/ | 07:20 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/project-config master: Remove python-jobs from pynotedb https://review.openstack.org/455827 | 07:21 |
| *** andreas-f has joined #openstack-infra | 07:21 | |
| *** nadya has joined #openstack-infra | 07:22 | |
| SotK | AJaeger: thanks for catching the things I missed out of that | 07:22 |
| *** shardy has joined #openstack-infra | 07:24 | |
| *** hashar has joined #openstack-infra | 07:25 | |
| gmann | tonyb: AJaeger i am ok as long as we cleanup (including tempest check pipeline) before repo are gone. otherwise it block tempest gate like in liberty EOL case | 07:26 |
| AJaeger | gmann: sure. | 07:26 |
| *** wolverineav has joined #openstack-infra | 07:27 | |
| *** flepied has quit IRC | 07:28 | |
| tonyb | gmann: Yup now that I knwo that was/is a problem I'll factor it in | 07:28 |
| AJaeger | gmann, tonyb : let's use "mitaka-eol" as topic | 07:29 |
| gmann | tonyb: AJaeger cool. thanks | 07:29 |
| *** andreas-f has quit IRC | 07:31 | |
| *** wolverineav has quit IRC | 07:32 | |
| *** nadya has quit IRC | 07:32 | |
| *** nadya has joined #openstack-infra | 07:33 | |
| *** jkilpatr_ has quit IRC | 07:34 | |
| *** jkilpatr_ has joined #openstack-infra | 07:35 | |
| *** rcernin has quit IRC | 07:37 | |
| *** rcernin_ has joined #openstack-infra | 07:37 | |
| *** rcernin_ has quit IRC | 07:38 | |
| *** Julien-zte has quit IRC | 07:38 | |
| *** rcernin_ has joined #openstack-infra | 07:38 | |
| *** Julien-zte has joined #openstack-infra | 07:38 | |
| *** eranrom has quit IRC | 07:39 | |
| *** thorst has joined #openstack-infra | 07:40 | |
| *** tuanluong has joined #openstack-infra | 07:41 | |
| *** eranrom has joined #openstack-infra | 07:41 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove python-jobs from pynotedb https://review.openstack.org/455827 | 07:42 |
| *** jkilpatr_ has quit IRC | 07:43 | |
| openstackgerrit | Merged openstack-infra/project-config master: Demote Senlin py35 job to non-voting https://review.openstack.org/455726 | 07:43 |
| *** jkilpatr_ has joined #openstack-infra | 07:43 | |
| *** rossella_s has joined #openstack-infra | 07:44 | |
| *** nadya has quit IRC | 07:45 | |
| *** thorst has quit IRC | 07:45 | |
| *** Julien-zte has quit IRC | 07:45 | |
| *** Julien-zte has joined #openstack-infra | 07:46 | |
| *** Julien-zte has quit IRC | 07:50 | |
| *** Julien-zte has joined #openstack-infra | 07:50 | |
| *** e0ne has joined #openstack-infra | 07:51 | |
| *** nadya has joined #openstack-infra | 07:56 | |
| *** flepied has joined #openstack-infra | 07:56 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** nadya has quit IRC | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| *** e0ne has quit IRC | 08:00 | |
| *** e0ne has joined #openstack-infra | 08:01 | |
| *** nadya has joined #openstack-infra | 08:01 | |
| *** ralonsoh has joined #openstack-infra | 08:02 | |
| *** lucas-afk is now known as lucasagomes | 08:04 | |
| *** nadya has quit IRC | 08:06 | |
| *** Julien-zte has quit IRC | 08:07 | |
| *** Julien-zte has joined #openstack-infra | 08:08 | |
| *** Julien-zte has quit IRC | 08:12 | |
| *** Julien-zte has joined #openstack-infra | 08:12 | |
| *** dixudx has joined #openstack-infra | 08:14 | |
| *** Julien-zte has quit IRC | 08:15 | |
| *** shardy has quit IRC | 08:15 | |
| *** Julien-zte has joined #openstack-infra | 08:16 | |
| *** alexchadin has joined #openstack-infra | 08:16 | |
| *** shardy has joined #openstack-infra | 08:17 | |
| *** ssbarnea has joined #openstack-infra | 08:17 | |
| *** dixudx has left #openstack-infra | 08:17 | |
| *** Julien-zte has quit IRC | 08:19 | |
| *** Julien-zte has joined #openstack-infra | 08:19 | |
| *** dmellado has joined #openstack-infra | 08:23 | |
| *** aarefiev_afk is now known as aarefiev | 08:24 | |
| *** Hal has quit IRC | 08:25 | |
| *** Julien-zte has quit IRC | 08:26 | |
| openstackgerrit | Merged openstack-infra/project-config master: Load nova configuration for devstack-plugin-ceph job https://review.openstack.org/455818 | 08:26 |
| *** Julien-zte has joined #openstack-infra | 08:26 | |
| *** radez has quit IRC | 08:26 | |
| openstackgerrit | Merged openstack-infra/project-config master: Move DVR+HA multinode tempest job to check queue https://review.openstack.org/455406 | 08:26 |
| openstackgerrit | Merged openstack-infra/project-config master: Bump dsvm-nodepool to 90min timeout https://review.openstack.org/455838 | 08:27 |
| *** s-shiono_ has quit IRC | 08:28 | |
| openstackgerrit | Chhavi Agarwal proposed openstack/diskimage-builder master: Centos7 support for ppc64le https://review.openstack.org/456055 | 08:29 |
| openstackgerrit | Steven Hardy proposed openstack-infra/tripleo-ci master: DNM add python-heat-agent to the delorean-current whitelist https://review.openstack.org/456111 | 08:30 |
| *** Julien-zte has quit IRC | 08:31 | |
| *** knangia has quit IRC | 08:31 | |
| *** Julien-zte has joined #openstack-infra | 08:31 | |
| *** efoley has joined #openstack-infra | 08:34 | |
| *** links has quit IRC | 08:35 | |
| openstackgerrit | Merged openstack-infra/project-config master: Delete running tempest with all plugins https://review.openstack.org/455207 | 08:35 |
| *** efoley_ has joined #openstack-infra | 08:36 | |
| *** GeraldK has joined #openstack-infra | 08:36 | |
| openstackgerrit | Merged openstack-infra/project-config master: Create a repo for container devstack plugin https://review.openstack.org/453230 | 08:36 |
| ttx | FTR we are experiencing issues uploading releases to Pypi | 08:37 |
| GeraldK | Hi. I want to move a page in wiki, however, even after being logged in I cannot see the "Action", "move" as described in the How To: https://wiki.openstack.org/wiki/How_To_Use_The_Wiki | 08:37 |
| ttx | "twine upload" fails with requests.exceptions.ConnectionError: HTTPSConnectionPool(host='upload.pypi.io', port=443): Max retries exceeded with url: /legacy/ (Caused by <class 'socket.gaierror'>: [Errno -2] Name or service not known) | 08:37 |
| GeraldK | I want to rename https://wiki.openstack.org/wiki/ProductTeam/User_Stories to https://wiki.openstack.org/wiki/ProductTeam/Development_Proposals as agreed in the team | 08:38 |
| ttx | GeraldK: let me see | 08:39 |
| ttx | "Move" might be a wiki admin action | 08:39 |
| *** efoley has quit IRC | 08:39 | |
| ttx | yeah, apparently I can do it. Should I do the rename for you now ? | 08:40 |
| ttx | Should I leave a redirect behind (probably yes) | 08:40 |
| GeraldK | ttx: yes. a redirect would be great. I will also manually update the 3 pages that are currently referring to this page | 08:41 |
| *** thorst has joined #openstack-infra | 08:41 | |
| ttx | GeraldK: done | 08:41 |
| ttx | please check it works | 08:41 |
| GeraldK | ttx: should we also update the How To with a note that this is an Admin action? | 08:41 |
| ttx | yeah probably | 08:42 |
| GeraldK | ttx: great. thanks for the quick help. | 08:42 |
| *** neiljerram has joined #openstack-infra | 08:42 | |
| ttx | might have been changed to limit damage that can be done by spammers | 08:42 |
| GeraldK | ttx: should I change the How To page or you want to do it? | 08:42 |
| ttx | GeraldK: please do it if you have the page open :) | 08:42 |
| *** Julien-zte has quit IRC | 08:44 | |
| *** kwazar has joined #openstack-infra | 08:44 | |
| *** Julien-zte has joined #openstack-infra | 08:45 | |
| GeraldK | ttx: okay | 08:45 |
| *** electrofelix has joined #openstack-infra | 08:45 | |
| *** Julien-zte has quit IRC | 08:47 | |
| *** Julien-zte has joined #openstack-infra | 08:48 | |
| *** priteau has joined #openstack-infra | 08:49 | |
| *** yonglihe has quit IRC | 08:49 | |
| *** Julien-zte has quit IRC | 08:51 | |
| *** Julien-zte has joined #openstack-infra | 08:51 | |
| *** yamahata has quit IRC | 08:51 | |
| *** links has joined #openstack-infra | 08:52 | |
| *** jklare_ has joined #openstack-infra | 08:55 | |
| *** mhickey has joined #openstack-infra | 08:55 | |
| *** ociuhandu has joined #openstack-infra | 08:55 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: WIP: test containers with single-nic-vlan https://review.openstack.org/447973 | 08:57 |
| *** jklare has quit IRC | 08:57 | |
| *** jklare_ is now known as jklare | 08:57 | |
| *** shardy is now known as shardy_mtg | 08:57 | |
| *** thorst has quit IRC | 09:00 | |
| *** Julien-zte has quit IRC | 09:05 | |
| *** Julien-zte has joined #openstack-infra | 09:05 | |
| *** jkilpatr_ has quit IRC | 09:06 | |
| *** Julien-zte has quit IRC | 09:07 | |
| *** jkilpatr_ has joined #openstack-infra | 09:07 | |
| *** Julien-zte has joined #openstack-infra | 09:08 | |
| *** GeraldK has quit IRC | 09:10 | |
| *** csmart1 has quit IRC | 09:11 | |
| *** masayukig[m] has quit IRC | 09:11 | |
| *** jkilpatr_ has quit IRC | 09:11 | |
| *** jkilpatr_ has joined #openstack-infra | 09:12 | |
| *** csmart has joined #openstack-infra | 09:13 | |
| *** dtantsur|afk is now known as dtantsur | 09:15 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/storyboard-webclient master: Change minlength to 3 for projects and project_groups name https://review.openstack.org/456125 | 09:16 |
| *** neiljerram has quit IRC | 09:17 | |
| *** neiljerram has joined #openstack-infra | 09:18 | |
| *** Julien-zte has quit IRC | 09:19 | |
| openstackgerrit | Steven Dake proposed openstack-infra/project-config master: Promote compute-kit to check jobs https://review.openstack.org/456130 | 09:20 |
| *** Julien-zte has joined #openstack-infra | 09:20 | |
| *** links has quit IRC | 09:20 | |
| *** masayukig[m] has joined #openstack-infra | 09:20 | |
| *** numans has quit IRC | 09:21 | |
| *** Hal has joined #openstack-infra | 09:21 | |
| *** Julien-zte has quit IRC | 09:24 | |
| *** Julien-zte has joined #openstack-infra | 09:24 | |
| *** Julien-zte has quit IRC | 09:26 | |
| *** Julien-zte has joined #openstack-infra | 09:26 | |
| *** Julien-zte has quit IRC | 09:28 | |
| *** Julien-zte has joined #openstack-infra | 09:28 | |
| *** xarses has quit IRC | 09:30 | |
| *** liujiong_lj has joined #openstack-infra | 09:30 | |
| *** xarses_ has joined #openstack-infra | 09:30 | |
| *** liujiong has quit IRC | 09:31 | |
| *** numans has joined #openstack-infra | 09:32 | |
| *** tosky has joined #openstack-infra | 09:32 | |
| *** links has joined #openstack-infra | 09:33 | |
| *** Julien-zte has quit IRC | 09:33 | |
| *** liujiong_lj is now known as liujiong | 09:34 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Remove the Stable team meeting https://review.openstack.org/455848 | 09:36 |
| *** wolverineav has joined #openstack-infra | 09:38 | |
| *** john-davidge has joined #openstack-infra | 09:41 | |
| *** wolverineav has quit IRC | 09:42 | |
| openstackgerrit | Roman Podoliaka proposed openstack-infra/project-config master: osc-placement: experimental job for running of functional tests https://review.openstack.org/452210 | 09:47 |
| *** neiljerram has quit IRC | 09:48 | |
| *** neiljerram has joined #openstack-infra | 09:51 | |
| *** sambetts|afk is now known as sambetts | 09:52 | |
| *** john-dav_ has joined #openstack-infra | 09:55 | |
| *** john-davidge has quit IRC | 09:55 | |
| *** thorst has joined #openstack-infra | 09:57 | |
| *** wolverineav has joined #openstack-infra | 09:58 | |
| *** thorst has quit IRC | 10:01 | |
| *** wolverineav has quit IRC | 10:03 | |
| *** askb has quit IRC | 10:07 | |
| openstackgerrit | Martin André proposed openstack-infra/project-config master: Switch gate-tripleo-buildimage to use trunk.rdoproject.org https://review.openstack.org/456148 | 10:11 |
| asettle | mordred: we appear to be having another element failing doc patches this morning \o/ | 10:14 |
| *** Hal has quit IRC | 10:14 | |
| asettle | have you seen this before? "UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 51: character maps to <undefined>" | 10:14 |
| asettle | It's failing three of my patches in the 'niceness' tests. | 10:15 |
| asettle | Example: https://review.openstack.org/#/c/456120/ | 10:15 |
| *** tphummel has quit IRC | 10:15 | |
| *** rhallisey has quit IRC | 10:16 | |
| openstackgerrit | Merged openstack-infra/project-config master: Fix syntax error in blazar.yaml https://review.openstack.org/455683 | 10:16 |
| *** wolverineav has joined #openstack-infra | 10:18 | |
| *** sdague has joined #openstack-infra | 10:22 | |
| *** wolverineav has quit IRC | 10:22 | |
| *** adarazs is now known as adarazs_afk | 10:26 | |
| *** wolverineav has joined #openstack-infra | 10:28 | |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: doc: Document Sphinx integration https://review.openstack.org/456154 | 10:28 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: docs: Use definition lists https://review.openstack.org/456155 | 10:28 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Stop building man pages by default https://review.openstack.org/456156 | 10:28 |
| *** wolverineav has quit IRC | 10:32 | |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Remove 'build_sphinx_latex' https://review.openstack.org/456157 | 10:32 |
| openstackgerrit | Sean Dague proposed openstack-infra/os-loganalyze master: Fix systemd line parsing https://review.openstack.org/455831 | 10:32 |
| *** rossella_s has quit IRC | 10:33 | |
| *** nicolasbock has joined #openstack-infra | 10:33 | |
| *** aspiers has quit IRC | 10:33 | |
| *** neiljerram has quit IRC | 10:34 | |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Remove 'build_sphinx_latex' https://review.openstack.org/456157 | 10:34 |
| *** rossella_s has joined #openstack-infra | 10:34 | |
| *** efoley__ has joined #openstack-infra | 10:35 | |
| *** neiljerram has joined #openstack-infra | 10:35 | |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Stop building man pages by default https://review.openstack.org/456156 | 10:36 |
| *** aspiers has joined #openstack-infra | 10:36 | |
| *** esikache1 has quit IRC | 10:38 | |
| *** esikachev has joined #openstack-infra | 10:38 | |
| *** efoley_ has quit IRC | 10:38 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul feature/zuulv3: Only grab the gerrit change if necessary https://review.openstack.org/456162 | 10:41 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul feature/zuulv3: Only grab the gerrit change if necessary https://review.openstack.org/456162 | 10:42 |
| *** tuanluong has quit IRC | 10:43 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/project-config master: Remove check-requirements template from pynotedb https://review.openstack.org/456163 | 10:45 |
| *** hashar has quit IRC | 10:46 | |
| *** aspiers has quit IRC | 10:46 | |
| *** jkilpatr_ has quit IRC | 10:49 | |
| *** yanyanhu has quit IRC | 10:50 | |
| *** efoley_ has joined #openstack-infra | 10:54 | |
| *** liujiong has quit IRC | 10:56 | |
| *** aspiers has joined #openstack-infra | 10:58 | |
| *** efoley__ has quit IRC | 10:58 | |
| *** alexchadin has quit IRC | 11:00 | |
| *** dtantsur is now known as dtantsur|brb | 11:01 | |
| openstackgerrit | Ildiko Vancsa proposed openstack-infra/system-config master: Add #openstack-upstream-institute to meetbot channels https://review.openstack.org/442004 | 11:02 |
| *** alexchadin has joined #openstack-infra | 11:04 | |
| *** john-dav_ is now known as john-davidge | 11:06 | |
| *** wolverineav has joined #openstack-infra | 11:07 | |
| *** wolverineav has quit IRC | 11:12 | |
| *** rhallisey has joined #openstack-infra | 11:13 | |
| *** shardy_mtg is now known as shardy | 11:13 | |
| *** rfolco has joined #openstack-infra | 11:15 | |
| *** adarazs_afk is now known as adarazs | 11:17 | |
| *** shardy is now known as shardy_lunch | 11:19 | |
| *** jkilpatr_ has joined #openstack-infra | 11:20 | |
| *** zhurong has quit IRC | 11:26 | |
| *** wolverineav has joined #openstack-infra | 11:27 | |
| *** askb has joined #openstack-infra | 11:28 | |
| *** wolverineav has quit IRC | 11:31 | |
| *** lucasagomes is now known as lucas-hungry | 11:32 | |
| AJaeger | pabelanger, cmurphy, clarkb, we have a problem with current images, see asettle's comment from an hour ago about the UnicodeError http://logs.openstack.org/20/456120/1/check/gate-openstack-manuals-tox-checkniceness-ubuntu-xenial/3c6cd7d/console.html#_2017-04-12_10_00_56_238447 . Is that related to the image changes we're doing? | 11:39 |
| asettle | thanks AJaeger :) | 11:39 |
| cmurphy | AJaeger: I'm not sure but I wouldn't think so because it is using python2 | 11:41 |
| *** thorst has joined #openstack-infra | 11:41 | |
| cmurphy | cool, I can reproduce it on my machine \o/ | 11:44 |
| AJaeger | cmurphy: so, it's unrelated? New package release that's broken? | 11:46 |
| *** baoli has joined #openstack-infra | 11:49 | |
| *** prateek_ has joined #openstack-infra | 11:49 | |
| cmurphy | AJaeger: that is my guess | 11:49 |
| cmurphy | AJaeger: looks like chardet released a new version yesterday | 11:49 |
| *** psachin has quit IRC | 11:51 | |
| *** prateek has quit IRC | 11:52 | |
| *** chlong_ has joined #openstack-infra | 11:53 | |
| *** chlong has joined #openstack-infra | 11:53 | |
| cmurphy | AJaeger: asettle rolling chardet back to 2.3.0 from 3.0.1 makes the problem go away for me, so maybe it should get pinned in requirements and a bug filed with them | 11:54 |
| *** chlong_ has quit IRC | 11:54 | |
| *** shu-mutou is now known as shu-mutou-AWAY | 11:55 | |
| *** radez has joined #openstack-infra | 11:56 | |
| *** wolverineav has joined #openstack-infra | 11:57 | |
| *** shardy_lunch is now known as shardy | 12:00 | |
| *** dtantsur|brb is now known as dtantsur | 12:00 | |
| *** wolverineav has quit IRC | 12:02 | |
| rakhmerov | hi, are you guys aware of any problems brought by oslo.messaging 5.21.0? | 12:02 |
| *** Zhongjun__ has joined #openstack-infra | 12:02 | |
| rakhmerov | that's what we started getting today: http://paste.openstack.org/show/606269/ | 12:02 |
| rakhmerov | in Mistral | 12:02 |
| *** mat128 has joined #openstack-infra | 12:02 | |
| rakhmerov | trying to figure out if it's a known problem and if someone is working on it.. | 12:02 |
| openstackgerrit | Jiri Stransky proposed openstack-infra/tripleo-ci master: Support container upgrades in multinode OOOQ CI https://review.openstack.org/450784 | 12:04 |
| *** winggundamth has joined #openstack-infra | 12:07 | |
| *** psachin has joined #openstack-infra | 12:08 | |
| *** hashar has joined #openstack-infra | 12:09 | |
| *** sbezverk has quit IRC | 12:09 | |
| openstackgerrit | Emma Foley proposed openstack-infra/project-config master: Re-order plugins in collectd-ceiloemter-plugin job https://review.openstack.org/456193 | 12:09 |
| *** krtaylor has joined #openstack-infra | 12:11 | |
| *** gouthamr has joined #openstack-infra | 12:14 | |
| *** lucas-hungry is now known as lucasagomes | 12:14 | |
| *** gongysh has quit IRC | 12:15 | |
| *** esikachev has quit IRC | 12:15 | |
| *** gongysh has joined #openstack-infra | 12:17 | |
| *** armax has quit IRC | 12:17 | |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Remove 'build_sphinx_latex' https://review.openstack.org/456157 | 12:21 |
| *** kgiusti has joined #openstack-infra | 12:21 | |
| *** gordc has joined #openstack-infra | 12:25 | |
| *** Goneri has joined #openstack-infra | 12:30 | |
| *** mriedem has joined #openstack-infra | 12:31 | |
| *** pgadiya has quit IRC | 12:31 | |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: Stop building man pages by default https://review.openstack.org/456156 | 12:31 |
| *** jcoufal has joined #openstack-infra | 12:34 | |
| *** askb has quit IRC | 12:36 | |
| *** hashar has quit IRC | 12:37 | |
| *** markvoelker has joined #openstack-infra | 12:37 | |
| *** baoli has quit IRC | 12:39 | |
| *** eharney has joined #openstack-infra | 12:43 | |
| mordred | sfinucan: those all look great | 12:49 |
| mordred | sfinucan: thank you for taking on improvements there | 12:49 |
| sfinucan | mordred: No problemo | 12:50 |
| trown | pabelanger: yolanda: when you get a chance could you review https://review.openstack.org/456148 ... our gate on tripleoclient is broken without that | 12:51 |
| *** mika has quit IRC | 12:52 | |
| *** mika has joined #openstack-infra | 12:52 | |
| *** mika has quit IRC | 12:53 | |
| *** mika has joined #openstack-infra | 12:53 | |
| *** mika has joined #openstack-infra | 12:53 | |
| *** jcoufal_ has joined #openstack-infra | 12:53 | |
| *** mika has quit IRC | 12:54 | |
| *** mika has joined #openstack-infra | 12:54 | |
| dhellmann | good morning infra team | 12:55 |
| dhellmann | earlier this morning we saw several jobs uploading to pypi fail with what looks like DNS errors, http://logs.openstack.org/a5/a52ae17a85b4743a41bf104e1d621d29cc345185/release/python-tricircleclient-pypi-both-upload/b0633cb/console.html | 12:55 |
| dhellmann | are you aware of similar issues in other types of jobs, or should I be contacting the pypi folks to see if they had some sort of provider issue around then? | 12:56 |
| *** jcoufal has quit IRC | 12:56 | |
| *** hashar has joined #openstack-infra | 12:58 | |
| ricardoas | morning everybody... maybe a silly question... but how updated a third party ci project-config should be wrt openstack-infra/project-config? | 12:59 |
| *** armax has joined #openstack-infra | 13:00 | |
| asettle | cmurphy and AJaeger - sorry, was out | 13:00 |
| ricardoas | specially jenkins/scripts, nodepool/scripts and nodepool/elements dirs | 13:00 |
| *** jklare has quit IRC | 13:01 | |
| *** baoli has joined #openstack-infra | 13:02 | |
| *** mat128 has quit IRC | 13:02 | |
| *** hrubi has quit IRC | 13:04 | |
| *** jklare has joined #openstack-infra | 13:04 | |
| *** hrubi has joined #openstack-infra | 13:04 | |
| *** mat128 has joined #openstack-infra | 13:05 | |
| *** hrubi has quit IRC | 13:06 | |
| *** psachin has quit IRC | 13:06 | |
| *** hrubi has joined #openstack-infra | 13:07 | |
| *** Julien-zte has joined #openstack-infra | 13:11 | |
| frickler | dhellmann: ttx mentioned similar issues earlier, but I think noone has taken any action yet | 13:11 |
| openstackgerrit | Major Hayden proposed openstack-infra/bindep master: Add Amazon Linux support to bindep https://review.openstack.org/454240 | 13:14 |
| frickler | dhellmann: looking at the logs, I'd assume contacting pypi would be the proper step | 13:15 |
| *** dprince has joined #openstack-infra | 13:15 | |
| *** prateek_ has quit IRC | 13:16 | |
| *** links has quit IRC | 13:17 | |
| *** esberglu has joined #openstack-infra | 13:18 | |
| *** xyang1 has joined #openstack-infra | 13:18 | |
| *** Julien-zte has quit IRC | 13:21 | |
| *** erlon has joined #openstack-infra | 13:23 | |
| dhellmann | frickler : ttx and I are working on the same issue | 13:23 |
| *** tmorin has joined #openstack-infra | 13:23 | |
| *** Julien-zte has joined #openstack-infra | 13:24 | |
| dhellmann | I'm able to upload to pypi now by hand using twine, so I think the next step is to resubmit one of the jobs and see if it was a dns issue with our build node | 13:24 |
| tmorin | hi yolanda... we've got neutron liaison ack for https://review.openstack.org/#/c/451897, can you push the button ? | 13:24 |
| yolanda | hi, trown , tmorin , looking | 13:25 |
| trown | yolanda: morning :) | 13:25 |
| tmorin | thanks yolanda ! | 13:26 |
| *** mattmceuen has joined #openstack-infra | 13:26 | |
| frickler | dhellmann: ah, o.k., I assumed that you had seen independent failures, but then it may have been a local node issue indeed | 13:26 |
| *** psachin has joined #openstack-infra | 13:27 | |
| dhellmann | the full list of failures is in https://etherpad.openstack.org/p/fSNrLxz3Hx | 13:27 |
| *** neiljerram has quit IRC | 13:28 | |
| dhellmann | who's on duty this week who can requeue jobs? | 13:28 |
| *** dimtruck is now known as zz_dimtruck | 13:29 | |
| *** zz_dimtruck is now known as dimtruck | 13:29 | |
| *** neiljerram has joined #openstack-infra | 13:30 | |
| *** vikrant has quit IRC | 13:32 | |
| *** efoley__ has joined #openstack-infra | 13:33 | |
| openstackgerrit | Merged openstack-infra/project-config master: networking-bagpipe: add an experimental fullstack job https://review.openstack.org/451897 | 13:33 |
| *** efoley_ has quit IRC | 13:37 | |
| *** markvoelker has quit IRC | 13:37 | |
| *** hrubi has quit IRC | 13:37 | |
| *** hrubi has joined #openstack-infra | 13:37 | |
| *** markvoelker has joined #openstack-infra | 13:40 | |
| *** Julien-zte has quit IRC | 13:40 | |
| *** Julien-zte has joined #openstack-infra | 13:41 | |
| *** dimtruck is now known as zz_dimtruck | 13:43 | |
| *** mriedem has quit IRC | 13:44 | |
| *** mriedem has joined #openstack-infra | 13:44 | |
| *** ihrachys has joined #openstack-infra | 13:45 | |
| *** Julien-zte has quit IRC | 13:45 | |
| *** Julien-zte has joined #openstack-infra | 13:46 | |
| *** zhurong has joined #openstack-infra | 13:46 | |
| *** hamzy has quit IRC | 13:47 | |
| *** mattmceuen has quit IRC | 13:48 | |
| *** Julien-zte has quit IRC | 13:48 | |
| *** Julien-zte has joined #openstack-infra | 13:49 | |
| *** kjackal_ has joined #openstack-infra | 13:50 | |
| *** adarazs is now known as adarazs_afk | 13:50 | |
| *** schang has joined #openstack-infra | 13:53 | |
| *** amotoki has joined #openstack-infra | 13:54 | |
| *** Julien-zte has quit IRC | 13:54 | |
| *** kjackal_ has quit IRC | 13:54 | |
| *** Julien-zte has joined #openstack-infra | 13:54 | |
| *** erikmwilson has quit IRC | 13:56 | |
| *** erikmwil_ has joined #openstack-infra | 13:56 | |
| mwhahaha | mgagne: can you either close down #openstack-puppet or transfer ownership over to infra so we can get it redirected to #puppet-openstack | 13:57 |
| *** kjackal_ has joined #openstack-infra | 13:57 | |
| *** tongli has joined #openstack-infra | 13:57 | |
| *** rbrndt has joined #openstack-infra | 13:58 | |
| *** zhurong_ has joined #openstack-infra | 13:59 | |
| *** zhurong has quit IRC | 14:01 | |
| *** kjackal_ has quit IRC | 14:02 | |
| *** alexchadin has quit IRC | 14:02 | |
| *** zz_dimtruck is now known as dimtruck | 14:02 | |
| *** mhickey has quit IRC | 14:03 | |
| *** kjackal_ has joined #openstack-infra | 14:03 | |
| AJaeger | cmurphy : Blacklisting chardet helps, thanks! I opened a bug now but have too little details (https://github.com/chardet/chardet/issues/112) . asettle, FYI. | 14:04 |
| *** smarcet has joined #openstack-infra | 14:04 | |
| asettle | Thanks AJaeger ;) | 14:04 |
| hwoarang | Hi. does anyone know at which point 'tox' is installed on the openstack ci images? as far as I can see, an image generated by dib does not have tox in it, and the run-tox.sh script from project-config does not install tox. so i presume it's installed by a puppet module at some point? | 14:09 |
| *** felipemonteiro has joined #openstack-infra | 14:10 | |
| *** pblaho has quit IRC | 14:11 | |
| *** david-lyle has quit IRC | 14:15 | |
| *** jcoufal has joined #openstack-infra | 14:16 | |
| *** jcoufal_ has quit IRC | 14:16 | |
| *** mhickey has joined #openstack-infra | 14:18 | |
| *** ldnunes has joined #openstack-infra | 14:19 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Don't fail on security_groups=None https://review.openstack.org/456231 | 14:19 |
| *** hamzy has joined #openstack-infra | 14:21 | |
| mordred | adam_g: ^^ that should fix a bug you submitted - sorry for the delay | 14:22 |
| *** hrubi has quit IRC | 14:25 | |
| *** hrubi has joined #openstack-infra | 14:25 | |
| *** Julien-zte has quit IRC | 14:27 | |
| openstackgerrit | chenying proposed openstack-infra/project-config master: Karbor: update the $PROJECTS variable of the job definition https://review.openstack.org/456233 | 14:27 |
| *** adarazs_afk is now known as adarazs | 14:27 | |
| *** armax has quit IRC | 14:28 | |
| *** hrubi has quit IRC | 14:29 | |
| *** hrubi has joined #openstack-infra | 14:29 | |
| *** Julien-zte has joined #openstack-infra | 14:30 | |
| *** EricGonczer_ has joined #openstack-infra | 14:30 | |
| *** rossella_s has quit IRC | 14:33 | |
| *** links has joined #openstack-infra | 14:33 | |
| *** rossella_s has joined #openstack-infra | 14:34 | |
| *** witek has joined #openstack-infra | 14:36 | |
| witek | hello | 14:36 |
| *** links has quit IRC | 14:36 | |
| witek | does anyone has an idea why maven-build jobs fail at the moment? | 14:37 |
| witek | e.g. gate-monasca-api-maven-build-ubuntu-xenial | 14:37 |
| witek | 2017-04-12 13:08:10.351894 | /tmp/04-8a792176869044e48a7ddb783cf72e34.sh: line 4: mvn: command not found | 14:37 |
| pabelanger | morning | 14:38 |
| *** rkukura_ has joined #openstack-infra | 14:38 | |
| pabelanger | hwoarang: tox is installed by our puppet manifest, however it will be moving to a DIB element shortly | 14:38 |
| cmurphy | pabelanger: I bet that one is our fault ^ | 14:38 |
| pabelanger | cmurphy: looks like it | 14:39 |
| *** efoley_ has joined #openstack-infra | 14:39 | |
| pabelanger | we can add it to bindep | 14:39 |
| *** eharney has quit IRC | 14:40 | |
| AJaeger | witek: can you add a bindep.txt file, please? | 14:40 |
| pabelanger | witek: you should be able to add maven to your project bindep.txt file | 14:40 |
| pabelanger | AJaeger: thanks | 14:40 |
| openstackgerrit | Tom Barron proposed openstack-infra/project-config master: Add experimental manila cephfs CentOS jobs https://review.openstack.org/453910 | 14:40 |
| AJaeger | witek: https://docs.openstack.org/infra/manual/drivers.html#package-requirements | 14:40 |
| *** rkukura has quit IRC | 14:40 | |
| *** rkukura_ is now known as rkukura | 14:40 | |
| hwoarang | pabelanger: thank you. can you point me to that manifest file? I am curious why 'tox' execution on openstack ci results to /usr/local/bin/dib-python being used as interp. | 14:41 |
| hwoarang | sounds like that manifest modifies tox to use the python wrapper from dib but i like to see how it's done | 14:42 |
| rajinir | zuul process dies suddenly. Anyone know why this would happen? I have zuul-2.5.1installed. | 14:42 |
| pabelanger | hwoarang: oh, that is a bug. and should be fixed now | 14:42 |
| *** efoley__ has quit IRC | 14:42 | |
| *** liusheng has quit IRC | 14:42 | |
| pabelanger | hwoarang: we are doing new image builds today, which will not have /usr/local/bin/dib-python | 14:43 |
| hwoarang | oh | 14:43 |
| *** felipemonteiro__ has joined #openstack-infra | 14:43 | |
| pabelanger | hwoarang: https://review.openstack.org/#/c/455482/ | 14:43 |
| *** cshastri has quit IRC | 14:43 | |
| pabelanger | http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/single_use_slave.pp#n57 | 14:44 |
| pabelanger | where we install tox | 14:44 |
| hwoarang | thank you pabelanger | 14:44 |
| pabelanger | hwoarang: np | 14:44 |
| openstackgerrit | Stephen Finucane proposed openstack-dev/pbr master: doc: Document Sphinx integration https://review.openstack.org/456154 | 14:44 |
| *** jcoufal has quit IRC | 14:44 | |
| *** eharney has joined #openstack-infra | 14:45 | |
| *** yamahata has joined #openstack-infra | 14:45 | |
| *** Julien-zte has quit IRC | 14:46 | |
| openstackgerrit | Monty Taylor proposed openstack/os-client-config master: Add ability to pass in user_agent https://review.openstack.org/452550 | 14:46 |
| *** felipemonteiro_ has quit IRC | 14:47 | |
| *** Julien-zte has joined #openstack-infra | 14:47 | |
| *** jcoufal has joined #openstack-infra | 14:48 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Run ovb jobs in transition mode https://review.openstack.org/454134 | 14:48 |
| *** hongbin has joined #openstack-infra | 14:50 | |
| witek | AJaeger, pabelanger: should I just add 'mvn' in bindep.txt and refer to it in tox.ini? | 14:51 |
| *** Julien-zte has quit IRC | 14:51 | |
| pabelanger | witek: either maven or maven2 IIRC | 14:53 |
| *** Julien-zte has joined #openstack-infra | 14:53 | |
| cmurphy | witek: pabelanger looks like 'maven' http://git.openstack.org/cgit/openstack-infra/puppet-jenkins/tree/manifests/params.pp#n55 | 14:53 |
| cmurphy | for xenial | 14:53 |
| *** EricGonc_ has joined #openstack-infra | 14:54 | |
| witek | pabelanger, cmurphy: thanks | 14:54 |
| *** EricGonczer_ has quit IRC | 14:55 | |
| openstackgerrit | Tom Barron proposed openstack-infra/project-config master: Add experimental manila cephfs CentOS jobs https://review.openstack.org/453910 | 14:55 |
| *** Julien-zte has quit IRC | 14:55 | |
| AJaeger | witek: and copy over what we use today, bindep.txt replaces our fallback file from project-config | 14:56 |
| asettle | So, cmurphy this popped up in the swift channel today: https://review.openstack.org/#/c/454386/2 And this one too: https://review.openstack.org/#/c/453071/ | 14:56 |
| asettle | Should we be fixing the rest of the guide? :/ | 14:56 |
| asettle | I went looking for bug reports today and found naught | 14:57 |
| *** zhurong_ has quit IRC | 14:57 | |
| *** Julien-zte has joined #openstack-infra | 14:57 | |
| *** annegentle has joined #openstack-infra | 14:57 | |
| cmurphy | asettle: I went through pretty much the entire guide and those were the only two instances I noticed it | 14:58 |
| asettle | cmurphy: oh phew. Thank you!!! | 14:58 |
| *** mkoderer has joined #openstack-infra | 14:58 | |
| asettle | I was about to start the same shenanigans | 14:59 |
| *** armax has joined #openstack-infra | 14:59 | |
| *** jkilpatr_ has quit IRC | 14:59 | |
| *** florianf has quit IRC | 14:59 | |
| cmurphy | asettle: :) | 15:00 |
| asettle | :D | 15:00 |
| asettle | Hahahha you really just made my day. | 15:00 |
| asettle | I was seriously dreading that | 15:00 |
| *** srobert has joined #openstack-infra | 15:00 | |
| *** jkilpatr has joined #openstack-infra | 15:00 | |
| *** john-davidge has quit IRC | 15:00 | |
| *** Julien-zte has quit IRC | 15:00 | |
| openstackgerrit | Andy McCrae proposed openstack-infra/project-config master: Add os_karbor to OpenStack-Ansible https://review.openstack.org/456244 | 15:00 |
| *** marst has joined #openstack-infra | 15:01 | |
| pabelanger | dhellmann: ttx: is anybody help requeue pypi jobs failures yet? | 15:01 |
| *** john-davidge has joined #openstack-infra | 15:01 | |
| *** Julien-zte has joined #openstack-infra | 15:01 | |
| cmurphy | asettle: I was looking more for issues in the SUSE instructions than for that specific issue so I by no means guarantee I found them all | 15:01 |
| *** srobert has quit IRC | 15:01 | |
| asettle | cmurphy: oh no I trust you implicitly you seem like a nice person :p | 15:02 |
| asettle | (I'll review the guide, and make sure :) ) | 15:02 |
| *** srobert has joined #openstack-infra | 15:02 | |
| cmurphy | asettle: yay :) | 15:02 |
| *** srobert has quit IRC | 15:02 | |
| *** jaosorior is now known as jaosorior_away | 15:02 | |
| *** srobert has joined #openstack-infra | 15:03 | |
| pabelanger | AJaeger: are you aware of any issues with the gate-project-config-layout job? | 15:03 |
| *** rcernin_ has quit IRC | 15:04 | |
| AJaeger | pabelanger: saw a failure a few minutes ago and had no time to check further. Something looked very odd ;( | 15:04 |
| AJaeger | pabelanger: in a meeting now, won't have time to dig deeper ;( | 15:04 |
| pabelanger | AJaeger: cool thanks | 15:05 |
| *** Julien-zte has quit IRC | 15:06 | |
| *** florianf has joined #openstack-infra | 15:06 | |
| *** Julien-zte has joined #openstack-infra | 15:07 | |
| pabelanger | AJaeger: looks like new voluptuous release today | 15:07 |
| *** Julien-zte has quit IRC | 15:10 | |
| *** Apoorva has joined #openstack-infra | 15:10 | |
| *** Julien-zte has joined #openstack-infra | 15:11 | |
| *** Apoorva has quit IRC | 15:13 | |
| AJaeger | pabelanger: so, blacklist it in both project-config and zuul - and file a bug? | 15:15 |
| *** ggnel_t has quit IRC | 15:16 | |
| *** xarses has joined #openstack-infra | 15:16 | |
| *** mhickey has quit IRC | 15:16 | |
| *** andreww has joined #openstack-infra | 15:17 | |
| openstackgerrit | chenying proposed openstack-infra/project-config master: Karbor: update the $PROJECTS variable of the job definition https://review.openstack.org/456233 | 15:18 |
| AJaeger | witek: bindep.txt is self-testing, so if it passes all tests, merge it ;) I commented, I fear your file is too short | 15:20 |
| openstackgerrit | Rabi Mishra proposed openstack-infra/project-config master: WIP Make heat py35 dsvm job voting https://review.openstack.org/456253 | 15:20 |
| *** xarses has quit IRC | 15:20 | |
| *** wolverineav has joined #openstack-infra | 15:21 | |
| witek | AJaeger: thanks | 15:21 |
| *** xarses has joined #openstack-infra | 15:21 | |
| *** andreww has quit IRC | 15:21 | |
| *** xarses has quit IRC | 15:21 | |
| *** xarses has joined #openstack-infra | 15:22 | |
| clarkb | cmurphy: thats a fun doc bug | 15:24 |
| *** beslemon has joined #openstack-infra | 15:24 | |
| clarkb | AJaeger: I +2'd the infra manual rebase change, thanks for following up on that. Do you have a link to one of the zuul layout failures? | 15:24 |
| AJaeger | clarkb: thanks for review. For layout, see http://logs.openstack.org/44/456244/1/check/gate-project-config-layout/989b03a/console.html#_2017-04-12_15_07_29_393722 | 15:25 |
| cmurphy | clarkb: ya i think there's a funny character on this line that it doesn't understand http://git.openstack.org/cgit/openstack/openstack-manuals/tree/doc/contributor-guide/source/docs-builds.rst#n140 | 15:25 |
| *** Julien-zte has quit IRC | 15:28 | |
| *** Julien-zte has joined #openstack-infra | 15:28 | |
| *** baoli has quit IRC | 15:28 | |
| AJaeger | cmurphy: do you want to propose a change that updates that line and removes the blacklist of chardet? | 15:28 |
| clarkb | AJaeger: pabelanger almost looks like voluptuous is just being more strict now? let me try pushing a change | 15:29 |
| *** baoli has joined #openstack-infra | 15:29 | |
| *** mhickey has joined #openstack-infra | 15:29 | |
| cmurphy | AJaeger: yep working on it | 15:30 |
| AJaeger | thanks, cmurphy ! | 15:30 |
| *** d0ugal has quit IRC | 15:30 | |
| * clarkb runs tests locally first | 15:30 | |
| *** d0ugal has joined #openstack-infra | 15:30 | |
| *** d0ugal has quit IRC | 15:30 | |
| *** d0ugal has joined #openstack-infra | 15:30 | |
| *** d0ugal has quit IRC | 15:31 | |
| *** d0ugal has joined #openstack-infra | 15:31 | |
| *** d0ugal has quit IRC | 15:31 | |
| *** d0ugal has joined #openstack-infra | 15:31 | |
| *** mdrabe has joined #openstack-infra | 15:31 | |
| *** Julien-zte has quit IRC | 15:33 | |
| *** Julien-zte has joined #openstack-infra | 15:35 | |
| *** knangia has joined #openstack-infra | 15:36 | |
| *** winggundamth_ has joined #openstack-infra | 15:36 | |
| *** tphummel has joined #openstack-infra | 15:36 | |
| *** Julien-zte has quit IRC | 15:37 | |
| *** Julien-zte has joined #openstack-infra | 15:38 | |
| pabelanger | clarkb: thanks, got side tracked with meeting | 15:39 |
| *** florianf has quit IRC | 15:39 | |
| *** winggundamth has quit IRC | 15:40 | |
| clarkb | this portion of the config validation is somewhat complicated in zuul, looks like connection drivers can specify their own schemas here | 15:40 |
| *** john-davidge has quit IRC | 15:40 | |
| *** john-dav_ has joined #openstack-infra | 15:40 | |
| pabelanger | IIRC, it changes a little in zuulv3 | 15:41 |
| *** john-dav_ is now known as john-davidge | 15:41 | |
| clarkb | ya but this is failing in v2 right? | 15:41 |
| *** florianf has joined #openstack-infra | 15:41 | |
| clarkb | gerrit_reporter = v.Any(str, v.Schema({}, extra=True)) is what I think the valid schema is for what fails | 15:41 |
| *** jianghuaw has joined #openstack-infra | 15:42 | |
| *** Rockyg has joined #openstack-infra | 15:42 | |
| *** EricGonc_ has quit IRC | 15:42 | |
| clarkb | schema = Schema({}, extra=ALLOW_EXTRA) # don't do this schema = Schema(dict) # do this instead maybe we are hitting that | 15:45 |
| *** jpich has quit IRC | 15:45 | |
| dhellmann | pabelanger : not yet, but I dropped off to go to the dentist a while ago so I wasn't expecting anything right off. If you'll have time to help today, that'd be great. | 15:46 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Pass in client name information to keystoneauth https://review.openstack.org/456268 | 15:47 |
| clarkb | trying to reproduce in zuul test suite to see | 15:48 |
| clarkb | Exception: Error has already been tested: expected a list for dictionary value @ data['pipelines'][0]['failure']['review_gerrit'] <- easy reproduction using tox -re py27 -- test_layouts | 15:49 |
| *** Julien-zte has quit IRC | 15:49 | |
| *** vhosakot has joined #openstack-infra | 15:50 | |
| *** Julien-zte has joined #openstack-infra | 15:50 | |
| *** flepied has quit IRC | 15:51 | |
| cmurphy | asettle: AJaeger https://review.openstack.org/#/c/456263/ passes | 15:51 |
| cmurphy | I don't have a good explanation for the why, there were 104 commits between chardet 2.3.0 and 3.0.0 | 15:52 |
| asettle | cmurphy: :o | 15:52 |
| asettle | What | 15:52 |
| asettle | What | 15:52 |
| cmurphy | ¯\_(ツ)_/¯ | 15:52 |
| asettle | What | 15:52 |
| * asettle glares at sphinx | 15:52 | |
| asettle | I've had *enough* of its shenanigans this week. | 15:52 |
| asettle | :; | 15:52 |
| asettle | :P * | 15:52 |
| * asettle merges | 15:53 | |
| *** Sukhdev_ has joined #openstack-infra | 15:55 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Handle dict extras in Voluptuous Schema https://review.openstack.org/456270 | 15:55 |
| clarkb | pabelanger: AJaeger ^ that seems to fix it locally for me using tox -re py27 -- test_layouts as test | 15:55 |
| pabelanger | clarkb: great | 15:56 |
| pabelanger | clarkb: mordred: do you mind reviewing, +3 https://review.openstack.org/#/c/455803/ for todays images. Now that DIB is managing those files | 15:57 |
| *** tesseract has quit IRC | 15:58 | |
| mgagne | mwhahaha: re #openstack-puppet sure, is there a procedure I can follow? | 15:59 |
| *** ralonsoh_ has joined #openstack-infra | 15:59 | |
| mwhahaha | mgagne: not sure, maybe you could at least give infra perms on that channel as i believe you're the only owner | 16:00 |
| *** Julien-zte has quit IRC | 16:01 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul feature/zuulv3: Handle dict extras in Voluptuous Schema https://review.openstack.org/456273 | 16:01 |
| *** jascott1_ has joined #openstack-infra | 16:01 | |
| clarkb | pabelanger: trade you ^ | 16:01 |
| *** ralonsoh has quit IRC | 16:01 | |
| mwhahaha | mgagne: there is https://epir.at/2016/06/26/redirecting-freenode-channels/ | 16:01 |
| *** Julien-zte has joined #openstack-infra | 16:02 | |
| *** marst_ has joined #openstack-infra | 16:03 | |
| *** dharinic has quit IRC | 16:03 | |
| *** marst has quit IRC | 16:03 | |
| *** Julien-zte has quit IRC | 16:04 | |
| mordred | pabelanger, clarkb: you know - as a next step after we're done with de-puppet - and on the path to zuul v3 - we should rewrite all of the jenkins slave_scripts as roles, install them on the 2.5 launcher and put in a hacky translation piece for the jjb translator to just emit a role reference in the generated playbook instead of a shell task to run the slave_script | 16:05 |
| mordred | seems like a thing we could do one script at a time and end up with more real v3 content | 16:05 |
| mordred | (similar to the devstack-gate translation) | 16:05 |
| *** aarefiev is now known as aarefiev_afk | 16:06 | |
| *** Julien-zte has joined #openstack-infra | 16:06 | |
| *** jcoufal_ has joined #openstack-infra | 16:06 | |
| *** jcoufal has quit IRC | 16:06 | |
| openstackgerrit | Merged openstack-infra/system-config master: Remove /usr/local/jenkins from single_use_slave.pp https://review.openstack.org/455803 | 16:07 |
| pabelanger | ya, should get easier moving forward | 16:07 |
| *** baoli has quit IRC | 16:07 | |
| *** Julien-zte has quit IRC | 16:08 | |
| clarkb | some of them really want to be in $programminglanguage though so shelling out likely won't go away soon. Specifically thinking of the translation tooling stuff | 16:08 |
| *** rwsu has joined #openstack-infra | 16:09 | |
| *** rossella_s has quit IRC | 16:09 | |
| *** Julien-zte has joined #openstack-infra | 16:09 | |
| mgagne | mwhahaha: added openstackinfra as founder | 16:10 |
| mwhahaha | mgagne: thanks | 16:10 |
| mgagne | mwhahaha: will you take it from here? you can remove myself | 16:10 |
| mwhahaha | mgagne: sure | 16:10 |
| mgagne | mwhahaha: thanks =) | 16:10 |
| *** tmorin has quit IRC | 16:11 | |
| *** Julien-zte has quit IRC | 16:12 | |
| *** Kevin_Zheng has quit IRC | 16:13 | |
| openstackgerrit | Merged openstack-infra/zuul master: Handle dict extras in Voluptuous Schema https://review.openstack.org/456270 | 16:13 |
| *** Julien-zte has joined #openstack-infra | 16:14 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul feature/zuulv3: Update voluptuous requirement https://review.openstack.org/456281 | 16:15 |
| mordred | clarkb: yah - I think you're right about that - but even if we made them stupid roles that contained a bash script, copied it to the node and ran the script, it would make them forward compat with zuulv3, and it would remove one more thing that requires an image rebuild to update | 16:15 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul master: Update voluptuous requirement https://review.openstack.org/456282 | 16:16 |
| mordred | in fact, we could probably make all of them actually stupid roles that did exactly that to get the mechanism in place, then there would be easy refactoring work available in the future - which we could hold off on until after v3 was live even | 16:16 |
| *** baoli has joined #openstack-infra | 16:16 | |
| *** Julien-zte has quit IRC | 16:18 | |
| *** Julien-zte has joined #openstack-infra | 16:18 | |
| *** flepied has joined #openstack-infra | 16:18 | |
| *** Julien-zte has quit IRC | 16:18 | |
| *** esikachev has joined #openstack-infra | 16:21 | |
| *** Julien-zte has joined #openstack-infra | 16:21 | |
| *** Julien-zte has quit IRC | 16:22 | |
| clarkb | ya that would be a trivial way to get it in place | 16:22 |
| *** smarcet has quit IRC | 16:22 | |
| *** florianf has quit IRC | 16:23 | |
| *** baoli has quit IRC | 16:24 | |
| *** jcoufal has joined #openstack-infra | 16:25 | |
| *** jcoufal_ has quit IRC | 16:26 | |
| *** dtantsur is now known as dtantsur|afk | 16:26 | |
| *** dharinic has joined #openstack-infra | 16:27 | |
| *** gongysh has quit IRC | 16:29 | |
| *** baoli has joined #openstack-infra | 16:31 | |
| *** Sukhdev_ has quit IRC | 16:31 | |
| *** esikachev has quit IRC | 16:31 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul feature/zuulv3: Handle dict extras in Voluptuous Schema https://review.openstack.org/456273 | 16:34 |
| openstackgerrit | Clark Boylan proposed openstack-infra/zuul feature/zuulv3: Update voluptuous requirement https://review.openstack.org/456281 | 16:34 |
| *** lucasagomes is now known as lucas-afk | 16:36 | |
| *** gk_1wm_su has joined #openstack-infra | 16:36 | |
| *** gk_1wm_su has left #openstack-infra | 16:37 | |
| *** beslemon has quit IRC | 16:39 | |
| openstackgerrit | Paul Belanger proposed openstack/diskimage-builder master: Add DIB_DISTRIBUTION_MIRROR support for centos-minimal https://review.openstack.org/455852 | 16:43 |
| *** david-lyle has joined #openstack-infra | 16:45 | |
| pabelanger | centos-7 and ubuntu-trusty currently building now | 16:45 |
| *** gyee has joined #openstack-infra | 16:46 | |
| *** ociuhandu has quit IRC | 16:47 | |
| *** e0ne has quit IRC | 16:49 | |
| *** psachin has quit IRC | 16:49 | |
| pabelanger | clarkb: looks like dib 2.3.2 didn't get uploaded to pypi | 16:51 |
| pabelanger | wonder if it was dns issues like above | 16:51 |
| *** tongli has quit IRC | 16:52 | |
| pabelanger | http://logs.openstack.org/1f/1f200386061fe145645c9f8ffde627829cf998c9/release/diskimage-builder-pypi-both-upload/077722e/console.html | 16:53 |
| openstackgerrit | Clark Boylan proposed openstack-infra/gear master: Don't cap pbr https://review.openstack.org/456297 | 16:54 |
| clarkb | pabelanger: mordred ^ that will also need a release | 16:54 |
| pabelanger | clarkb: https://review.openstack.org/#/c/439755 :) | 16:54 |
| clarkb | pabelanger: name or service not known looks like dns | 16:54 |
| clarkb | gah I even reviewed it | 16:54 |
| *** fandi has quit IRC | 16:54 | |
| pabelanger | clarkb: ya, checking logs to see why | 16:54 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/puppet-mqtt_statsd master: Add statsd_prefix default of mosquitto https://review.openstack.org/456298 | 16:55 |
| clarkb | ok abandoned my chagne, mordred can you review https://review.openstack.org/#/c/439755 | 16:55 |
| clarkb | pabelanger: did dns break due to changes to images? | 16:55 |
| *** hashar has quit IRC | 16:55 | |
| *** beslemon has joined #openstack-infra | 16:56 | |
| pabelanger | clarkb: that is what I am checking | 16:56 |
| pabelanger | unbound is still running | 16:56 |
| *** jcoufal has quit IRC | 16:56 | |
| clarkb | oh and that job runs on static host | 16:56 |
| clarkb | so that constrains its to chagnes to the puppet | 16:57 |
| *** jcoufal has joined #openstack-infra | 16:57 | |
| pabelanger | dig upload.pypi.io works | 16:57 |
| clarkb | ya just checked that myself | 16:57 |
| pabelanger | let me see how to requeue the upload | 16:57 |
| clarkb | maybe they changed dns (upload.pypi.io seems new) and it didn't work for a time | 16:57 |
| pabelanger | possible | 16:58 |
| openstackgerrit | Felipe Monteiro proposed openstack-infra/project-config master: Create heat-specific patrole gate https://review.openstack.org/456300 | 16:58 |
| *** beslemon has quit IRC | 16:59 | |
| pabelanger | clarkb: should we retrigger or retag? | 16:59 |
| clarkb | pabelanger: zuul enqueue-ref will run all the pipeline jobs or there is zuul/tools/trigger-job.py to trigger just the one job | 16:59 |
| *** smarcet has joined #openstack-infra | 16:59 | |
| clarkb | pabelanger: I would use zuul/tools/trigger-job.py I think | 16:59 |
| pabelanger | ok | 17:00 |
| *** bh526r has joined #openstack-infra | 17:00 | |
| clarkb | since it looks like we built the artifcats fine, its just the last step of publishing to pypi that failed | 17:00 |
| openstackgerrit | Merged openstack-infra/gear master: Update hacking dependency https://review.openstack.org/439761 | 17:00 |
| openstackgerrit | Merged openstack-infra/gear master: Uncap pbr dependency https://review.openstack.org/439755 | 17:00 |
| clarkb | pabelanger: sounded like the release team ran into this too? do we have a list from them of things that should be triggered? | 17:00 |
| pabelanger | clarkb: ya, I have that on my plate to do | 17:00 |
| pabelanger | I can do that next | 17:01 |
| clarkb | https://etherpad.openstack.org/p/fSNrLxz3Hx found it | 17:01 |
| clarkb | pabelanger: ya ithink use dib as guinea pig and if that works then likely upstream dns problem and we can move ahead with the others | 17:01 |
| *** markus_z has quit IRC | 17:01 | |
| pabelanger | now to get all the params right | 17:01 |
| *** esikachev has joined #openstack-infra | 17:02 | |
| clarkb | kfox1111: any luck collecting the network info off the subnode when those jobs fail? | 17:02 |
| *** beslemon has joined #openstack-infra | 17:03 | |
| *** mhickey has quit IRC | 17:03 | |
| *** psachin has joined #openstack-infra | 17:06 | |
| *** jascott1_ has quit IRC | 17:07 | |
| *** kzaitsev_mb has joined #openstack-infra | 17:07 | |
| *** jascott1_ has joined #openstack-infra | 17:07 | |
| *** esikachev has quit IRC | 17:09 | |
| pabelanger | clarkb: okay, think it worked: https://pypi.python.org/pypi/diskimage-builder | 17:11 |
| clarkb | pabelanger: looks like it to me | 17:11 |
| *** psachin_ has joined #openstack-infra | 17:11 | |
| pabelanger | let me start on release team failures | 17:11 |
| clarkb | pabelanger: the ones dhellmann pointed out are trickier because they have the jobs that run after for announcing and stuff, but you can do them in waves. Do all the publish, all the announce, etc | 17:11 |
| *** jascott1_ has quit IRC | 17:12 | |
| dhellmann | pabelanger, clarkb : doing it in waves makes sense | 17:13 |
| dhellmann | ttx: ^^ | 17:13 |
| dhellmann | pabelanger, clarkb : it's also fairly easy for me to run the announce job by hand locally | 17:13 |
| ttx | ok, cool looks like it's under control and more a question of how we reenqueue the jobs | 17:13 |
| *** psachin_ has quit IRC | 17:14 | |
| dhellmann | ttx: yeah, it looked like a dns failure, and I've been able to upload a release of my own using twine here | 17:14 |
| *** beslemon_ has joined #openstack-infra | 17:14 | |
| clarkb | ya and we were just able to publish new dib release. Looking at the name I have a hunch that pypi transitioned to a new server/frontend/name and dns hadn't quite propogated yet | 17:14 |
| pabelanger | http://logs.openstack.org/9e/9e376560e686dc604cdf35b365db24b0cf16d12d/release/gnocchi-pypi-both-upload/run1/console.html | 17:15 |
| *** shardy has quit IRC | 17:15 | |
| pabelanger | clarkb: ^ | 17:15 |
| pabelanger | https://pypi.python.org/pypi/gnocchi | 17:15 |
| pabelanger | dhellmann: any ideas on the exceptions?^ | 17:16 |
| dhellmann | ugh | 17:16 |
| dhellmann | that means at least one file was successfully uploaded on the previous run | 17:17 |
| clarkb | guessing, but pypi cached the fact were were going to upload things then didn't notice the uploads never happened | 17:17 |
| *** beslemon has quit IRC | 17:17 | |
| clarkb | dhellmann: if you read the logs they weren't though | 17:17 |
| *** neiljerram has quit IRC | 17:17 | |
| clarkb | oh wait | 17:17 |
| *** Gravious has quit IRC | 17:17 | |
| clarkb | this release is older than current release which is why it doesn't show up on pypi page but they are on https://pypi.python.org/simple/gnocchi/ | 17:17 |
| dhellmann | clarkb : yeah, https://pypi.python.org/pypi/gnocchi/3.0.5 | 17:18 |
| clarkb | so maybe thing to do is just double check the hashes? | 17:18 |
| dhellmann | I wonder if the others were, too | 17:18 |
| clarkb | and if they match up then we are good for pypi publishing | 17:18 |
| *** neiljerram has joined #openstack-infra | 17:19 | |
| *** psachin_ has joined #openstack-infra | 17:19 | |
| *** Gravious has joined #openstack-infra | 17:19 | |
| pabelanger | okay, holding off for the moment | 17:19 |
| clarkb | lets double check what made it into pypi and check hashes (sha1 or md5 or whatever) then see what needs reuploading | 17:20 |
| *** psachin_ has quit IRC | 17:21 | |
| clarkb | gnocchiclient is on pypi too | 17:21 |
| dhellmann | it looks like only the tricircleclient actually failed to upload | 17:21 |
| dhellmann | links to the others are in the etherpad | 17:21 |
| *** psachin_ has joined #openstack-infra | 17:22 | |
| dhellmann | while we're working on this, how about if I approve one more release so we can see if the uploads are actually working now | 17:22 |
| clarkb | sure. I'm going to check gnocciclient md5 sums now | 17:23 |
| dhellmann | oh, good, I didn't actually check the hashes | 17:23 |
| *** tushar has joined #openstack-infra | 17:23 | |
| tushar | Hi | 17:23 |
| *** psachin_ has quit IRC | 17:23 | |
| tushar | I am facing some issue in nodepool | 17:23 |
| tushar | Nodepool is not able to conect to the provider | 17:24 |
| tushar | any one has any idea | 17:24 |
| tushar | error snippet - File "/usr/local/lib/python2.7/dist-packages/nodepool/nodepool.py", line 366, in run self.nodepool._deleteNode(session, node) File "/usr/local/lib/python2.7/dist-packages/nodepool/nodepool.py", line 2251, in _deleteNode manager.cleanupServer(node.external_id) File "/usr/local/lib/python2.7/dist-packages/nodepool/provider_manager.py", line 604, in cleanupServer for ip in self.listFloatingIPs(): | 17:24 |
| pabelanger | tushar: its better to post into pastebin then like the log | 17:25 |
| *** Gravious has quit IRC | 17:26 | |
| tushar | pabelanger: ok | 17:26 |
| *** efoley_ has quit IRC | 17:26 | |
| clarkb | dhellmann: the md5sums do differ | 17:27 |
| clarkb | trying to sort out what is different now | 17:27 |
| *** kzaitsev_mb has quit IRC | 17:28 | |
| dhellmann | clarkb : are you comparing to the uploaded files or the newly built ones? | 17:28 |
| dhellmann | we might need to just retag all of these projects at the same point | 17:29 |
| clarkb | dhellmann: I grabbed the files on tarballs and the files from pypi and compared those | 17:29 |
| dhellmann | ok | 17:29 |
| clarkb | dhellmann: those should be identical right? | 17:29 |
| dhellmann | did we just regenerate files on the tarballs server by retriggering the job? or does that job only do the upload? | 17:29 |
| dhellmann | I don't remember which job does what | 17:30 |
| *** dtardivel has quit IRC | 17:30 | |
| tushar | https://pastebin.com/wW6jDX4w | 17:30 |
| *** [HeOS] has joined #openstack-infra | 17:30 | |
| *** gk_1wm_su has joined #openstack-infra | 17:31 | |
| *** beslemon_ has quit IRC | 17:31 | |
| clarkb | dhellmann: the build job will make the wheel and sdist and upload them to tarballs then the upload ot pypi job grabs thos and sends to pypi | 17:31 |
| dhellmann | ok | 17:31 |
| pabelanger | tushar: must be running an older version of nodepool | 17:31 |
| dhellmann | so, yeah, I would expect them to match | 17:31 |
| *** gk_1wm_su has left #openstack-infra | 17:31 | |
| dhellmann | it sounds like maybe we have partially uploaded files or something? | 17:32 |
| pabelanger | tushar: check your novaclient dependency, possible it got upgraded? | 17:32 |
| pabelanger | tushar: newer versions of nodepool use shade as our client to the cloud | 17:32 |
| dhellmann | clarkb, pabelanger : the solum client release upload job completed successfully | 17:32 |
| *** HeOS has quit IRC | 17:32 | |
| pabelanger | great | 17:32 |
| dhellmann | if we have bad files on pypi, we probably want to retag anyway, so why don't I go ahead and set that up for all 4 of the failures we had | 17:33 |
| *** trown is now known as trown|lunch | 17:33 | |
| pabelanger | no objection from me | 17:33 |
| *** psachin has quit IRC | 17:34 | |
| openstackgerrit | Felipe Monteiro proposed openstack-infra/project-config master: Create heat-specific patrole gate https://review.openstack.org/456300 | 17:34 |
| tushar | pabelanger : can you give me some links from where I can check the dependencies required for the particular version of nodepool | 17:34 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove unused python33 builder https://review.openstack.org/456318 | 17:34 |
| dhellmann | less work all around, and there's no sense announcing broken releases | 17:34 |
| dhellmann | ttx: have any thoughts on that? | 17:35 |
| pabelanger | tushar: they will be listed in the requirements.txt file for a specific release http://git.openstack.org/cgit/openstack-infra/nodepool | 17:35 |
| *** [HeOS] has quit IRC | 17:36 | |
| *** HeOS_ has joined #openstack-infra | 17:36 | |
| *** ralonsoh_ has quit IRC | 17:36 | |
| *** sambetts is now known as sambetts|afk | 17:36 | |
| tushar | pabelanger | 17:37 |
| clarkb | dhellmann: ok tracked down the diff let me get a second to paste it and then you will likeyl decide its not worth worrying about. I do not know what it ended up changing though | 17:37 |
| dhellmann | oh, it looks like the gnocchi team is taking over their releases, so I'll just forward those things to jd_ | 17:37 |
| dhellmann | clarkb : ok | 17:38 |
| ttx | dhellmann: yeah, no point in announcing broken releases | 17:38 |
| tushar | pabelanger: ok. In my setup nodepool version is - nodepool (0.2.0) | 17:38 |
| clarkb | dhellmann: http://paste.openstack.org/show/606329/ basically dict order got flipped around so we must be generating that file more than once somewhere | 17:38 |
| dhellmann | clarkb : ah | 17:38 |
| clarkb | I am guessing the only affect that will have in real world is possibly invalidating the gpg key? or maybe the process of signing is what causes that to happen? | 17:38 |
| *** Gravious has joined #openstack-infra | 17:39 | |
| tushar | pabelanger : and novaclient is - python-novaclient (8.0.0) | 17:39 |
| dhellmann | dunno | 17:39 |
| pabelanger | tushar: ya, 2.0 is a few years old. It is likely your dependencies have upgraded too much for it | 17:39 |
| pabelanger | tushar: you might want to consider upgrading nodepool, or downgrading your dependencies | 17:39 |
| tushar | pabelanger : okk | 17:40 |
| *** tosky has quit IRC | 17:40 | |
| *** beslemon has joined #openstack-infra | 17:40 | |
| *** gk_1wm_su has joined #openstack-infra | 17:41 | |
| dhellmann | clarkb : do the signatures on pypi match the files on pypi? | 17:41 |
| *** esikachev has joined #openstack-infra | 17:41 | |
| tushar | pabelanger: You mean whatever there is in requirement.txt for nodepol 0.2.0 ...that needs to be installed ? | 17:41 |
| *** gk_1wm_su has left #openstack-infra | 17:42 | |
| *** esberglu has quit IRC | 17:42 | |
| clarkb | dhellmann: oh good question let me check | 17:42 |
| pabelanger | tushar: right, this is the requirements for 2.0.0 http://git.openstack.org/cgit/openstack-infra/nodepool/tree/requirements.txt?h=0.2.0 | 17:43 |
| mordred | tushar, pabelanger: novaclient 8.0 removed nova-net support | 17:43 |
| clarkb | dhellmann: they differ so maybe I am looking at the wrong thing in tarballs? | 17:43 |
| pabelanger | but as you see, there is limited caps. So, when we did 2.0.0 we used novaclient 2.21.0 | 17:43 |
| mordred | the latest shade works around that | 17:44 |
| dhellmann | clarkb : if the hash on pypi doesn't match the file on pypi, that seems like an issue on pypi. Does the hash match the file on the tarballs server? | 17:44 |
| AJaeger | team, could you review https://review.openstack.org/456081 - to remove a completely useless mitaka translation sync, please? | 17:44 |
| mordred | but I doubt that'll help much if there is a pre-shade nodepool in use | 17:44 |
| clarkb | dhellmann: the md5sum in pypi matches the file in pypi | 17:44 |
| dhellmann | oh, you said they differ | 17:44 |
| clarkb | dhellmann: sorry, the asc and md5sums differ between tarballs and pypi | 17:45 |
| dhellmann | aha | 17:45 |
| *** jtomasek has joined #openstack-infra | 17:45 | |
| dhellmann | that's odd | 17:45 |
| tushar | pabelanger : ok | 17:45 |
| mordred | tushar: yes - your issue is with novaclient 8.0.0 | 17:45 |
| dhellmann | clarkb : I would not expect anything to repackage the sdist or wheel as part of the signing or upload process | 17:45 |
| mordred | if you downgrade python-novaclient to <8.0.0 nodepool will start working for you again | 17:46 |
| *** esikachev has quit IRC | 17:46 | |
| *** jtomasek has quit IRC | 17:46 | |
| clarkb | dhellmann: right thats why I'm confused btu something is causing the pbr.json to be rewritten with a different dict ordering causing the hasehse to not match | 17:46 |
| *** jtomasek has joined #openstack-infra | 17:46 | |
| mordred | when we release nodepool for zuul v3 which is entirely shade based, this will not be a problem | 17:46 |
| mordred | but for now, the version of nodepool you are running will not work with novaclient >=8.0.0 | 17:46 |
| *** jtomasek has quit IRC | 17:47 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove periodic mitaka bitrot jobs https://review.openstack.org/456324 | 17:48 |
| *** esikachev has joined #openstack-infra | 17:48 | |
| tushar | morderd : ok. which version of novaclient can be used ? because in requirement.txt its like - python-novaclient>=2.21.0 | 17:49 |
| *** jtomasek has joined #openstack-infra | 17:49 | |
| mordred | tushar: 7.1.0 should work just fine | 17:49 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Handle dict extras in Voluptuous Schema https://review.openstack.org/456273 | 17:49 |
| tushar | mordred : ok | 17:50 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Update voluptuous requirement https://review.openstack.org/456281 | 17:50 |
| *** ociuhandu has joined #openstack-infra | 17:50 | |
| *** jtomasek_ has joined #openstack-infra | 17:50 | |
| AJaeger | clarkb, tox -e zuul still fails with the voluptous error in project-config. Do we need a release of zuul? | 17:51 |
| clarkb | AJaeger: yes we will need a release with my change | 17:52 |
| *** jtomasek has quit IRC | 17:53 | |
| AJaeger | ok | 17:54 |
| dhellmann | who owns dib these days? tripleo or infra? | 17:54 |
| AJaeger | dhellmann: infra now | 17:54 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove periodic mitaka bitrot jobs https://review.openstack.org/456324 | 17:55 |
| dhellmann | AJaeger : thanks | 17:55 |
| *** Goneri has quit IRC | 17:55 | |
| AJaeger | clarkb: does it make sense to stop the periodic mitaka jobs? ^ | 17:55 |
| dhellmann | pabelanger, clarkb, AJaeger : since the dib release was manually tagged, the release team's policy is to let the owner deal with the failure. So, if you want to retag 2.3.2 to ensure a clean build, you may want to do that. | 17:55 |
| *** jtomasek_ has quit IRC | 17:56 | |
| *** esberglu has joined #openstack-infra | 17:56 | |
| dhellmann | pabelanger, clarkb : the gnocchi releases were also manually tagged, so I've emailed the telemetry team to handle any updates they want to make | 17:56 |
| dhellmann | I will re-process the tricircleclient release | 17:56 |
| AJaeger | let's tell ianw - he's now on vacation but wasn't aware to use releases process | 17:56 |
| AJaeger | dhellmann: could you send him an email, please? or better email openstack-infra mailing list? | 17:57 |
| dhellmann | AJaeger : ok, I'll send the same email to the ML that I did for the gnocchi team | 17:57 |
| pabelanger | dhellmann: clarkb: sounds good, thanks | 17:58 |
| *** esberglu has quit IRC | 17:58 | |
| *** esikachev has quit IRC | 17:58 | |
| AJaeger | thanks, dhellmann | 17:58 |
| *** esberglu has joined #openstack-infra | 17:58 | |
| *** neiljerram has quit IRC | 17:58 | |
| dhellmann | sent | 17:59 |
| dhellmann | pabelanger, clarkb : thanks for your help with debugging those releases | 17:59 |
| *** annegentle has quit IRC | 18:01 | |
| *** annegentle has joined #openstack-infra | 18:01 | |
| *** mattmceuen has joined #openstack-infra | 18:02 | |
| *** Apoorva has joined #openstack-infra | 18:02 | |
| *** Apoorva has quit IRC | 18:02 | |
| *** rbrndt has quit IRC | 18:03 | |
| *** Apoorva has joined #openstack-infra | 18:03 | |
| *** baoli has quit IRC | 18:03 | |
| *** baoli has joined #openstack-infra | 18:04 | |
| *** krtaylor has quit IRC | 18:05 | |
| *** Goneri has joined #openstack-infra | 18:05 | |
| *** ociuhandu has quit IRC | 18:05 | |
| *** ociuhandu has joined #openstack-infra | 18:05 | |
| *** ociuhandu has quit IRC | 18:05 | |
| *** tosky has joined #openstack-infra | 18:05 | |
| *** ociuhandu has joined #openstack-infra | 18:05 | |
| *** MasterOfBugs has joined #openstack-infra | 18:06 | |
| *** pramodrj07 has joined #openstack-infra | 18:06 | |
| *** MasterOfBugs has quit IRC | 18:06 | |
| *** MasterOfBugs has joined #openstack-infra | 18:06 | |
| *** pramodrj07 has joined #openstack-infra | 18:06 | |
| *** jcoufal_ has joined #openstack-infra | 18:09 | |
| *** esikachev has joined #openstack-infra | 18:10 | |
| *** dharinic has quit IRC | 18:10 | |
| *** akscram has quit IRC | 18:10 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/project-config master: Remove thin param in puppet builds https://review.openstack.org/456334 | 18:11 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Remove /opt/requirements from slave_common.pp https://review.openstack.org/455003 | 18:11 |
| *** rderose has joined #openstack-infra | 18:11 | |
| pabelanger | mordred: clarkb: ^ should be an easy review | 18:11 |
| *** jcoufal has quit IRC | 18:11 | |
| *** akscram has joined #openstack-infra | 18:12 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config master: Remove thin parameter from single_use_slave https://review.openstack.org/456337 | 18:12 |
| *** jcoufal has joined #openstack-infra | 18:14 | |
| *** esikachev has quit IRC | 18:14 | |
| *** esikachev has joined #openstack-infra | 18:16 | |
| *** jcoufal_ has quit IRC | 18:16 | |
| *** ociuhandu has quit IRC | 18:17 | |
| *** ociuhandu has joined #openstack-infra | 18:17 | |
| *** ociuhandu has quit IRC | 18:17 | |
| *** ociuhandu has joined #openstack-infra | 18:18 | |
| openstackgerrit | Merged openstack-infra/system-config master: Remove puppet pinning logic https://review.openstack.org/455017 | 18:18 |
| clarkb | pabelanger: I looked up the history behind that. We were/did use requirements repo for mirro selection? huh | 18:18 |
| pabelanger | clarkb: ya, cmurphy found the history of it. Never knew my self | 18:20 |
| cmurphy | it might have been used for something else, that was all i could find | 18:20 |
| *** esikachev has quit IRC | 18:21 | |
| pabelanger | ya, codesearch was also empty | 18:21 |
| clarkb | with /opt/openstack/requirements being part of our builds I think its fine | 18:22 |
| clarkb | if we break anything we just point them at ^ | 18:22 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config master: Remove ssh_config NoRoaming setting https://review.openstack.org/455018 | 18:22 |
| cmurphy | that should also be an easy one ^ | 18:22 |
| *** trown|lunch is now known as trown | 18:23 | |
| cmurphy | and https://review.openstack.org/#/c/455014/ | 18:23 |
| pabelanger | +2 | 18:23 |
| *** esikachev has joined #openstack-infra | 18:23 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove most python 34 jobs https://review.openstack.org/456340 | 18:23 |
| AJaeger | what a spring cleaning time ;) Thanks, cmurphy, clarkb, pabelanger ! | 18:23 |
| cmurphy | it's very therapeutic to delete things | 18:24 |
| pabelanger | thanks to cmurphy! this will make it much easier to bring new images online | 18:24 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DONT REVIEW: test host discovery https://review.openstack.org/456342 | 18:24 |
| AJaeger | you all inspired me ;) unfortunately my contribution is tiny, see https://review.openstack.org/456318 and https://review.openstack.org/#/q/status:open+project:openstack-infra/project-config+branch:master+topic:mitaka-eol | 18:25 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config master: Move puppet service disablement to o_p::server https://review.openstack.org/455016 | 18:27 |
| *** winggundamth_ has quit IRC | 18:27 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove most python 34 jobs https://review.openstack.org/456340 | 18:28 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Add jenkins user sudoers file to nodepool-base https://review.openstack.org/456346 | 18:29 |
| clarkb | https://review.openstack.org/#/c/455014/1 makes me paranoid for some reason. Like I'm going to prevent ssh from working afterwards | 18:29 |
| clarkb | (I checked review.o.o and its reflects the absent state so removing the blocks should be a noop) | 18:29 |
| sdague | clarkb: https://review.openstack.org/#/c/455831/ (also mordred ) systemd log fix | 18:30 |
| *** electrofelix has quit IRC | 18:30 | |
| openstackgerrit | Merged openstack-infra/system-config master: Remove ssh_config NoRoaming setting https://review.openstack.org/455018 | 18:30 |
| *** EricGonczer_ has joined #openstack-infra | 18:30 | |
| cmurphy | clarkb: you could keep a session open while it merges, if it does something weird to prevent new logins it at least shouldn't kill an existing one | 18:31 |
| *** esikachev has quit IRC | 18:31 | |
| clarkb | cmurphy: ya except there are about 100 hosts to worry about. Also having read and checked things I'm fairly certain it is a noop | 18:32 |
| *** rderose has quit IRC | 18:32 | |
| clarkb | I'm just paranoidops | 18:32 |
| cmurphy | :) | 18:32 |
| clarkb | cmurphy: pabelanger I have approved it, would be great if people could keep an eye on general ssh things but ya should be fine | 18:33 |
| *** rderose has joined #openstack-infra | 18:33 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove most python 34 jobs https://review.openstack.org/456340 | 18:33 |
| pabelanger | clarkb: ack | 18:33 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Remove /etc/sudoers.d/jenkins-sudo support https://review.openstack.org/456349 | 18:34 |
| *** tphummel has quit IRC | 18:36 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/project-config master: Remove unused params in puppet builds https://review.openstack.org/456334 | 18:36 |
| openstackgerrit | Merged openstack-infra/system-config master: Remove old ssh_authorized_key resources https://review.openstack.org/455014 | 18:36 |
| AJaeger | pabelanger: FYI, https://review.openstack.org/454916 is in merge conflict | 18:37 |
| *** gk_1wm_su has joined #openstack-infra | 18:37 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config master: Remove thin and mysql params from single_use_slave https://review.openstack.org/456337 | 18:38 |
| AJaeger | cmurphy: you have a nodepool/elements/build.log in https://review.openstack.org/#/c/456334 | 18:38 |
| *** gk_1wm_su has left #openstack-infra | 18:38 | |
| cmurphy | AJaeger: wow oops | 18:38 |
| pabelanger | AJaeger: ya, doing that next | 18:38 |
| openstackgerrit | Vitaly Gridnev proposed openstack-infra/project-config master: sahara scenario jobs: fix mod_wsgi job https://review.openstack.org/456352 | 18:39 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/project-config master: Remove unused params in puppet builds https://review.openstack.org/456334 | 18:40 |
| *** Zhongjun__ has quit IRC | 18:41 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config master: Remove thin and mysql params from single_use_slave https://review.openstack.org/456337 | 18:43 |
| pabelanger | clarkb: did we want to tag another dib release, to address pypi md5 issue? | 18:43 |
| pabelanger | I think ianw is on PTO | 18:43 |
| hongbin | hi, could anyone add me to this group https://review.openstack.org/#/admin/groups/1751,members , thx | 18:43 |
| pabelanger | hongbin: can you link the project creation review | 18:44 |
| clarkb | pabelanger: do we think another release would solve it? | 18:44 |
| hongbin | https://review.openstack.org/#/c/453230/ | 18:44 |
| clarkb | pabelanger: I have a hunch its something that happens every time due to some pbr thing | 18:45 |
| pabelanger | clarkb: oh | 18:45 |
| *** e0ne has joined #openstack-infra | 18:45 | |
| clarkb | pabelanger: problem is pbr.json is an unordered dict being serialized to json so they can be different if generated multiple times | 18:46 |
| clarkb | I guess I should make a change to pbr to make that ordered so its consistent. I will work on that now before I forget as its reasonably important | 18:46 |
| pabelanger | clarkb: okay, that make sense | 18:46 |
| mordred | clarkb: why does the json being unordered matter for pypi md5s? | 18:47 |
| openstackgerrit | Kaitlin Farr proposed openstack-infra/project-config master: Enable security configuration for barbican grenade gate https://review.openstack.org/456356 | 18:48 |
| *** krtaylor has joined #openstack-infra | 18:48 | |
| *** ihrachys is now known as ihrachys|afk | 18:49 | |
| *** jkilpatr_ has joined #openstack-infra | 18:49 | |
| clarkb | mordred: because the order can change since dicts are unordered when we serialize to json which results in different sdist and whl md5sums | 18:50 |
| *** jkilpatr has quit IRC | 18:50 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Remove puppet-ssh from zuul workers https://review.openstack.org/454916 | 18:51 |
| mordred | I mean - I get that - but the md5 sum for the sdist should be the md5 of the tarball we built the one time, and we shouldn't be rebuilding it again | 18:51 |
| mordred | like, the only thing that matters for the sdist md5 is that it's the md5 of the tarball we actually upload, right? | 18:51 |
| clarkb | mordred: I think the assertion that we shouldn't be rebuilding it again is wrong. I don't know why yet but we have different sdist and whl in tarballs than what is in pypi | 18:52 |
| *** EricGonczer_ has quit IRC | 18:52 | |
| clarkb | mordred: and the only difference is pbr.jsobn | 18:52 |
| mordred | yah - so - I'd argue we need to verify that we're not rebuilding the tarball | 18:52 |
| clarkb | yup we should do that too. But also if I make an sdist on my laptop from $tag it should match md5sum as what is on pypi | 18:52 |
| clarkb | so I want to fix pbr too | 18:52 |
| mordred | no, that's not true | 18:53 |
| clarkb | why not? | 18:53 |
| mordred | timestamps | 18:53 |
| mordred | it's a tarball | 18:53 |
| mordred | it contains dates | 18:53 |
| clarkb | in my diff of the tarballs there were no timestamps that differed | 18:53 |
| *** nadya has joined #openstack-infra | 18:53 | |
| clarkb | the only difference was pbr.json | 18:53 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Add jenkins user sudoers file to nodepool-base https://review.openstack.org/456346 | 18:53 |
| mordred | ok. so - I agree that if you build a tarball from a tag the contents should match the contents of what's in pypi | 18:53 |
| *** baoli has quit IRC | 18:54 | |
| mordred | (I'm in favor of the ordered json patch - just want to make sure we're not also having a different issue that the pbr.json thing might be showing) | 18:54 |
| pabelanger | hongbin: done | 18:54 |
| clarkb | find . -exec md5sum {} \; | sort -k 2 is what I did to delta | 18:54 |
| *** baoli has joined #openstack-infra | 18:54 | |
| mordred | clarkb: you untarred each tarballs and did that? | 18:55 |
| clarkb | mordred: ya | 18:55 |
| hongbin | pabelanger: thx! | 18:55 |
| clarkb | mordred: then I diffed the two lists and only item was pbr.json which I diffed directly | 18:55 |
| mordred | yah. so the md5 of the tarballs themselves will still differ because of the datetimes in the tar index itself | 18:55 |
| mordred | but I agree, that doesn't actually matter | 18:55 |
| mordred | and your verification is a good verifiation of content matching | 18:55 |
| *** Sukhdev_ has joined #openstack-infra | 18:56 | |
| *** beslemon has quit IRC | 18:56 | |
| *** tphummel has joined #openstack-infra | 18:56 | |
| mordred | but I fully expect our twine usage to not rebuild the tarball it fetches from tarballs.o.o and instead to just wget it and then upload it, right? so it should calculate md5 of the file it fetched from tarballs.o.o and then publish that tarball and the md5 it calculated | 18:57 |
| clarkb | mordred: but I also agree that we need to dig through what our build + sign + publish jobs do to make sure we don't do something silly | 18:57 |
| mordred | or do I completely misunderstand that | 18:57 |
| *** nadya has quit IRC | 18:57 | |
| mordred | cool | 18:57 |
| mordred | I'll look at that real quick | 18:57 |
| clarkb | mordred: I'm wondering if the signing stteps introduced something new | 18:57 |
| AJaeger | pabelanger: do you want to tag manually - or use releases repo and let dhellmann do it? | 18:57 |
| clarkb | mordred: but could be anywhere in that process really | 18:57 |
| mordred | clarkb: maybe we're calculating the md5 then signing | 18:57 |
| mordred | or something | 18:57 |
| *** nadya has joined #openstack-infra | 18:57 | |
| *** beslemon has joined #openstack-infra | 18:58 | |
| pabelanger | AJaeger: lets see what mordred and clarkb come up with. As I think it is related to their discussions | 18:58 |
| *** dharinic has joined #openstack-infra | 18:58 | |
| clarkb | I think zuul is direct release so we'd tag it. But understanding how we got different arballs would be good first | 18:58 |
| mordred | clarkb++ | 18:58 |
| clarkb | pabelanger: you only did gnocci not gnocchclient right? when you direct enqueued? | 18:58 |
| clarkb | I was looking at gnocciclient under that assumption to rule out anything that re enqueing might have done | 18:59 |
| *** Sukhdev_ has quit IRC | 18:59 | |
| pabelanger | clarkb: ya, I left notes on the eitherpad of what I enqueued | 18:59 |
| mordred | dhellmann: do you know where the signing happens? | 19:00 |
| *** EricGonczer_ has joined #openstack-infra | 19:00 | |
| *** Sukhdev_ has joined #openstack-infra | 19:00 | |
| dhellmann | mordred : there's a special node with the keys on it | 19:01 |
| *** nadya has quit IRC | 19:01 | |
| mordred | dhellmann: yah - I was just looking for the script/job and failing because I'm lame | 19:01 |
| dhellmann | oh, yeah, let me see if I can find it easily | 19:02 |
| openstackgerrit | Clark Boylan proposed openstack-dev/pbr master: Order pbr.json content https://review.openstack.org/456361 | 19:02 |
| clarkb | mordred: ^ fyi | 19:02 |
| *** gk_1wm_su has joined #openstack-infra | 19:02 | |
| dhellmann | mordred : here's a link to a log http://logs.openstack.org/9e/9e376560e686dc604cdf35b365db24b0cf16d12d/release/gnocchi-tarball-signing/280d8cc/console.html | 19:03 |
| dhellmann | signing01 | 19:03 |
| *** gk_1wm_su has left #openstack-infra | 19:03 | |
| openstackgerrit | Kaitlin Farr proposed openstack-infra/project-config master: Enable security configuration for barbican grenade gate https://review.openstack.org/456356 | 19:04 |
| mordred | dhellmann: thank you | 19:05 |
| dhellmann | mordred, clarkb : another thing we've noticed with downstream packages is that setuptools rewrites the whitespace in files like setup.cfg as part of creating the sdist | 19:06 |
| pabelanger | clarkb: left comment on 456361 | 19:06 |
| dhellmann | I don't know if that's somehow related to the other file being modified | 19:07 |
| mordred | dhellmann: yah - I gotta tell you how thrilled I am about that :) | 19:07 |
| clarkb | pabelanger: hrm does python2.6 actually work as is? maybe I should make it python2.6 friendlt? | 19:07 |
| dhellmann | yeah, causes some pain for the release delivery team at redhat | 19:07 |
| clarkb | I know lifeless wanted python2.6 but at this point is it worth it? | 19:07 |
| pabelanger | clarkb: not sure if python2.6 actually. | 19:08 |
| pabelanger | works* | 19:08 |
| mordred | his argument was that pbr is used by more than openstack | 19:08 |
| *** e0ne has quit IRC | 19:08 | |
| *** kjackal_ has quit IRC | 19:08 | |
| openstackgerrit | Clark Boylan proposed openstack-dev/pbr master: Order pbr.json content https://review.openstack.org/456361 | 19:09 |
| mordred | so we shouldn't remove 2.6 support since other people still support 2.6 in places and also use pbr | 19:09 |
| clarkb | mordred: ya but where is python26 used anymore? | 19:09 |
| clarkb | I guess rhel6 hold outs? | 19:09 |
| mordred | but - that is hard to validate in a 'if it's not tested it's broken' world | 19:09 |
| *** esikachev has joined #openstack-infra | 19:09 | |
| *** Sukhdev_ has quit IRC | 19:09 | |
| mordred | I mean, ansible JUST RECENTLY dropped 2.4 support | 19:09 |
| clarkb | mordred: ya but red hat also explicitly says stop accomodating our old stuff | 19:09 |
| pabelanger | ya, I would guess rhel6 | 19:09 |
| mordred | the world is full of people who are not able to upgrade when upstream tells them they should | 19:09 |
| mordred | clarkb: yah | 19:09 |
| mordred | I'm just trying to be a good proxy for lifeless | 19:10 |
| mordred | I _barely_ think we should support 2.7 :) | 19:10 |
| Daviey | ! | 19:10 |
| *** smarcet has quit IRC | 19:11 | |
| mordred | clarkb, dhellmann: I have doublechecked the jobs - unless twine is changing the tarball, I do not see anything other than us downloading from tarballs then uploading to pypi | 19:12 |
| dhellmann | mordred, clarkb : I notice in https://etherpad.openstack.org/p/fSNrLxz3Hx that pabelanger reran a couple of the jobs. I wonder if that had anything to do with it? | 19:12 |
| dhellmann | maybe we should spot check one of the packages released recently that did not have to be redone | 19:13 |
| clarkb | dhellmann: tahts why I looked at gnocciclient not gnocchi just to avoid any potential there | 19:13 |
| dhellmann | ah | 19:13 |
| *** esikachev has quit IRC | 19:13 | |
| clarkb | but we should spot check others | 19:14 |
| *** john-davidge has quit IRC | 19:14 | |
| openstackgerrit | Felipe Monteiro proposed openstack-infra/project-config master: Create heat-specific patrole gate https://review.openstack.org/456300 | 19:15 |
| *** kzaitsev_mb has joined #openstack-infra | 19:15 | |
| mordred | yah - but even re-running tarballs and pypi should match - unless we reran the tarball creation job and then the pypi upload failed because duplicate | 19:16 |
| *** esikachev has joined #openstack-infra | 19:16 | |
| clarkb | dib matches, but dib is also not signed | 19:16 |
| clarkb | on the tarball /me looks at whel | 19:16 |
| clarkb | ya whl matches too | 19:17 |
| clarkb | mordred: oh! | 19:17 |
| openstackgerrit | Felipe Monteiro proposed openstack-infra/project-config master: Promote patrole admin/member gates to voting https://review.openstack.org/456365 | 19:17 |
| clarkb | you may have explained why it is that way if gnocchiclien was direct released by gnocchi? | 19:17 |
| clarkb | then we came along and did our automated thing, made new tarball and wheel which are different pbr.json order and then failed to upload because they were already there /dns issue | 19:18 |
| mordred | yah | 19:18 |
| mordred | maybe we should set immutable bits on tarballs after we upload them to tarballs.o.o | 19:18 |
| mordred | to prevent double-upload/overwriting | 19:18 |
| clarkb | well in this case I think gnocchi just bypassed us entirely | 19:18 |
| mordred | sure | 19:18 |
| mordred | oh - I see what you're saying | 19:19 |
| mordred | yah - we should probably have in our process to remove the original owner of projects on pypi once openstackinfra user owns it | 19:19 |
| dhellmann | yeah, I didn't realize gnocchi was self-tagging until I went to retag it with a new version | 19:19 |
| mordred | this is sort of like having zuul gate but also letting someone push commits directly | 19:19 |
| clarkb | mordred: yes | 19:19 |
| clarkb | and then wondering wtf the merge was different sha1s | 19:20 |
| dhellmann | oh, they ran the jobs to do the upload, they just pushed a tag by hand | 19:20 |
| mordred | but still - if we re-ran the whole stack including the tarball creation | 19:20 |
| *** esikachev has quit IRC | 19:20 | |
| dhellmann | we didn't run that part, did we? | 19:20 |
| dhellmann | I thought we only reran the upload job | 19:20 |
| mordred | when you say "pushed a tag by hand" | 19:21 |
| dhellmann | https://etherpad.openstack.org/p/fSNrLxz3Hx | 19:21 |
| mordred | thanks | 19:21 |
| clarkb | yes pabelanger only ran the upload job aiui | 19:21 |
| dhellmann | they tagged and used "git push" instead of using openstack/releases | 19:21 |
| pabelanger | yup | 19:21 |
| clarkb | dhellmann: and then separately uploaded to pypi too is my guess | 19:21 |
| pabelanger | sileht looks to have created the tag, maybe he can comment | 19:21 |
| dhellmann | clarkb : well, the original failure we were trying to correct was the upload job | 19:22 |
| clarkb | dhellmann: yes, my current assumption is the upload job would've failed either way | 19:22 |
| dhellmann | it's possible someone did that by hand, I guess, but that wasn't what the original failure said (it was a dns error of some sort) | 19:22 |
| AJaeger | gerrit/acls/openstack/gnocchi.config is the problem, it allows tag creation | 19:22 |
| clarkb | dhellmann: if dns was working then because someone did it by hand | 19:22 |
| dhellmann | AJaeger : it's an independent project, so that's ok | 19:22 |
| dhellmann | clarkb : yeah, that scenario makes sense | 19:22 |
| mordred | well - hang on - manual tagging of the git repo does not explain the content differences | 19:22 |
| clarkb | mordred: it does if they also pushed to pypi | 19:23 |
| AJaeger | dhellmann: now I remember... | 19:23 |
| mordred | so they would have needed to have uploaded to pypi as well for our hypothesis to be correct | 19:23 |
| mordred | clarkb: yes | 19:23 |
| dhellmann | mordred: it does if they also rebuilt the sdist locally to upload it | 19:23 |
| dhellmann | yeah | 19:23 |
| *** baoli has quit IRC | 19:23 | |
| dhellmann | I don't know if there's any way to tell | 19:23 |
| mordred | I just looked and don't see any indication of who uploaded | 19:23 |
| *** baoli has joined #openstack-infra | 19:23 | |
| dhellmann | look, before we all spend a bunch of time on this, the release team's policy is that if you're doing things on your own, those things include debugging failures | 19:23 |
| mordred | indeed - I mostly just want to make sure one of our systems aren't broken | 19:23 |
| dhellmann | so as long as we are not seeing differences in the releases for things that started with patches in openstack/releases to create the tag, we don't need to worry | 19:23 |
| dhellmann | so far we have not seen any cases where the signatures don't match in that case, have we? | 19:24 |
| mordred | dhellmann: I agree with that | 19:24 |
| mordred | I would, however, like to put in some sort of somethign to prevent us from pushing a tarball to tarballs.o.o overwriting one that's already there - since I don't think we did that this time, but I do believe it's technically possible | 19:25 |
| mordred | and _never_ a thing we want to do | 19:25 |
| dhellmann | yes, I would agree with that | 19:25 |
| *** Sukhdev_ has joined #openstack-infra | 19:25 | |
| *** Sukhdev_ has quit IRC | 19:26 | |
| *** beslemon has quit IRC | 19:26 | |
| *** beslemon has joined #openstack-infra | 19:27 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config master: Move pip management to openstack_project::server https://review.openstack.org/456367 | 19:27 |
| pabelanger | 2001:4800:1ae1:18:f816:3eff:feb4:5237 - - [12/Apr/2017:15:50:15 +0000] "GET /v2/tripleoupstream/centos-binary-redis/manifests/latest HTTP/1.1" 403 436 "-" "docker/1.12.6 go/go1.7.4 kernel/3.10.0-514.10.2.el7.x86_64 os/linux arch/amd64 UpstreamClient(docker-py/1.10.6)" | 19:28 |
| pabelanger | clarkb: mordred: ^ looks like docker proxy is getting used | 19:28 |
| pabelanger | now to figure out why a 403 | 19:29 |
| *** beslemon has quit IRC | 19:31 | |
| clarkb | image claims its public on dockerhub so forbidden is funny response | 19:34 |
| *** edmondsw_ has joined #openstack-infra | 19:34 | |
| pabelanger | ya, will debug with tripleo in a bit | 19:34 |
| *** andreas-f has joined #openstack-infra | 19:36 | |
| *** esikachev has joined #openstack-infra | 19:36 | |
| *** beslemon has joined #openstack-infra | 19:37 | |
| *** andreas-f has quit IRC | 19:38 | |
| *** edmondsw_ has quit IRC | 19:40 | |
| *** jkilpatr_ has quit IRC | 19:41 | |
| *** esikachev has quit IRC | 19:41 | |
| *** esikachev has joined #openstack-infra | 19:42 | |
| *** kfarr has joined #openstack-infra | 19:43 | |
| *** rderose has quit IRC | 19:45 | |
| *** rderose has joined #openstack-infra | 19:46 | |
| *** Hal has joined #openstack-infra | 19:46 | |
| *** vhosakot has quit IRC | 19:48 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Add mirror support for fedora-25 DIB https://review.openstack.org/456372 | 19:49 |
| *** vhosakot has joined #openstack-infra | 19:49 | |
| *** smarcet has joined #openstack-infra | 19:52 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/nodepool master: Switch to /etc/ci/mirror_info.sh for nodepool mirrors https://review.openstack.org/456374 | 19:53 |
| *** jkilpatr_ has joined #openstack-infra | 19:54 | |
| clarkb | pabelanger: I like how that made a bunch of code all over the place so much simpler | 19:54 |
| pabelanger | ++ | 19:55 |
| *** bh526r has quit IRC | 19:59 | |
| pabelanger | project-config still looks blocked on our zuul release | 20:02 |
| clarkb | oh right we need to make that realse now that we are happy with release process | 20:02 |
| clarkb | pabelanger: can you review https://review.openstack.org/#/c/456282/ first, I think we want to tag that | 20:05 |
| clarkb | as it makes the current voluptuous situation a little more correct | 20:05 |
| pabelanger | +3 | 20:06 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Remove curl from single_use_slave.pp https://review.openstack.org/456377 | 20:06 |
| clarkb | ok gonna go grab lunch and then if no one beats me to it can tag 456282 as new zuul release | 20:07 |
| *** dprince has quit IRC | 20:11 | |
| *** cody-somerville has joined #openstack-infra | 20:12 | |
| openstackgerrit | Merged openstack-infra/zuul master: Update voluptuous requirement https://review.openstack.org/456282 | 20:13 |
| *** EricGonczer_ has quit IRC | 20:13 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Create /usr/zuul-env in nodepool-base https://review.openstack.org/456381 | 20:14 |
| *** e0ne has joined #openstack-infra | 20:21 | |
| jbernard | is the live migration job failing today, or am i getting unlucky? | 20:22 |
| *** jcoufal_ has joined #openstack-infra | 20:24 | |
| *** dharinic has quit IRC | 20:26 | |
| *** jcoufal has quit IRC | 20:26 | |
| clarkb | jbernard: possibly related to switch to use uca? | 20:29 |
| clarkb | but I havent heard if its consistent | 20:29 |
| clarkb | how is it failing? have an example | 20:30 |
| clarkb | ? | 20:30 |
| *** dougwig has joined #openstack-infra | 20:32 | |
| *** e0ne has quit IRC | 20:33 | |
| jbernard | http://logs.openstack.org/92/456292/1/check/gate-tempest-dsvm-multinode-live-migration-ubuntu-xenial/355bf4e/console.html | 20:33 |
| jbernard | clarkb: ^ that's one example | 20:34 |
| kfarr | clarkb jbernard I can confirm that I also saw that failure on this patch, even after a recheck: https://review.openstack.org/#/c/456204/ | 20:36 |
| *** vhosakot has quit IRC | 20:36 | |
| pabelanger | it is a new image from today | 20:36 |
| pabelanger | wonder if some how related | 20:36 |
| *** dimtruck is now known as zz_dimtruck | 20:36 | |
| jbernard | i saw it twice on https://review.openstack.org/#/c/456292/ | 20:38 |
| *** dkehn has quit IRC | 20:40 | |
| AJaeger | pabelanger, clarkb I'm off now for a long easter weekend, probably back online next week Wednesday. | 20:41 |
| AJaeger | Have fun and don't break too much in your spring clean ;) | 20:41 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Create /usr/zuul-env in nodepool-base https://review.openstack.org/456381 | 20:43 |
| pabelanger | AJaeger: enjoy | 20:44 |
| clarkb | AJaeger: you too | 20:46 |
| pabelanger | k, think we are ready for zuul tag | 20:46 |
| clarkb | pabelanger: jbernard kfarr earlier today ( about 5 hours ago ) a change was merged to devstack to use cloud archive | 20:47 |
| clarkb | which would have bumped the libvirt version | 20:47 |
| pabelanger | Ya, seems like a good place to start | 20:47 |
| *** marst_ has quit IRC | 20:48 | |
| *** marst_ has joined #openstack-infra | 20:49 | |
| kfarr | Not sure if this is related at all, but I noticed a separate problem where "gate-project-config-layout" seems to be failing for project-config since about 6 hours ago | 20:49 |
| clarkb | http://logs.openstack.org/92/456292/1/check/gate-tempest-dsvm-multinode-live-migration-ubuntu-xenial/355bf4e/logs/screen-n-cpu.txt.gz?level=TRACE#_2017-04-12_19_09_02_607 | 20:49 |
| *** dkehn has joined #openstack-infra | 20:49 | |
| *** esikachev has quit IRC | 20:49 | |
| clarkb | kfarr: not related at all and I need to make tag to fix the project0-config issue | 20:49 |
| kfarr | clarkb, got it, thanks | 20:50 |
| clarkb | looking at that error netcat isn't installed? | 20:50 |
| clarkb | sh: 1: nc: not found: Input/output error | 20:50 |
| pabelanger | clarkb: mordred 2.5.2 on 9ada50e87a72d77d94cab0354ebb85b1de68c903 for zuul? We are okay with that | 20:52 |
| clarkb | pabelanger: yup | 20:52 |
| *** smarcet has quit IRC | 20:53 | |
| pabelanger | okay, pushed | 20:53 |
| clarkb | ok I just reproduced that error message on xenial by running a script with an uninstalled program | 20:55 |
| *** srobert_ has joined #openstack-infra | 20:55 | |
| *** srobert_ has quit IRC | 20:56 | |
| *** srobert_ has joined #openstack-infra | 20:56 | |
| *** trown is now known as trown|outtypewww | 20:56 | |
| *** srobert has quit IRC | 20:58 | |
| clarkb | kfarr: jbernard can you depends on https://review.openstack.org/456388 and see if that fixes? | 20:58 |
| openstackgerrit | Chi Lo proposed openstack-infra/project-config master: Enable plugins to support neutron-lbaas testing. https://review.openstack.org/456390 | 20:59 |
| kfarr | clarkb will try, thanks | 21:00 |
| clarkb | http://logs.openstack.org/92/456292/1/check/gate-tempest-dsvm-multinode-live-migration-ubuntu-xenial/355bf4e/logs/dpkg-l.txt.gz netcat is indeed missing from there | 21:00 |
| *** srobert_ has quit IRC | 21:01 | |
| *** thorst has quit IRC | 21:01 | |
| *** ociuhandu has quit IRC | 21:01 | |
| *** thorst has joined #openstack-infra | 21:02 | |
| *** jascott1_ has joined #openstack-infra | 21:02 | |
| *** kgiusti has left #openstack-infra | 21:02 | |
| *** jascott1_ has quit IRC | 21:03 | |
| *** dharinic has joined #openstack-infra | 21:04 | |
| *** jascott1_ has joined #openstack-infra | 21:04 | |
| *** ianychoi has quit IRC | 21:05 | |
| clarkb | looks like yakkety libvirt-daemon does recommend netcat-openbsd guessing this is just a UCA problem? /me looks there | 21:05 |
| *** eharney has quit IRC | 21:05 | |
| clarkb | Recommends: qemu-kvm | qemu (>= 0.9.1), libxml2-utils, netcat-openbsd | 21:06 |
| *** thorst has quit IRC | 21:06 | |
| clarkb | huh maybe we don't install recommends by default from UCA? | 21:06 |
| clarkb | mordred: pabelanger do you know what mechanism makes recommends get installed by default typically? | 21:06 |
| *** sdague has quit IRC | 21:07 | |
| pabelanger | APT::Install-Recommends | 21:07 |
| *** zz_dimtruck is now known as dimtruck | 21:08 | |
| pabelanger | I know we disable it in DIB, but pretty sure we re-enable it | 21:08 |
| pabelanger | maybe not | 21:09 |
| pabelanger | I think we set /etc/apt/apt.conf.d/95disable-recommends | 21:09 |
| *** Apoorva_ has joined #openstack-infra | 21:09 | |
| clarkb | http://logs.openstack.org/92/456292/1/check/gate-tempest-dsvm-multinode-live-migration-ubuntu-xenial/355bf4e/logs/devstacklog.txt.gz#_2017-04-12_18_53_05_297 thats where the install happens and sure enough it lists the recommends but doesn't seem to install it | 21:10 |
| pabelanger | ya, that would make sense if ^ is installed from above | 21:11 |
| clarkb | yup its there on a random xenial host I logged into | 21:12 |
| clarkb | ok that explains it | 21:12 |
| *** Apoorva has quit IRC | 21:13 | |
| *** bkero has quit IRC | 21:13 | |
| *** Sukhdev has joined #openstack-infra | 21:14 | |
| clarkb | has anyone rechecked layout jobs yet? | 21:15 |
| * clarkb can do that if not | 21:15 | |
| pabelanger | ya, did a few | 21:15 |
| pabelanger | making sure they pass | 21:15 |
| *** ldnunes has quit IRC | 21:15 | |
| kfarr | I just rechecked mine | 21:16 |
| clarkb | kfarr: can you point me at your depends on when its pushed too so that I can follow along if the devstack change fixes live migration errors | 21:17 |
| *** armax has quit IRC | 21:17 | |
| kfarr | clarkb: thanks, it's this one: https://review.openstack.org/#/c/456204/ | 21:17 |
| pabelanger | ya, they look to be good now | 21:17 |
| *** eharney has joined #openstack-infra | 21:18 | |
| pabelanger | clarkb: https://review.openstack.org/#/c/456346/ should be straightforward | 21:18 |
| pabelanger | sudoers files for DIB | 21:18 |
| *** mat128 has quit IRC | 21:18 | |
| clarkb | and gerrit just 500'd on me, is it time for weekly service restart? | 21:19 |
| clarkb | there is some GCing according to melody its not super bad, but likely we need to restart | 21:20 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/project-config master: Install tox and git-review with DIB https://review.openstack.org/456395 | 21:21 |
| *** gouthamr has quit IRC | 21:21 | |
| clarkb | pabelanger: done | 21:21 |
| clarkb | there are no jobs in the release pipeline. Going to restart gerrit now. | 21:22 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/system-config master: Stop installing tox and git-review from puppet https://review.openstack.org/456396 | 21:23 |
| kfarr | clarkb: my layout job recheck passed \o/ thanks! | 21:23 |
| clarkb | just double checking meolody trends | 21:24 |
| clarkb | #status notice Restarting Gerrit for our weekly memory leak cleanup. | 21:24 |
| openstackstatus | clarkb: sending notice | 21:24 |
| cmurphy | pabelanger: this recheck didn't pass https://review.openstack.org/#/c/456334/ not sure if something changed since then? | 21:24 |
| pabelanger | cmurphy: ya, I just did it again. I think I was a little too fast first time | 21:25 |
| cmurphy | pabelanger: okay | 21:25 |
| -openstackstatus- NOTICE: Restarting Gerrit for our weekly memory leak cleanup. | 21:25 | |
| pabelanger | don't think our pypi mirror had 2.5.2 yet | 21:25 |
| *** bkero has joined #openstack-infra | 21:25 | |
| *** bkero has quit IRC | 21:25 | |
| *** bkero has joined #openstack-infra | 21:25 | |
| *** john-davidge has joined #openstack-infra | 21:26 | |
| clarkb | ok gerrit is restarted | 21:26 |
| *** gildub has joined #openstack-infra | 21:27 | |
| openstackstatus | clarkb: finished sending notice | 21:27 |
| *** neiljerram has joined #openstack-infra | 21:28 | |
| *** esberglu has quit IRC | 21:28 | |
| *** esberglu has joined #openstack-infra | 21:29 | |
| *** jcoufal_ has quit IRC | 21:30 | |
| kfarr | I gotta run, will check back in later today or tomorrow | 21:30 |
| *** kfarr has quit IRC | 21:30 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add jenkins user sudoers file to nodepool-base https://review.openstack.org/456346 | 21:30 |
| openstackgerrit | Merged openstack-infra/project-config master: Revert "Revert "Stop installing pip from install_puppet.sh"" https://review.openstack.org/455843 | 21:31 |
| clarkb | bah I used the wrong nc version /me updates | 21:31 |
| openstackgerrit | Merged openstack-infra/project-config master: Stop enabling EPEL mirror by default https://review.openstack.org/453222 | 21:31 |
| *** john-davidge has quit IRC | 21:31 | |
| *** krtaylor has quit IRC | 21:32 | |
| *** askb has joined #openstack-infra | 21:32 | |
| *** beslemon has quit IRC | 21:33 | |
| *** esberglu has quit IRC | 21:33 | |
| ianw | pabelanger: what's the md5 issue? | 21:33 |
| clarkb | ianw: we think we sorted it out | 21:34 |
| clarkb | assuming its the same md5 issue I am thinking about. Basically gnocchi made their own push to pypi with tarballs that don't match our tarballs on tarballs.o.o | 21:35 |
| clarkb | initially this created concern | 21:35 |
| openstackgerrit | Kendall Nelson proposed openstack-infra/storyboard master: Handle Branches from Launchpad in Migration https://review.openstack.org/437469 | 21:37 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Allow using webapp from connections https://review.openstack.org/439831 | 21:38 |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Support GitHub PR webhooks https://review.openstack.org/439834 | 21:38 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove unused params in puppet builds https://review.openstack.org/456334 | 21:38 |
| *** marst_ has quit IRC | 21:38 | |
| ianw | clarkb: ok ... so the release would be to get a new push to pypi? | 21:39 |
| ianw | were it to be needed? | 21:39 |
| clarkb | ianw: oh sorry, its basically a non issue since gnocchi is doing their own thing so if their artifacts don't match ours we don't care too much. We were just concerned it was happening on the artifacts we did care for but those all seemed to match | 21:40 |
| clarkb | ianw: separately if you are not out today https://review.openstack.org/456388 could use a review as it should fix the nova live migration job 456204 depends on it and is checking for us | 21:41 |
| *** Qiming has quit IRC | 21:41 | |
| *** ociuhandu has joined #openstack-infra | 21:41 | |
| *** esberglu has joined #openstack-infra | 21:42 | |
| *** priteau has quit IRC | 21:42 | |
| *** Goneri has quit IRC | 21:42 | |
| *** neiljerram has quit IRC | 21:43 | |
| *** marst_ has joined #openstack-infra | 21:44 | |
| *** Qiming has joined #openstack-infra | 21:46 | |
| *** esberglu has quit IRC | 21:46 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/project-config master: Install tox and git-review with DIB https://review.openstack.org/456395 | 21:47 |
| clarkb | anyone know if there is a project config fix for the ceph devstack plugin git cloen fail? | 21:47 |
| pabelanger | not following | 21:48 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Remove /opt/zuul-env from slave_common.pp https://review.openstack.org/456401 | 21:48 |
| *** annegentle has quit IRC | 21:48 | |
| clarkb | sorry the ceph job is failing because it tries to clone the devstack ceph plugin as its not in PROJECTS | 21:48 |
| clarkb | looks like https://review.openstack.org/#/c/455818/1/jenkins/jobs/devstack-gate.yaml is causing it | 21:48 |
| clarkb | jbernard: fyi ^ going to push a fix now | 21:48 |
| *** baoli has quit IRC | 21:50 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config master: List devstack ceph plugin in PROJECTS when used https://review.openstack.org/456402 | 21:51 |
| clarkb | pabelanger: jbernard ^ | 21:51 |
| clarkb | http://logs.openstack.org/04/456204/3/check/gate-tempest-dsvm-full-devstack-plugin-ceph-ubuntu-xenial/65531b7/logs/devstacklog.txt.gz is example fail | 21:51 |
| *** pcaruana has quit IRC | 21:51 | |
| *** rhallisey has quit IRC | 22:01 | |
| pabelanger | ianw: mind a +3 https://review.openstack.org/#/c/455003/ | 22:03 |
| clarkb | I have enjoyed the historical spelunking in this cleanup | 22:04 |
| clarkb | "why did we do that? OH YA" | 22:04 |
| cmurphy | ++ | 22:04 |
| clarkb | the centos6 git fix repo that ianw made was another one | 22:05 |
| cmurphy | thie history of openstack-infra as told by git log | 22:05 |
| *** Zhongjun__ has joined #openstack-infra | 22:06 | |
| pabelanger | ya, that git fix finally made it into centos6 too | 22:06 |
| clarkb | ha nice | 22:06 |
| cmurphy | lol | 22:06 |
| ianw | i still get emails occasionally on that git bug | 22:06 |
| *** MasterOfBugs has quit IRC | 22:06 | |
| *** pramodrj07 has quit IRC | 22:06 | |
| clarkb | ianw: so it is more than just us that hit it in the wild? | 22:06 |
| *** pramodrj07 has joined #openstack-infra | 22:06 | |
| *** MasterOfBugs has joined #openstack-infra | 22:06 | |
| *** Apoorva_ has quit IRC | 22:07 | |
| pabelanger | I imagine a paying customer did | 22:07 |
| pabelanger | but cannot see that info | 22:07 |
| *** Apoorva has joined #openstack-infra | 22:07 | |
| *** mdrabe has quit IRC | 22:07 | |
| ianw | yeah. i just thank the stars i'm not in an environment where i have to think about rhel6 :) | 22:09 |
| clarkb | as a heads up I will be out tomorrow morning at kids allergy testing, but should be back for second half of the day | 22:09 |
| clarkb | ianw: pabelanger: it really is impressive how long the support is able to eb stretched out on some of these distro releases | 22:10 |
| openstackgerrit | Merged openstack-infra/system-config master: Remove /opt/requirements from slave_common.pp https://review.openstack.org/455003 | 22:10 |
| ianw | yeah, i guess the other person at random company is like "thank heavens i don't have to deal with all that modern stuff" | 22:11 |
| ianw | if it ain't broke, don't fix it | 22:11 |
| *** inc0 has quit IRC | 22:12 | |
| *** inc0 has joined #openstack-infra | 22:12 | |
| openstackgerrit | Merged openstack-infra/project-config master: List devstack ceph plugin in PROJECTS when used https://review.openstack.org/456402 | 22:13 |
| *** trown|outtypewww has quit IRC | 22:14 | |
| *** trown has joined #openstack-infra | 22:15 | |
| *** jkilpatr_ has quit IRC | 22:15 | |
| *** ihrachys|afk is now known as ihrachys | 22:18 | |
| *** eharney has quit IRC | 22:18 | |
| *** gordc has quit IRC | 22:18 | |
| *** felipemonteiro has quit IRC | 22:19 | |
| *** esberglu has joined #openstack-infra | 22:21 | |
| *** csomerville has joined #openstack-infra | 22:24 | |
| *** ianw is now known as ianw_pto | 22:26 | |
| *** cody-somerville has quit IRC | 22:27 | |
| *** esikachev has joined #openstack-infra | 22:33 | |
| *** esikachev has quit IRC | 22:38 | |
| *** ssbarnea has quit IRC | 22:40 | |
| *** esikachev has joined #openstack-infra | 22:42 | |
| *** marst_ has quit IRC | 22:43 | |
| *** pradk has quit IRC | 22:45 | |
| *** baoli has joined #openstack-infra | 22:45 | |
| *** esikachev has quit IRC | 22:47 | |
| *** jkilpatr_ has joined #openstack-infra | 22:49 | |
| *** baoli has quit IRC | 22:50 | |
| *** Goneri has joined #openstack-infra | 22:52 | |
| *** esikachev has joined #openstack-infra | 22:56 | |
| jhesketh | Morning | 22:57 |
| clarkb | good morning | 22:58 |
| *** claudiub has quit IRC | 22:58 | |
| *** esikachev has quit IRC | 23:01 | |
| pabelanger | fedora-25 dib just uploading now. This is will be the first image where we don't setup pip from install_puppet.sh | 23:02 |
| pabelanger | looking at build logs, our virtualenvs are correct now | 23:02 |
| pabelanger | http://nb02.openstack.org/dib.fedora-25.log | 23:02 |
| *** adriant has joined #openstack-infra | 23:02 | |
| *** xyang1 has quit IRC | 23:02 | |
| clarkb | cool, I'm unbreaking a nova job that I broke with UCA change | 23:02 |
| *** Apoorva_ has joined #openstack-infra | 23:04 | |
| *** rderose has quit IRC | 23:04 | |
| *** Sukhdev has quit IRC | 23:04 | |
| *** Apoorva has quit IRC | 23:07 | |
| *** xarses has quit IRC | 23:07 | |
| *** vhosakot has joined #openstack-infra | 23:07 | |
| *** jkilpatr_ is now known as jkilpatr | 23:10 | |
| pabelanger | clarkb: mordred: do we need pydistutils.cfg any more on workers? | 23:16 |
| clarkb | pabelanger: yes | 23:16 |
| clarkb | pabelanger: that controls anything that happens to invoke easy_install | 23:16 |
| clarkb | which can and still does happen due to corner cases of things | 23:16 |
| pabelanger | clarkb: right, but I think we set it up with configure_mirror.sh | 23:17 |
| pabelanger | I guess I should of asked, do we need it at image build time | 23:17 |
| pabelanger | because we current pin to rax | 23:17 |
| clarkb | oh I see what you mean. We need it there bceause not all instances are single use instances that get configure_mirror.sh | 23:17 |
| clarkb | you likely can split it off into the static host puppet though | 23:17 |
| pabelanger | kewl | 23:18 |
| *** thorst has joined #openstack-infra | 23:20 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Split off .pydistutils.cfg to static slaves https://review.openstack.org/456410 | 23:20 |
| *** baoli has joined #openstack-infra | 23:20 | |
| *** knangia has quit IRC | 23:21 | |
| *** baoli has quit IRC | 23:21 | |
| *** thorst has quit IRC | 23:21 | |
| *** baoli has joined #openstack-infra | 23:22 | |
| *** armax has joined #openstack-infra | 23:23 | |
| *** esikachev has joined #openstack-infra | 23:25 | |
| *** armax_ has joined #openstack-infra | 23:26 | |
| *** tosky has quit IRC | 23:26 | |
| *** armax has quit IRC | 23:28 | |
| *** armax_ is now known as armax | 23:28 | |
| *** esikachev has quit IRC | 23:29 | |
| *** john-davidge has joined #openstack-infra | 23:29 | |
| *** kzaitsev_mb has quit IRC | 23:34 | |
| *** john-davidge has quit IRC | 23:34 | |
| *** erlon has quit IRC | 23:35 | |
| *** eharney has joined #openstack-infra | 23:35 | |
| *** Apoorva_ has quit IRC | 23:36 | |
| *** Apoorva has joined #openstack-infra | 23:37 | |
| *** esberglu has quit IRC | 23:38 | |
| *** esberglu has joined #openstack-infra | 23:39 | |
| *** gongysh has joined #openstack-infra | 23:40 | |
| *** ianychoi has joined #openstack-infra | 23:41 | |
| *** zeroDivisible has quit IRC | 23:41 | |
| *** priteau has joined #openstack-infra | 23:42 | |
| *** esberglu has quit IRC | 23:43 | |
| *** zeroDivisible has joined #openstack-infra | 23:45 | |
| *** priteau has quit IRC | 23:47 | |
| *** s-shiono has joined #openstack-infra | 23:48 | |
| *** s-shiono_ has joined #openstack-infra | 23:49 | |
| jbernard | clarkb: that's my fault! | 23:51 |
| *** zeroDivisible has quit IRC | 23:52 | |
| jbernard | clarkb: i accidentely deleted that line in my last patch | 23:52 |
| *** EricGonczer_ has joined #openstack-infra | 23:52 | |
| *** s-shiono has quit IRC | 23:52 | |
| *** dimtruck is now known as zz_dimtruck | 23:52 | |
| clarkb | jbernard: no problem, got it sorted out for you I hope | 23:53 |
| jbernard | clarkb: yep, that's perfect, thank you | 23:53 |
| *** wolverineav has quit IRC | 23:53 | |
| clarkb | jbernard: I noticed it when I was debugging the live migration test issue. Wanted to make sure I didn't break ceph job too :) | 23:53 |
| *** wolverineav has joined #openstack-infra | 23:54 | |
| *** esikachev has joined #openstack-infra | 23:54 | |
| *** zeroDivisible has joined #openstack-infra | 23:54 | |
| clarkb | jbernard: also I got livemigration fixes pushed to nova and devsatck so hopefully that works again soon | 23:55 |
| *** wolverin_ has joined #openstack-infra | 23:56 | |
| jbernard | clarkb: awesome, should i depends on https://review.openstack.org/456388 for now? | 23:56 |
| clarkb | jbernard: you need https://review.openstack.org/456408 which depends on ^ | 23:57 |
| clarkb | had to patch nova too | 23:57 |
| *** esikachev has quit IRC | 23:58 | |
| *** wolverineav has quit IRC | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!