| *** slaweq has joined #openstack-infra | 00:00 | |
| *** tosky has quit IRC | 00:05 | |
| *** slaweq has quit IRC | 00:05 | |
| *** lseki has quit IRC | 00:08 | |
| *** wolverineav has quit IRC | 00:10 | |
| *** ianychoi has quit IRC | 00:17 | |
| *** ianychoi has joined #openstack-infra | 00:18 | |
| *** jamesmcarthur has joined #openstack-infra | 00:18 | |
| *** weshay has quit IRC | 00:19 | |
| *** mhu has joined #openstack-infra | 00:19 | |
| *** weshay has joined #openstack-infra | 00:20 | |
| *** sthussey has quit IRC | 00:20 | |
| *** wolverineav has joined #openstack-infra | 00:33 | |
| *** wolverineav has quit IRC | 00:38 | |
| *** wolverineav has joined #openstack-infra | 00:39 | |
| *** markvoelker has joined #openstack-infra | 00:40 | |
| *** hongbin has joined #openstack-infra | 00:40 | |
| *** hamzy has joined #openstack-infra | 00:44 | |
| *** stakeda has joined #openstack-infra | 00:46 | |
| *** wolverineav has quit IRC | 00:46 | |
| *** jistr has quit IRC | 01:00 | |
| *** jistr has joined #openstack-infra | 01:00 | |
| dmsimard | seeing node failures on opensuse things | 01:02 |
|---|---|---|
| fungi | yeah, if it's for opensuse-423 it's currently trying to build, this one's made it 25 minutes so far without failing | 01:04 |
| fungi | but not sure how much longer it has to go | 01:05 |
| ianw | fungi: looks like it usually runs ~ 55 minutes http://grafana.openstack.org/d/NcFtwKriz/nodepool-dib-status?orgId=1 | 01:05 |
| *** yolanda has quit IRC | 01:07 | |
| dmsimard | ianw: those are some neat graphs | 01:07 |
| ianw | amazing what you can find in graphite when you check what firewall ports are open :) | 01:08 |
| dmsimard | fungi: I saw the last set was about 3 hours old, I'll try 423 when it lands :D | 01:09 |
| dmsimard | how do we typically troubleshoot node failures btw ? | 01:09 |
| dmsimard | I'm not finding anything super helpful in the zuul debug logs -- I thought about checking the nodepool launchers but there's multiple of them | 01:10 |
| *** slaweq has joined #openstack-infra | 01:11 | |
| ianw | yeah really got to find the logs on the launcher | 01:14 |
| ianw | and if they don't reveal anything ... prepare for a long session of debugging and cross your fingers systemd isn't involved :) | 01:14 |
| fungi | in this case it'll be because we have no images for 423. i deleted them since they were all booting broken nodes anyway | 01:14 |
| *** yolanda has joined #openstack-infra | 01:15 | |
| fungi | and the current rebuild attempt is going on 37 minutes now | 01:15 |
| fungi | so... fingers crossed | 01:15 |
| dmsimard | ianw: this is what the docs currently have on node_failure: https://zuul-ci.org/docs/zuul/user/jobs.html?highlight=node_failure#build-status | 01:15 |
| *** slaweq has quit IRC | 01:16 | |
| dmsimard | fungi: worst case I'll submit another patch to set the job I forgot as non-voting. Was hoping not to have to do that haha :p | 01:16 |
| fungi | yeah, we're getting close to having them working again, i hope | 01:17 |
| *** whoami-rajat has joined #openstack-infra | 01:19 | |
| *** ricolin has joined #openstack-infra | 01:21 | |
| openstackgerrit | Kendall Nelson proposed openstack-infra/project-config master: Trove: use storyboard https://review.openstack.org/648320 | 01:23 |
| *** wolverineav has joined #openstack-infra | 01:48 | |
| *** wolverineav has quit IRC | 01:51 | |
| *** jamesmcarthur has quit IRC | 01:51 | |
| *** jamesmcarthur has joined #openstack-infra | 01:51 | |
| *** wolverineav has joined #openstack-infra | 01:53 | |
| *** rlandy has quit IRC | 01:55 | |
| *** jamesmcarthur has quit IRC | 01:56 | |
| *** jamesmcarthur has joined #openstack-infra | 02:02 | |
| *** wolverineav has quit IRC | 02:03 | |
| *** jamesmcarthur has quit IRC | 02:06 | |
| fungi | | opensuse-423-0000036173 | opensuse-423 | nb02 | qcow2,raw,vhd | ready | 00:00:38:37 | | 02:07 |
| fungi | dmsimard: ianw: ^ | 02:07 |
| *** slaweq has joined #openstack-infra | 02:11 | |
| *** slaweq has quit IRC | 02:16 | |
| *** bhavikdbavishi has joined #openstack-infra | 02:18 | |
| *** jamesmcarthur has joined #openstack-infra | 02:22 | |
| pabelanger | ianw: any thoughts? http://paste.openstack.org/show/748559/ | 02:26 |
| pabelanger | ianw: I'm not able to build fedora-29 minimal using latest dib | 02:26 |
| *** diablo_rojo has quit IRC | 02:28 | |
| pabelanger | ianw: I guess we should first check if enable, and if not enable it | 02:28 |
| pabelanger | I'll have to dig more into it in the morning | 02:29 |
| *** ramishra has joined #openstack-infra | 02:43 | |
| *** bhavikdbavishi has quit IRC | 02:45 | |
| ianw | pabelanger: hrm, theres a change on that | 02:49 |
| ianw | https://review.openstack.org/#/c/634105/ -- change between dbus-daemon to dbus-broker. i wonder if something else has changed? | 02:49 |
| mnaser | I'm almost sure ovh-gra1 has some I/o issues.. | 02:53 |
| mnaser | I'm getting a lot of timeouts | 02:53 |
| *** jamesmcarthur has quit IRC | 02:54 | |
| *** psachin has joined #openstack-infra | 02:56 | |
| clarkb | amorin is our contact at ovh | 02:57 |
| clarkb | I know they were looking at io slowness in the context of etcd at one point | 02:57 |
| clarkb | michal dulko wasworking with them according to email and found ~1.5% of fsyncs took longer than a second | 03:00 |
| *** jamesmcarthur has joined #openstack-infra | 03:02 | |
| *** slaweq has joined #openstack-infra | 03:12 | |
| *** slaweq has quit IRC | 03:16 | |
| *** jamesmcarthur has quit IRC | 03:18 | |
| *** jamesmcarthur has joined #openstack-infra | 03:19 | |
| *** jamesmcarthur has quit IRC | 03:21 | |
| *** ykarel has joined #openstack-infra | 03:22 | |
| *** otherwiseguy has quit IRC | 03:24 | |
| *** weshay has quit IRC | 03:24 | |
| *** weshay has joined #openstack-infra | 03:24 | |
| *** pabelanger has quit IRC | 03:24 | |
| *** weshay has quit IRC | 03:24 | |
| *** diablo_rojo has joined #openstack-infra | 03:25 | |
| *** weshay has joined #openstack-infra | 03:25 | |
| *** bhavikdbavishi has joined #openstack-infra | 03:30 | |
| mnaser | it's really hurting our jobs I think :\ | 03:32 |
| *** otherwiseguy has joined #openstack-infra | 03:32 | |
| mnaser | cloudnull: shall we drop stable/ocata at this point? at the time we discussed leaving it for now because of some rax requirements (odyssey4me brought it up).. is it still relevant or? | 03:43 |
| mnaser | https://review.openstack.org/#/q/projects:openstack/openstack-ansible+is:open+branch:stable/ocata looks like we have some trivial patches we can just abandon | 03:43 |
| *** hongbin has quit IRC | 03:52 | |
| *** raukadah is now known as chandankumar | 03:53 | |
| *** udesale has joined #openstack-infra | 03:56 | |
| *** wolverineav has joined #openstack-infra | 04:03 | |
| *** wolverineav has quit IRC | 04:08 | |
| *** amansi26 has joined #openstack-infra | 04:08 | |
| mnaser | odyssey4me: any reason behind why we can't run horizon on metal here? https://review.openstack.org/#/c/648595/ | 04:08 |
| *** eernst has joined #openstack-infra | 04:09 | |
| *** eernst has quit IRC | 04:11 | |
| mnaser | jeez, I'm starting to use the wrong channels, oops. | 04:21 |
| mnaser | anyhow, infra-root: just saw this Download error on http://mirror.dfw.rax.openstack.org/pypi/simple/pbr/: timed out -- Some packages may not be found! | 04:21 |
| mnaser | also, infra-root: http://mirror.sjc1.vexxhost.openstack.org/wheel/ubuntu-18.04-x86_64/ is there any reason 'pip' is missing there? it seems to be in all the other distro/os | 04:24 |
| ianw | mnaser: that's a proxy url ... so issues at the other end? | 04:25 |
| mnaser | ianw: the first or the latter? | 04:25 |
| ianw | mnaser: the wheels are built from requirements.txt ... so it's not there? | 04:26 |
| mnaser | ok, I guess at some point we removed pip from there perhaps | 04:26 |
| mnaser | I'll go try figure out what's going wrong | 04:26 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zone-opendev.org master: Add DS record for acme.opendev.org https://review.openstack.org/648607 | 04:31 |
| *** ykarel has quit IRC | 04:47 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zone-opendev.org master: Add DS record for acme.opendev.org https://review.openstack.org/648607 | 04:50 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zone-opendev.org master: Fix trailing dots for acme-challenge records https://review.openstack.org/648608 | 04:50 |
| ianw | i wonder how many hours of humanity has been lost to trailing .'s in bind config | 04:50 |
| *** ykarel has joined #openstack-infra | 05:02 | |
| *** slaweq has joined #openstack-infra | 05:11 | |
| *** slaweq has quit IRC | 05:15 | |
| *** diablo_rojo has quit IRC | 05:20 | |
| *** jamesmcarthur has joined #openstack-infra | 05:22 | |
| *** jamesmcarthur has quit IRC | 05:27 | |
| *** kjackal has joined #openstack-infra | 05:37 | |
| *** kjackal has quit IRC | 05:53 | |
| *** yboaron_ has quit IRC | 06:02 | |
| *** e0ne has joined #openstack-infra | 06:06 | |
| *** janki has joined #openstack-infra | 06:10 | |
| *** e0ne has quit IRC | 06:10 | |
| *** slaweq has joined #openstack-infra | 06:37 | |
| *** quiquell|off is now known as quiquell | 06:37 | |
| *** rcernin has quit IRC | 06:38 | |
| *** udesale has quit IRC | 06:41 | |
| *** udesale has joined #openstack-infra | 06:41 | |
| *** slaweq has quit IRC | 06:42 | |
| *** kopecmartin|off is now known as kopecmartin | 07:07 | |
| *** slaweq has joined #openstack-infra | 07:08 | |
| *** dpawlik has joined #openstack-infra | 07:17 | |
| quiquell | Hello we are facing issues accessing docker registry mirrors at zuul | 07:22 |
| quiquell | http://logs.openstack.org/19/647419/6/check/tripleo-ci-centos-7-standalone-upgrade/c5dd168/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2019-03-28_13_44_31 | 07:23 |
| quiquell | Inspecting with skopeo works | 07:23 |
| quiquell | kopeo inspect docker://docker.io/tripleostein/centos-binary-cinder-api:fbb797b891c6f2530b18ee904a93c35b9e7490c1_2c625b49 | 07:23 |
| quiquell | skopeo inspect docker://docker.io/tripleostein/centos-binary-cinder-api:fbb797b891c6f2530b18ee904a93c35b9e7490c1_2c625b49 | 07:23 |
| *** yamamoto has joined #openstack-infra | 07:23 | |
| quiquell | ianw: any idea on this ^ ? | 07:23 |
| quiquell | ykarel: ^ | 07:24 |
| *** quiquell is now known as quiquell|brb | 07:25 | |
| *** apetrich has joined #openstack-infra | 07:30 | |
| *** e0ne has joined #openstack-infra | 07:32 | |
| ianw | quiquell: hrm, i'm pretty sure that's coming from the remote end | 07:36 |
| *** slaweq has quit IRC | 07:36 | |
| *** pgaxatte has joined #openstack-infra | 07:36 | |
| *** jaosorior has quit IRC | 07:37 | |
| ianw | yeah | 07:38 |
| ianw | https://registry-1.docker.io/v2/tripleostein/centos-binary-cinder-api/manifests/fbb797b891c6f2530b18ee904a93c35b9e7490c1_2c625b49 | 07:38 |
| ianw | so maybe something up with the registry-1 bit? | 07:39 |
| *** rpittau|afk is now known as rpittau | 07:39 | |
| openstackgerrit | Dirk Mueller proposed openstack-infra/system-config master: Skip mirroring live and jeos images for opensuse https://review.openstack.org/648632 | 07:40 |
| dirk | infra-root: looks like the opensuse mirror still hasn't updated. any idea what the issue of today is? | 07:41 |
| *** ykarel is now known as ykarel|lunch | 07:42 | |
| dirk | infra-root: ah, nevermind. CTRL-R was missing (and maybe more coffee..) | 07:42 |
| icey | AJaeger: looks like sphinx's parsing has gotten a lot stricter, putting ellipses in JSON is no longer acceptable either :-/ | 07:44 |
| AJaeger | icey: what fun ;( | 07:44 |
| icey | AJaeger: looks like some `rm`s and some new `\` may resolve this though - hopefully have a new review up soon | 07:45 |
| AJaeger | icey: thanks. We could also disable highlighting for that one if it's too tricky | 07:45 |
| icey | AJaeger: we'd have to disable highlighting for a lot more than jsut that one | 07:45 |
| icey | I'm onto the third file it doesn't like now | 07:46 |
| icey | ./source/identity/federated-keystone.rst ./source/identity/policies.rst ./source/instance-management/security-services-for-instances.rst so far | 07:46 |
| *** slaweq has joined #openstack-infra | 07:48 | |
| *** slaweq has quit IRC | 07:52 | |
| icey | AJaeger: not too bad, although I'm not sure how I feel about having to make those changes: https://review.openstack.org/648634 - some of them feel hacky | 07:52 |
| quiquell|brb | ianw: humm thanks, maybe we are missing something else at creating the tripleostein namespace | 07:55 |
| quiquell|brb | ykarel|lunch: ^ do you know something we can be missing ? | 07:55 |
| *** quiquell|brb is now known as quiquell | 08:06 | |
| *** tkajinam has quit IRC | 08:07 | |
| *** ccamacho has joined #openstack-infra | 08:07 | |
| *** ginopc has joined #openstack-infra | 08:08 | |
| *** tosky has joined #openstack-infra | 08:15 | |
| *** yboaron_ has joined #openstack-infra | 08:15 | |
| *** yamamoto has quit IRC | 08:20 | |
| *** yamamoto has joined #openstack-infra | 08:22 | |
| *** rcernin has joined #openstack-infra | 08:22 | |
| *** helenaAM has joined #openstack-infra | 08:24 | |
| *** e0ne has quit IRC | 08:31 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Add job and integration tests for the upload-git-mirror role https://review.openstack.org/643435 | 08:31 |
| *** jangutter has quit IRC | 08:32 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Get executor job params https://review.openstack.org/607078 | 08:35 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Separate out executor server from runner https://review.openstack.org/607079 | 08:35 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: runner: implement prep-workspace https://review.openstack.org/607082 | 08:35 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: runner: add configuration schema https://review.openstack.org/640672 | 08:35 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: runner: add execute sub-command https://review.openstack.org/630944 | 08:35 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: add command line interface https://review.openstack.org/644770 | 08:35 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: runner: add job parameters listing https://review.openstack.org/644795 | 08:35 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Add depends-on support to frozen jobs API https://review.openstack.org/639022 | 08:35 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: add support for depends-on https://review.openstack.org/632064 | 08:35 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Move repository preparation into common class https://review.openstack.org/648642 | 08:35 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Separate out executor concerns from AnsibleJob https://review.openstack.org/648643 | 08:35 |
| AJaeger | icey: | 08:36 |
| AJaeger | icey: let me look and see whether I can make it less hacky - thanks | 08:37 |
| icey | AJaeger: thanks! I'm annoyed in some way about all three of the files - although the new `\` in the shell json bit is probably the most irritating to me | 08:37 |
| *** gfidente has joined #openstack-infra | 08:38 | |
| *** jpena|off is now known as jpena | 08:40 | |
| *** yamamoto has quit IRC | 08:49 | |
| *** ykarel|lunch is now known as ykarel | 08:49 | |
| *** yamamoto has joined #openstack-infra | 08:49 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Add a role to mirror a git repository to a remote git server https://review.openstack.org/643394 | 08:51 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Add a job that runs the upload-git-mirror role https://review.openstack.org/648230 | 08:51 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Fix typo in upload-git-mirror README https://review.openstack.org/648423 | 08:51 |
| *** jpich has joined #openstack-infra | 08:52 | |
| *** bhavikdbavishi has quit IRC | 08:53 | |
| frickler | corvus: jhesketh: thanks for merging my cacti patch, seems to have worked well and created e.g. http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=66569&rra_id=all | 08:54 |
| *** yboaron_ has quit IRC | 08:56 | |
| *** rcernin has quit IRC | 08:57 | |
| *** udesale has quit IRC | 08:57 | |
| *** udesale has joined #openstack-infra | 08:58 | |
| openstackgerrit | Hannu Hartikainen proposed openstack-infra/git-review master: Push with --no-follow-tags https://review.openstack.org/648646 | 08:59 |
| *** e0ne has joined #openstack-infra | 09:00 | |
| *** lucasagomes has joined #openstack-infra | 09:02 | |
| *** lucasagomes has quit IRC | 09:03 | |
| *** lucasagomes has joined #openstack-infra | 09:03 | |
| *** stakeda has quit IRC | 09:05 | |
| *** udesale has quit IRC | 09:07 | |
| *** udesale has joined #openstack-infra | 09:10 | |
| *** pcaruana has joined #openstack-infra | 09:15 | |
| *** udesale has quit IRC | 09:15 | |
| ykarel | quiquell, no idea | 09:22 |
| frickler | dirk: fyi, there's another error in the tumbleweed sync now in addition to the broken tmp dirs that still exist, but it is successfully ignored with your patch now: http://paste.openstack.org/show/748572/ | 09:24 |
| ykarel | quiquell, so just checked the patch and i can see job moved , so stein containers are working | 09:27 |
| ykarel | http://logs.openstack.org/19/647419/8/check/tripleo-ci-centos-7-standalone-upgrade/281fb53/logs/undercloud/var/log/extra/docker/docker_allinfo.log.txt.gz | 09:28 |
| ykarel | the job failed at validate-services | 09:28 |
| quiquell | ok so maybe there were not there | 09:28 |
| quiquell | ok then as expected | 09:28 |
| quiquell | :-) | 09:28 |
| quiquell | nice | 09:28 |
| *** priteau has joined #openstack-infra | 09:33 | |
| *** electrofelix has joined #openstack-infra | 09:37 | |
| *** dtantsur|afk is now known as dtantsur | 09:43 | |
| *** e0ne has quit IRC | 09:49 | |
| *** yamamoto has quit IRC | 09:52 | |
| *** e0ne has joined #openstack-infra | 09:54 | |
| openstackgerrit | renminmin proposed openstack/diskimage-builder master: Add DIB_LOCAL_IMAGE to use ubuntu local cloud image https://review.openstack.org/643885 | 09:59 |
| *** yamamoto has joined #openstack-infra | 10:00 | |
| *** wolverineav has joined #openstack-infra | 10:00 | |
| *** amansi26 has quit IRC | 10:01 | |
| *** wolverineav has quit IRC | 10:05 | |
| *** yamamoto has quit IRC | 10:11 | |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Separate out executor concerns from AnsibleJob https://review.openstack.org/648643 | 10:11 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: runner: implement prep-workspace https://review.openstack.org/607082 | 10:11 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: runner: add configuration schema https://review.openstack.org/640672 | 10:11 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: runner: add execute sub-command https://review.openstack.org/630944 | 10:11 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: add command line interface https://review.openstack.org/644770 | 10:11 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: runner: add job parameters listing https://review.openstack.org/644795 | 10:12 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: Add depends-on support to frozen jobs API https://review.openstack.org/639022 | 10:12 |
| openstackgerrit | Joshua Hesketh proposed openstack-infra/zuul master: zuul-runner: add support for depends-on https://review.openstack.org/632064 | 10:12 |
| *** yamamoto has joined #openstack-infra | 10:13 | |
| openstackgerrit | Hannu Hartikainen proposed openstack-infra/git-review master: Push with --no-follow-tags https://review.openstack.org/648646 | 10:28 |
| *** Lucas_Gray has joined #openstack-infra | 10:31 | |
| *** ricolin has quit IRC | 10:33 | |
| openstackgerrit | Luigi Toscano proposed openstack-infra/zuul-jobs master: stage-output: fix dotfiles handling https://review.openstack.org/648658 | 10:34 |
| *** Lucas_Gray has quit IRC | 10:48 | |
| *** Lucas_Gray has joined #openstack-infra | 10:50 | |
| *** e0ne has quit IRC | 10:50 | |
| *** Lucas_Gray has quit IRC | 10:50 | |
| *** Lucas_Gray has joined #openstack-infra | 10:51 | |
| *** jaosorior has joined #openstack-infra | 10:55 | |
| *** slaweq has joined #openstack-infra | 11:00 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: trigger: add job filter event https://review.openstack.org/639905 | 11:07 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: webtrigger: add initial driver and event https://review.openstack.org/555153 | 11:07 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: webtrigger: add web route and rpclistener https://review.openstack.org/554839 | 11:07 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add build button to trigger job https://review.openstack.org/635716 | 11:07 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: webtrigger: add support for parameterized trigger https://review.openstack.org/644484 | 11:07 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add build modal with a parameter form https://review.openstack.org/644485 | 11:07 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add support for checkbox and list parameters https://review.openstack.org/648661 | 11:07 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: add triggers information to pipeline list https://review.openstack.org/637670 | 11:08 |
| *** jaosorior has quit IRC | 11:09 | |
| *** slaweq has quit IRC | 11:09 | |
| *** slaweq has joined #openstack-infra | 11:14 | |
| *** whoami-rajat has quit IRC | 11:42 | |
| *** bhavikdbavishi has joined #openstack-infra | 11:45 | |
| *** fatema_ has joined #openstack-infra | 11:53 | |
| *** e0ne has joined #openstack-infra | 11:54 | |
| *** wolverineav has joined #openstack-infra | 12:01 | |
| *** e0ne has quit IRC | 12:02 | |
| *** wolverineav has quit IRC | 12:06 | |
| *** weshay is now known as weshay|rover | 12:06 | |
| *** jamesmcarthur has joined #openstack-infra | 12:18 | |
| redrobot | Good morning Infra friends! | 12:19 |
| *** janki has quit IRC | 12:20 | |
| redrobot | I'm trying to debug a failing gate in Barbican. The problem seems to be that get_python_exec_prefix in DevStack is evaluating to the wrong path | 12:20 |
| redrobot | http://git.openstack.org/cgit/openstack-dev/devstack/tree/inc/python#n41 | 12:20 |
| *** trown|outtypewww is now known as trown | 12:20 | |
| redrobot | It appears to be looking for /usr/local/bin/keystone-manage | 12:20 |
| redrobot | but in Fedora, it is being installed to /usr/bin/keystone-manage | 12:21 |
| *** EmilienM is now known as EvilienM | 12:21 | |
| fungi | redrobot: did this behavior change recently? on debian-based distros pip installs into /usr/local while pip on red hat distros installs into /usr | 12:22 |
| fungi | redrobot: you also may have more luck asking in #openstack-qa since this looks like a devstack behavior and the qa team are the maintainers of devstack | 12:23 |
| *** kgiusti has joined #openstack-infra | 12:26 | |
| *** kgiusti has quit IRC | 12:29 | |
| redrobot | fungi, yeah, our gates just started failing last week. I'll check #openstack-qa, thanks! | 12:32 |
| *** markvoelker has quit IRC | 12:32 | |
| *** jpena is now known as jpena|lunch | 12:32 | |
| *** jamesmcarthur has quit IRC | 12:33 | |
| *** e0ne has joined #openstack-infra | 12:36 | |
| zigo | fungi: Hello there! | 12:37 |
| zigo | fungi: Is the mirroring done this time? | 12:37 |
| fungi | zigo: yes, it finished yesterday but it was probably very late in your day by that time | 12:44 |
| zigo | fungi: Ok, so I can ask for the review of the Debian Buster image then? :) | 12:44 |
| fungi | yes! | 12:47 |
| *** jtomasek has joined #openstack-infra | 12:47 | |
| fungi | what was the review number on that one again? | 12:47 |
| *** pabelanger has joined #openstack-infra | 12:49 | |
| *** jamesmcarthur has joined #openstack-infra | 12:50 | |
| *** aaronsheffield has joined #openstack-infra | 12:54 | |
| *** lseki has joined #openstack-infra | 12:55 | |
| *** quiquell is now known as quiquell|lunch | 12:56 | |
| *** jaosorior has joined #openstack-infra | 12:56 | |
| *** kgiusti has joined #openstack-infra | 12:57 | |
| *** efried is now known as fried_rice | 12:58 | |
| *** e0ne has quit IRC | 13:02 | |
| fungi | nevermind, found it once i got a bit of coffee in me ;) | 13:02 |
| fungi | ianw: if you're not done yet for the week, have a moment to look back over 645574 again? | 13:04 |
| *** yamamoto has quit IRC | 13:07 | |
| *** eharney has joined #openstack-infra | 13:07 | |
| *** yamamoto has joined #openstack-infra | 13:09 | |
| zigo | fungi: https://review.openstack.org/#/c/645574/ | 13:10 |
| zigo | Yeah. | 13:11 |
| zigo | fungi: Sorry, was busy with colleagues. | 13:11 |
| *** e0ne has joined #openstack-infra | 13:11 | |
| fungi | no need to apologize! | 13:11 |
| zigo | I've done quite some bits for packaging Stein already. | 13:11 |
| zigo | keystone, glance, cinder, neutron, nova, all done ... :P | 13:11 |
| fungi | we're all busy, so totally understand | 13:11 |
| zigo | (plus all other 120+ dependencies) | 13:11 |
| fungi | yeah, that's a massive pile of work | 13:11 |
| *** e0ne has quit IRC | 13:13 | |
| *** lpetrut has joined #openstack-infra | 13:15 | |
| odyssey4me | hmm, is it possible that some of the pypi mirrors are a bit broken? I'm seeing the same task pass in some regions, but fail in others - eg: http://logs.openstack.org/66/648566/4/check/openstack-ansible-deploy-aio_lxc-centos-7/e6a1d27/logs/ara-report/result/dfc3e494-f38b-4219-b5d0-c7c6fa921573/ vs http://logs.openstack.org/66/648566/4/check/openstack-ansible-deploy-aio_proxy-ubuntu-bionic/c9b3de9/logs/ara-report/resul | 13:23 |
| odyssey4me | t/e286f165-6b72-4e44-9e3e-832d9c130c04/ | 13:23 |
| odyssey4me | ugh, http://logs.openstack.org/66/648566/4/check/openstack-ansible-deploy-aio_proxy-ubuntu-bionic/c9b3de9/logs/ara-report/result/e286f165-6b72-4e44-9e3e-832d9c130c04/ | 13:23 |
| *** rlandy has joined #openstack-infra | 13:26 | |
| openstackgerrit | Markus Hosch proposed openstack-infra/zuul master: Evaluate CODEOWNERS settings during canMerge check https://review.openstack.org/644557 | 13:26 |
| fungi | odyssey4me: we don't mirror pypi any longer, we just have caching web proxies. pip shouldn't be falling back on the wheel mirrors we build unless the proxy was unable to fetch https://files.pythonhosted.org/packages/d8/f3/413bab4ff08e1fc4828dfc59996d721917df8e8583ea85385d51125dceff/pip-19.0.3-py2.py3-none-any.whl or the index from pypi pointing to it | 13:28 |
| fungi | could point to issues with the proxies or with the fastly cdn pypi fronts their service with | 13:29 |
| *** jaosorior has quit IRC | 13:30 | |
| fungi | i seem to be able to retrieve http://mirror.iad.rax.openstack.org/pypi/packages/d8/f3/413bab4ff08e1fc4828dfc59996d721917df8e8583ea85385d51125dceff/pip-19.0.3-py2.py3-none-any.whl at the moment so i don't think it's constant | 13:30 |
| frickler | also bionic seems to try to look for a distro specific wheel, centos installs the generic one | 13:31 |
| fungi | well, the generic one is what pypi (via our cache) provides, while the platform-specific wheels we only build from the global requirements list (which pip isn't in anyway) so that shouldn't exist regardless | 13:32 |
| odyssey4me | yeah, I find it a bit odd that it's failing with a 404 rather than just realising it's not on that mirror | 13:33 |
| *** jtomasek has quit IRC | 13:34 | |
| *** jtomasek has joined #openstack-infra | 13:34 | |
| *** jpena|lunch is now known as jpena | 13:35 | |
| *** jtomasek has quit IRC | 13:36 | |
| cloudnull | mnaser +1 on dropping ocata from the gates. | 13:37 |
| *** priteau has quit IRC | 13:37 | |
| *** whoami-rajat has joined #openstack-infra | 13:38 | |
| *** e0ne has joined #openstack-infra | 13:42 | |
| fungi | odyssey4me: and you say it's not happening consistently? | 13:43 |
| *** jaosorior has joined #openstack-infra | 13:43 | |
| odyssey4me | fungi well, for https://review.openstack.org/648566 it does appear to be consistent | 13:44 |
| odyssey4me | it looks like it's bionic only, but that's likely because pip.conf on bionic has the extra wheel links configured - whereas centos does not | 13:44 |
| fungi | http://logs.openstack.org/66/648566/4/check/openstack-ansible-deploy-aio_lxc-ubuntu-bionic/dfb9540/logs/etc/host/pip.conf.txt.gz | 13:45 |
| fungi | that's only a fallback though | 13:46 |
| odyssey4me | hmm, I don't think pip works that way | 13:46 |
| fungi | well, sorry, "fallback" is a poor term here | 13:46 |
| odyssey4me | it queries all indexes | 13:46 |
| fungi | yes | 13:46 |
| fungi | and it should be finding pip at http://mirror.dfw.rax.openstack.org/pypi/simple | 13:46 |
| fungi | so the fact that it doesn't find pip at http://mirror.dfw.rax.openstack.org/wheel/ubuntu-18.04-x86_64 ought to be irrelevant to it | 13:47 |
| odyssey4me | right, so in this case pip doesn't find it in the index and it 404's... that seems like a bug to me | 13:47 |
| odyssey4me | yeah, as long as it finds it in any one of its indexes, it should be fine | 13:47 |
| *** bnemec is now known as beekneemech | 13:47 | |
| fungi | there are plenty of other packages we install which aren't in the built wheel set there too, and pip doesn't normally choke | 13:48 |
| *** stephenfin is now known as finucannot | 13:48 | |
| fungi | i have a feeling it's hiding an earlier failure to hit the http://mirror.dfw.rax.openstack.org/pypi/simple index | 13:48 |
| fungi | so the error about not finding pip at http://mirror.dfw.rax.openstack.org/wheel/ubuntu-18.04-x86_64 is likely a red herring | 13:48 |
| fungi | oh, indeed, http://mirror.dfw.rax.openstack.org/simple/pip/ | 13:49 |
| fungi | that's what pip in the simple index is linking to and it's a 404 | 13:49 |
| fungi | it should instead link to http://mirror.dfw.rax.openstack.org/pypi/simple/pip/ | 13:50 |
| fungi | which has content | 13:50 |
| *** psachin has quit IRC | 13:52 | |
| fungi | this seems to be the case for all entries in the index | 13:53 |
| odyssey4me | aha, so this is perhaps a content rewrite issue in the reverse proxy? | 13:53 |
| fungi | or could be pypi recently changed how packages are linked in the simple index | 13:53 |
| mordred | oh good | 13:53 |
| fungi | currently the simple index contains entries like <a href="/simple/pip/">pip</a> | 13:55 |
| *** yamamoto has quit IRC | 13:55 | |
| fungi | possible that until recently they did something like <a href="pip/">pip</a> instead | 13:55 |
| mordred | fungi: ugh. so it's anchored now | 13:56 |
| openstackgerrit | Luigi Toscano proposed openstack-infra/zuul-jobs master: stage-output: fix dotfiles handling https://review.openstack.org/648658 | 13:56 |
| fungi | yes, i just fetched https://pypi.org/simple/ to confirm it | 13:56 |
| mordred | :( | 13:56 |
| fungi | having to stick our pypi proxy on a separate tcp port would be unfun | 13:57 |
| mnaser | why not have two proxies, one listening at /simple ? | 13:57 |
| * mnaser will gladly do what's necessary as this blocks osa | 13:58 | |
| mnaser | so point directions and I can get at it | 13:58 |
| *** yamamoto has joined #openstack-infra | 13:59 | |
| fungi | well, also possible it's always been this way and pip checks simple/$packagename if the url for it in simple/index.html returns a 404 | 13:59 |
| *** ginopc has quit IRC | 14:00 | |
| fungi | so this might not be the actual problem | 14:00 |
| *** yamamoto has quit IRC | 14:00 | |
| fungi | but yeah, instead of a separate socket we could rewrite requests for ^/simple/.* to /pypi/simple/$1 | 14:00 |
| mordred | I would expect that potentially there would be much more breakage if all of the links are broken | 14:01 |
| fungi | and just remember that we can't ever have a separate tree called /simple for some other proxy purpose | 14:01 |
| *** jaosorior has quit IRC | 14:01 | |
| *** quiquell|lunch is now known as quiquell | 14:02 | |
| *** jaosorior has joined #openstack-infra | 14:02 | |
| *** wolverineav has joined #openstack-infra | 14:02 | |
| mnaser | mordred: maybe most projects are just using the infra built wheels | 14:02 |
| mnaser | and we're the only weirdos consuming something like 'pip' which is probably not part of requirements | 14:03 |
| frickler | also devstack is still capping pip<10, this may only be affecting latest pip | 14:03 |
| mordred | excellent. more variables | 14:04 |
| *** bauzas is now known as DaddyBear | 14:09 | |
| *** wolverineav has quit IRC | 14:09 | |
| fungi | next steps are probably to either attempt to reproduce with the same pip.conf and --verbose enabled (or sniffing communication to the mirror via tcpdump), or checking the apache access log on a mirror server to see what nodes are trying to hit | 14:09 |
| *** DaddyBear is now known as PapaOurs | 14:10 | |
| mnaser | fungi: perhaps we can get a hold on an openstack-ansible job and we can run a recheck and once it fails we can come back and sync up? | 14:12 |
| mnaser | which is probably the most automated and efficient way to getting us a broken environment to do our testing | 14:13 |
| openstackgerrit | Chris Dent proposed openstack-infra/project-config master: Stop nova-specs announcing in #openstack-placement https://review.openstack.org/648698 | 14:13 |
| *** cmurphy is now known as cmurpheus | 14:18 | |
| *** cmurpheus is now known as cmorpheus | 14:18 | |
| *** yamamoto has joined #openstack-infra | 14:18 | |
| fungi | that's an option | 14:19 |
| fungi | i'll set one for change 648566 job openstack-ansible-deploy-aio_proxy-ubuntu-bionic | 14:20 |
| mnaser | fungi: "openstack-ansible-deploy-aio_lxc-ubuntu-bionic" is bit better | 14:20 |
| mnaser | it has less magic going on, that one has a squid proxy and tries to test we work in proxied environments | 14:20 |
| fungi | k, will do | 14:21 |
| fungi | okay, autohold is set so when that job fails from odyssey4me's 13:42z recheck we should be able to poke around on it | 14:23 |
| *** yamamoto has quit IRC | 14:24 | |
| openstackgerrit | Malek Karray proposed openstack-infra/storyboard-webclient master: Converting a search query into a browse https://review.openstack.org/644424 | 14:24 |
| *** yamamoto has joined #openstack-infra | 14:25 | |
| *** yamamoto has quit IRC | 14:25 | |
| *** yamamoto has joined #openstack-infra | 14:26 | |
| *** yamamoto has quit IRC | 14:30 | |
| mnaser | fungi: the job failed at 14:24 UTC and you put the hold in at 14:23 UTC | 14:31 |
| mnaser | did we catch it or was it too late? :P | 14:31 |
| fungi | okay, i've got 158.69.65.235 now held from that build but i need to disappear to run errands in a few before some guests arrive for the weekend | 14:31 |
| fungi | who wants ssh keys added to it? | 14:31 |
| mnaser | https://github.com/mnaser.keys | 14:31 |
| mnaser | and ill then try to orchestrate cause ill be around all day | 14:32 |
| fungi | added | 14:32 |
| mnaser | I'm in, thank you sir | 14:32 |
| fungi | excellent | 14:32 |
| fungi | thanks! | 14:32 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Re-enable voting for opensuse15, opensuse423 and opensuse-tumbleweed https://review.openstack.org/648482 | 14:36 |
| clarkb | fwiw our mirrors/caches are publicly accessible in part to enable this sort of debugging | 14:36 |
| clarkb | you should be able to pip install pip locally against them to test | 14:36 |
| clarkb | --index-url and --trusted-hostname or similar | 14:37 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: web: add tenant and project scoped, JWT-protected actions https://review.openstack.org/576907 | 14:39 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: [WIP] Allow operator to generate auth tokens through the CLI https://review.openstack.org/636197 | 14:39 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: [WIP] Zuul CLI: allow access via REST https://review.openstack.org/636315 | 14:39 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: [WIP] Add Authorization Rules configuration https://review.openstack.org/639855 | 14:39 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: [WIP] Web: plug the authorization engine https://review.openstack.org/640884 | 14:40 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: Zuul Web: add /api/user/actions endpoint https://review.openstack.org/641099 | 14:40 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: authentication config: add optional token_expiry https://review.openstack.org/642408 | 14:40 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: [WIP] admin REST API: docker-compose PoC, frontend https://review.openstack.org/643536 | 14:40 |
| AJaeger | dmsimard: want to abandon https://review.openstack.org/648494 now? | 14:41 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: WIP: Add test demonstrating image build leak https://review.openstack.org/648706 | 14:47 |
| fungi | clarkb: yeah, that was my first suggestion, but i wasn't going to be around to actually set up and try that | 14:48 |
| fungi | so if folks want to have a node held to rule out something extra funky in the ansible aio setup, who am i to argue | 14:49 |
| mnaser | I think we might be hitting a pip bug? | 14:53 |
| mnaser | http://paste.openstack.org/show/748588/ | 14:54 |
| mnaser | it seems to actually find it fine, it might be doing its own introspection and finds the right path | 14:54 |
| mnaser | the problem is that when it fails to find it in the second index... it exits? | 14:54 |
| *** jamesmcarthur has quit IRC | 14:54 | |
| mnaser | cc odyssey4me ^ | 14:54 |
| clarkb | what version is listed in that constraints file? | 14:56 |
| clarkb | could it be that version isnt in the first index so not finding it in the second is an error? | 14:56 |
| mnaser | clarkb: ok that's interesting -- the command I ran is -- /openstack/venvs/wheel-builder/bin/pip2 install -U --constraint /var/www/repo/os-releases/19.0.0.0b2.dev48/utility-19.0.0.0b2.dev48-source-constraints.txt --find-links /var/www/repo/os-releases/19.0.0.0b2.dev48/ubuntu-18.04-x86_64/ --log /var/log/python_venv_build.log pip setuptools wheel -vvvv | 14:57 |
| mnaser | let me double check | 14:57 |
| mnaser | /var/www/repo/os-releases/19.0.0.0b2.dev48/utility-19.0.0.0b2.dev48-source-constraints.txt contains --constraint https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt?h=1785672e676e025db75a33fae370e42f1c70bec8 | 14:57 |
| mnaser | which doesn't have pip there | 14:57 |
| mnaser | so im assuming with -U it's trying to get the latest and greatest | 14:57 |
| *** jcoufal has joined #openstack-infra | 14:57 | |
| mnaser | and it is trying to seek out all indexes to find the latest version (I guess) | 14:58 |
| *** e0ne has quit IRC | 14:58 | |
| fungi | okay, heading out for a little while but should be back in an hour or so | 15:01 |
| *** e0ne has joined #openstack-infra | 15:07 | |
| *** jamesmcarthur has joined #openstack-infra | 15:08 | |
| *** smarcet has joined #openstack-infra | 15:11 | |
| smarcet | fungi:: morning | 15:11 |
| smarcet | fungi: yesterday i received by email an error from the former openstackid production instance | 15:12 |
| smarcet | fungi: i think that we could take it down | 15:12 |
| *** e0ne has quit IRC | 15:12 | |
| smarcet | fungi: not sure why got hitted | 15:12 |
| *** eharney has quit IRC | 15:12 | |
| *** jamesmcarthur has quit IRC | 15:13 | |
| odyssey4me | mnaser fungi this is slightly orthoganal, but why are we serving the infra built wheels using an index, instead of just putting them in an auto-index folder on the web server and using find-links? | 15:15 |
| odyssey4me | that would mean the wheels can just be in a flat folder - not need to make an index | 15:16 |
| *** dpawlik has quit IRC | 15:16 | |
| * mnaser doesn't know enough about the inner workings those pieces | 15:17 | |
| *** jamesmcarthur has joined #openstack-infra | 15:18 | |
| *** slaweq has quit IRC | 15:19 | |
| odyssey4me | the advantage of using find-links are threefold - one is that the implementation of the local wheel serving simpler, another is that pip doesn't expect anything fancy from the thing serving the wheels and another is that anything found in find-links is preferred automatically above the indexes | 15:19 |
| *** jaosorior has quit IRC | 15:21 | |
| clarkb | odyssey4me: I don't think we want that last behavior fwiw | 15:21 |
| clarkb | we want the packages to be resolved properly: use the newest valid version then prefer wheels if available | 15:21 |
| mnaser | I think newer versions of pip don't hit this bug because they just gracefully ignore things when it can't be found | 15:21 |
| clarkb | mnaser: what version of pip are you using there? | 15:22 |
| odyssey4me | clarkb it will only happen if the links and indexes have the same version - if the index has a newer version, that still wins | 15:22 |
| clarkb | odyssey4me: ok so that behavior already exists in our setup | 15:22 |
| mnaser | clarkb: 9.0.1 in here | 15:22 |
| mnaser | it doesn't seem to happen in centos which I think ships newer centos (via rod) | 15:22 |
| mnaser | rod** | 15:22 |
| mnaser | RDO* | 15:22 |
| *** kgiusti has quit IRC | 15:23 | |
| clarkb | mnaser: are you downgrading to 9.0.1? | 15:23 |
| mnaser | no, we're running -U pip setuptools wheel | 15:23 |
| clarkb | where does 9.0.1 come from then? (I just checked recently and our images do start with latest pip) | 15:24 |
| clarkb | at least bionic and xenial | 15:24 |
| *** dhellmann_ has joined #openstack-infra | 15:24 | |
| mnaser | I think 9.0.1 comes from the fact we install python-setuptools | 15:24 |
| mnaser | err | 15:24 |
| mnaser | python-virtualenv | 15:24 |
| *** gouthamr has quit IRC | 15:24 | |
| *** dmellado has quit IRC | 15:24 | |
| mnaser | and then create a virtualenv which in turn uses whatever pip it decides to use in there | 15:24 |
| *** dhellmann has quit IRC | 15:24 | |
| *** dulek has quit IRC | 15:24 | |
| mnaser | (this happens inside the OSA containers, perhaps odyssey4me can correct me if im wrong) | 15:24 |
| *** ruijing has quit IRC | 15:25 | |
| odyssey4me | yeah, we install the distro package for virtualenv, then use pip to upgrade pip | 15:25 |
| *** dmellado has joined #openstack-infra | 15:26 | |
| *** kgiusti has joined #openstack-infra | 15:26 | |
| *** roman_g has quit IRC | 15:26 | |
| odyssey4me | clarkb the issue here is that an index and find-links are treated differently - if find-links was being used, pip wouldn't care if the package wasn't there | 15:26 |
| clarkb | odyssey4me: as for how we end up with that index setup it is how pip wheel caches them on disk and we just copy them over iirc | 15:27 |
| clarkb | odyssey4me: its never cared about the indexes before | 15:27 |
| clarkb | we've done this for years | 15:27 |
| clarkb | and its been fine | 15:27 |
| mnaser | clarkb: the thing is, pip is missing from ubuntu-18.04 | 15:27 |
| *** dulek has joined #openstack-infra | 15:27 | |
| mnaser | but it is in 16.04 and centos-7 | 15:27 |
| odyssey4me | yeah, it is odd | 15:27 |
| *** dhellmann_ is now known as dhellmann | 15:27 | |
| odyssey4me | mnaser actually pip is not on centos - only virtualenv ;) | 15:27 |
| *** gouthamr has joined #openstack-infra | 15:27 | |
| *** slaweq has joined #openstack-infra | 15:28 | |
| clarkb | mnaser: we install it via get-pip | 15:28 |
| mnaser | odyssey4me, clarkb: by this I mean the 3 packages we want: pip, setuptools and wheel all have wheels in the mirrors for centos and ubuntu-16.04 | 15:28 |
| frickler | to be more specific http://mirror.bhs1.ovh.openstack.org/wheel/centos-7-x86_64/pip/ has 9.0.1, so does 16.04, but nothing for 18.04 | 15:28 |
| odyssey4me | oh, that's odd | 15:28 |
| mnaser | we could work around this just by having pip/setuptools/wheel there at least. | 15:28 |
| mnaser | it will just find 9.0.1 there and pick 19.0.3 from pypi | 15:29 |
| clarkb | as was mentioned before I think we removed themfrom the wheel build list | 15:29 |
| odyssey4me | bbiab conf call | 15:29 |
| clarkb | the total number of pacakges we build wheels for is quite tiny and I know we have things like nodepool and zuul consuming package that don't exist there | 15:29 |
| frickler | so maybe just having an empty http://mirror.bhs1.ovh.openstack.org/wheel/ubuntu-18.04-x86_64/pip/ dir would solve the issue | 15:30 |
| *** chandankumar is now known as raukadah | 15:30 | |
| mnaser | if someone wants to mkdir that, I can test it. | 15:30 |
| *** smarcet has quit IRC | 15:30 | |
| *** gmann is now known as gmann_afk | 15:32 | |
| *** slaweq has quit IRC | 15:32 | |
| tosky | another issue found in zuul-jobs/stage-output: https://review.openstack.org/#/c/648658/ | 15:32 |
| *** kopecmartin is now known as kopecmartin|off | 15:32 | |
| clarkb | mnaser: is your pip config using the wheels as an extra-index-url or an index-url? | 15:33 |
| mnaser | clarkb: our pip config inside our containers is cp'd from the host's /etc/pip.conf (the one that infra ships) | 15:33 |
| mnaser | https://www.irccloud.com/pastebin/sYzfWSal/ | 15:33 |
| mnaser | which is that ^ | 15:33 |
| odyssey4me | clarkb the pip config comes from the host, which is provided in the image - so it's using http://logs.openstack.org/66/648566/4/check/openstack-ansible-deploy-aio_lxc-ubuntu-bionic/dfb9540/logs/etc/host/pip.conf.txt.gz | 15:33 |
| *** sshnaidm|off has quit IRC | 15:34 | |
| clarkb | ok I am unable to reproduce the behaviro you have locally | 15:34 |
| clarkb | giev me a second to paste up what I get locally | 15:34 |
| openstackgerrit | Malek Karray proposed openstack-infra/storyboard-webclient master: Converting a search query into a browse https://review.openstack.org/644424 | 15:35 |
| clarkb | need to find the right number v's to use without making this output huge | 15:35 |
| odyssey4me | well, so - let me find that bug I was looking at earlier - I think the issue might be with the pip in the package provided by ubuntu | 15:37 |
| odyssey4me | https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=837764 | 15:37 |
| openstack | Debian bug 837764 in src:python-pip "python-pip: Using `--extra-index-url` results in `HTTPError: 404 Client Error: NOT FOUND`" [Important,Open] | 15:37 |
| clarkb | http://paste.openstack.org/show/748593/ is how pip 9.0.1 behaves when asking it to pip install -U against those two indexes with the wheels as an extrai ndex url | 15:37 |
| clarkb | using pip install pip 9.0.1 not debuntu packages | 15:38 |
| clarkb | it correctly finds the 404 and ignores that in the extra index url | 15:38 |
| clarkb | wow so debian has patched pip that causes this to break | 15:38 |
| clarkb | I don't think this is something we can fix for you in the mirror | 15:38 |
| clarkb | any idea which debian patch is referred to there? | 15:39 |
| odyssey4me | well, this is a really old bug | 15:40 |
| *** ykarel has quit IRC | 15:40 | |
| mnaser | ok I can replicate this | 15:40 |
| mnaser | docker run -it --rm ubuntu:18.04 /bin/bash | 15:40 |
| mnaser | apt-get update | 15:40 |
| mnaser | apt-get install curl python-virtualenv | 15:40 |
| mnaser | curl http://logs.openstack.org/66/648566/4/check/openstack-ansible-deploy-aio_lxc-ubuntu-bionic/dfb9540/logs/etc/host/pip.conf.txt.gz -o /etc/pip.conf | 15:40 |
| mnaser | virtualenv --no-download venv | 15:40 |
| mnaser | ./venv/bin/pip2 install -U pip | 15:40 |
| mnaser | damnit, sorry. | 15:40 |
| mnaser | https://www.irccloud.com/pastebin/A3Mxhv5F/ | 15:41 |
| mnaser | that will fail | 15:41 |
| *** knikolla is now known as nikrin | 15:41 | |
| clarkb | right seems to be a bug in the distro packaging | 15:41 |
| clarkb | what if you pip install virtualenv instead of using the distro packaging | 15:41 |
| mordred | yeah. I've become a big fan of not touching distro packages for these things | 15:42 |
| clarkb | apt-get install python-pip && pip install virtualenv && virtualenv venv && venv/bin/pip do things | 15:42 |
| mnaser | centos doesn't ship pip | 15:42 |
| mordred | install pip via get-pip and then things are clean from there ... but I'm also me | 15:42 |
| *** slaweq has joined #openstack-infra | 15:42 | |
| clarkb | mnaser: get-pip.py works on centos | 15:42 |
| mnaser | so our best 'path to success' is get virtualenv which we have available across all systems | 15:42 |
| clarkb | also I thought centos did pcakge pip | 15:42 |
| mnaser | and then go from there | 15:42 |
| mnaser | python-pip is shipped through RDO only | 15:42 |
| *** ricolin has joined #openstack-infra | 15:43 | |
| mordred | mnaser: yeah. that's the thing don't install pip from distro at all - just use get-pip because otherwise madness starts happening really quickly | 15:43 |
| clarkb | looks like it is in epel | 15:43 |
| clarkb | so ya either use epel or get-pip | 15:43 |
| mordred | but - that's jsut me running my mouth | 15:43 |
| mnaser | ok, well, I'll ask odyssey4me if he's okay with us maybe switching to get-pip | 15:43 |
| odyssey4me | clarkb the trouble is that centos doesn't have a pip package, so we'd have to revert to using get-pip, and that increases complexity somewhat | 15:43 |
| clarkb | but I don't think we can be responsible for broken distro packaging of pip | 15:43 |
| mordred | I'm rather annoyed that distro packaging has patched and broken pip | 15:43 |
| odyssey4me | when we started using the distro packages, we got to remove several hundreds of lines of code from all over the place - that's really not a place we'd like to return to | 15:44 |
| mnaser | not really much we can do I guess thanks to distros | 15:45 |
| mordred | nod. maybe we can get the debian package unbroken? | 15:45 |
| * mnaser still thinks we should build our own distro one day | 15:45 | |
| mnaser | I mean this _only_ affects our gate | 15:45 |
| mordred | zigo is around these days - I don't know how much magic he's got | 15:45 |
| mnaser | we don't make use of extra-index-urls these days | 15:45 |
| mnaser | at all | 15:45 |
| mnaser | so the only thing is if we somehow got maybe an empty pip/wheel folder, it might think that there's nothing there and it'll be happy afterwards | 15:46 |
| clarkb | mnaser: you could remove the extra index url when installing pip | 15:46 |
| mnaser | clarkb: I tried to look for command line options to 'un-enable' the extra index url and couldn't find any | 15:46 |
| clarkb | like make your installation line pip install --index-url $somespecific source | 15:46 |
| zigo | o/ | 15:46 |
| clarkb | mnaser: I think if you set a specific index-url on the command line only command line indexes are taken | 15:46 |
| *** eharney has joined #openstack-infra | 15:46 | |
| mnaser | clarkb: yeah but then we get into the mess of where we get the index-url from, we don't have this ever set so it takes the python default | 15:47 |
| mnaser | but in our gates it takes the value from /etc/pip.conf so we don't have to do magical introspection | 15:47 |
| zigo | mordred: So, pip install is broken in Ubuntu? | 15:47 |
| mnaser | zigo: my repro https://www.irccloud.com/pastebin/A3Mxhv5F/ | 15:47 |
| clarkb | mnaser: you could set it to pypi | 15:47 |
| *** lpetrut has quit IRC | 15:47 | |
| clarkb | then just hope that is reliable enough for a small number of package installs | 15:48 |
| mnaser | kinda defeats our purpose of leveraging our mirrors? :\ | 15:48 |
| *** sshnaidm|off has joined #openstack-infra | 15:48 | |
| odyssey4me | mnaser we could work around it by having our AIO bootstrap remove that line, but that would be unfortunate because we'd lose out on using the pre-built wheels | 15:48 |
| clarkb | zigo: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=837764 is the debian bug | 15:48 |
| openstack | Debian bug 837764 in src:python-pip "python-pip: Using `--extra-index-url` results in `HTTPError: 404 Client Error: NOT FOUND`" [Important,Open] | 15:48 |
| * mnaser doesn't want to be a pain but really wonders if creating an empty pip and wheel folder would help get us through | 15:48 | |
| mordred | mnaser: yeah - but you only need to do it for installing pip step, right? | 15:48 |
| mordred | like, once your virtualenv has newer pip, you should be able to not do --index-url=pypi? | 15:49 |
| clarkb | mnaser: if you want to do that convince requirements team to add those packages back to constraints and we'll install them | 15:49 |
| mnaser | clarkb: I don't want to add them as constraint, just an empty folder to work around it, I think that might do the trick | 15:49 |
| clarkb | mnaser: but I don't want people to depend on the infra team manually mkdiring everytime we discover debian is silly | 15:49 |
| openstackgerrit | Malek Karray proposed openstack-infra/storyboard-webclient master: Enabling suggestions on all user fields https://review.openstack.org/648525 | 15:49 |
| odyssey4me | mordred in theory, yes - however that would involve putting workaround code in just for the purpose of openstack CI, which is not ideal | 15:49 |
| mnaser | ok, fair | 15:50 |
| zigo | Why not simply apt-get install python-pip ? | 15:50 |
| clarkb | mnaser: the issue is you'll need this for any such package | 15:50 |
| zigo | What's wrong with it? | 15:50 |
| clarkb | zigo: its broken as per the bug above | 15:50 |
| *** slaweq has quit IRC | 15:50 | |
| *** psachin has joined #openstack-infra | 15:50 | |
| clarkb | zigo: it doesn't handle extra-index-urls properly supposedly due to a local debian patch | 15:50 |
| mnaser | clarkb: hmm.. ok, that's valid | 15:50 |
| mnaser | right, if it was those 2 only that makes sense but it _could_ be more. | 15:50 |
| odyssey4me | zigo in this case we are installing python-virtualenv, creating a virtualenv, then trying to upgrade pip inside that virtualenv... but the pi inside the virtualenv is not treating extra indexes properly | 15:50 |
| odyssey4me | *the pip | 15:50 |
| zigo | clarkb: Yeah, but there's a workaround ... :P | 15:51 |
| zigo | (ie: reinstall pip with pip ...) | 15:52 |
| clarkb | zigo: there are a few (whcih we've suggested above), but ideally debian's package would also be fixed | 15:52 |
| odyssey4me | that is exactly what we're trying to do | 15:52 |
| mordred | yeah. I'd love to get the debian package to not be broken - even though I don't use it - just because it's the right thing to do and it's broken due to a local patch | 15:52 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Add logs spec https://review.openstack.org/648714 | 15:52 |
| mordred | but I doubt that'll be a _short-term_ fix for anybody | 15:52 |
| zigo | clarkb: If I understand the bug report, it's a missing patch? | 15:52 |
| zigo | clarkb: Buster is still in freeze, so I could address the bug and fix it ... | 15:53 |
| clarkb | zigo: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=837764#15 says that the fix is to install upstream pip so that the debian patch is overwritten | 15:53 |
| openstack | Debian bug 837764 in src:python-pip "python-pip: Using `--extra-index-url` results in `HTTPError: 404 Client Error: NOT FOUND`" [Important,Open] | 15:53 |
| clarkb | unfortunately they haven't linked to the debian patch from what I can see so not sure what patch that is | 15:53 |
| mnaser | I wonder if there's a command line option like --no-extra-index-url | 15:53 |
| * mnaser try | 15:53 | |
| mnaser | nope, doesn't exist | 15:54 |
| mnaser | even running it with `--isolated` seems to fail because it still seems to care about `pip.conf` | 15:54 |
| zigo | clarkb: The bug is about version 8.1.2-2, Buster has version 18.1-4 ... | 15:54 |
| zigo | clarkb: Could you test with the new version? | 15:54 |
| clarkb | mnaser: even if youset a --index-url? | 15:55 |
| zigo | Can you give me a working example of this pip command with extra index? | 15:56 |
| zigo | So I can try to reproduce and fix ... | 15:56 |
| mnaser | /openstack/venvs/wheel-builder/bin/pip2 install -vvv -U --index-url '' pip => 1 location(s) to search for versions of pip: * http://mirror.bhs1.ovh.openstack.org/wheel/ubuntu-18.04-x86_64/pip/ | 15:56 |
| clarkb | mnaser: same thing if you give --index-url a value? | 15:57 |
| *** yamamoto has joined #openstack-infra | 15:57 | |
| mnaser | clarkb: same for `/openstack/venvs/wheel-builder/bin/pip2 install -vvv -U --index-url https://pypi.org/simple/ pip` | 15:57 |
| clarkb | zigo: using pypi provided pip you can pip install --extra-index-url $url_that_will_404 $pacakge | 15:57 |
| clarkb | zigo: one example you can use is starting from older pip (say 9.0.1) `pip install --extra-index-url http://mirror.dfw.rax.openstack.org/wheel/ubuntu-18.04-x86_64 -U pip | 15:58 |
| *** trown is now known as trown|lunch | 15:58 | |
| clarkb | zigo: that extra index url does not have a pip directory so it will 404. The expectation is that pip will use the pypi package (default index-url) and ignore the 404 on the extra-index-url. The debian packaged pip errors on the 404 | 15:59 |
| *** quiquell is now known as quiquell|off | 15:59 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-sphinx master: Update to new docs jobs https://review.openstack.org/648718 | 15:59 |
| zigo | clarkb: What's the version of python-pip that you guys are using? The one in Bionic? | 15:59 |
| mnaser | zigo: correct | 15:59 |
| zigo | 9.0.1-2.3~ubuntu1 ? | 15:59 |
| mnaser | zigo: we use virtualenv to get pip though, if that makes sense | 15:59 |
| mnaser | zigo: apt-get install python-virtualenv; virtualenv venv; venv/bin/pip2 install --extra-index-url http://mirror.dfw.rax.openstack.org/wheel/ubuntu-18.04-x86_64 -U pip | 16:00 |
| mnaser | and that should fail | 16:00 |
| zigo | mnaser: Ok, I'll try. This means using the system's pip though... :P | 16:00 |
| mnaser | zigo: well yeah, we want to upgrade to non-system pip inside a venv (which is pretty neat and clean imho, we don't clobber things) | 16:01 |
| *** yamamoto has quit IRC | 16:01 | |
| zigo | mnaser: There's really no reason why you'd want to do that, you only want a pip that works ... :P | 16:01 |
| * zigo starts an ubuntu image in his internal cloud | 16:02 | |
| clarkb | reading the patches in the debian packages I don't see an obvious culprit fwiw | 16:02 |
| clarkb | I wonder if it is actually a patch in requests or similar. Like the exception type is different somehow making pip not catch it properly | 16:02 |
| zigo | clarkb: Where may I download the Bionic qcow that you are using in infra, so I can use the same image? | 16:02 |
| mnaser | zigo: if you feel like using docker, I have a very small repro | 16:03 |
| zigo | mnaser: I don't ! :) | 16:03 |
| mnaser | https://www.irccloud.com/pastebin/A3Mxhv5F/ | 16:03 |
| zigo | I prefer a VM. | 16:03 |
| mnaser | you can run the same in a VM :) | 16:03 |
| mnaser | just ignore the first line and you'll get the exact failure | 16:03 |
| zigo | I don't want to do that, I want to fix the python-pip package of Ubuntu. | 16:04 |
| *** PapaOurs is now known as bauzas | 16:04 | |
| odyssey4me | zigo you won't need the infra image - any stock ubuntu 18 will do | 16:04 |
| clarkb | ya since this is ubuntu's packaging our image shouldn't matter | 16:04 |
| mnaser | https://www.irccloud.com/pastebin/dK7bLzIN/ | 16:05 |
| mnaser | zigo: ^ that fails in ubuntu 18.04 | 16:05 |
| zigo | This ? https://cloud-images.ubuntu.com/bionic/20190327/bionic-server-cloudimg-amd64.img | 16:05 |
| mnaser | should fail in a VM with no problems | 16:06 |
| odyssey4me | zigo and this is only with installing python-virtualenv/virtualenv via ubuntu packages, then creating a virtualenv, then trying to update pip in the virtualenv... and the reason we do that is to ensure that regardless of distribution preferences, all our virtualenv use the same pip package so we know what to expect from it | 16:06 |
| mnaser | fail with no problems, ha. but that's a repro | 16:06 |
| *** ricolin has quit IRC | 16:07 | |
| clarkb | HTTPError is the correct exception, then in pip.index.HTMLPage.get_page the try except basically catches HTTPError and says error ignoring and moves on | 16:10 |
| clarkb | that is how upstream pip handles this properly | 16:11 |
| clarkb | now to find debuntu's index.py | 16:11 |
| clarkb | https://salsa.debian.org/python-team/modules/python-pip/blob/master/src/pip/_internal/index.py#L76-79 upstreams handling of that should go through there | 16:14 |
| clarkb | http://paste.openstack.org/show/748593/ and my paste using upstream pip shows that message | 16:14 |
| *** e0ne has joined #openstack-infra | 16:14 | |
| *** e0ne has quit IRC | 16:14 | |
| clarkb | my hunch is that that doesn't get raised as requests.HTTPError and is instead debubuntu.somewhere.requests.HTTPError | 16:18 |
| zigo | mnaser: What you pasted worked for me, it's *after* that upgrading pip broke stuff... | 16:19 |
| zigo | Oh, ok, got the error... | 16:21 |
| zigo | So what patch is to be removed then? | 16:22 |
| zigo | There's quite a few in the Debian package. | 16:23 |
| clarkb | zigo: I haven't sorted that out and unfortunately that bug doesn't say anything more specific | 16:23 |
| clarkb | however, my hunch is that it has to do with the handling of unvendoring code in pip intersecting with debian somehow | 16:23 |
| *** tosky has quit IRC | 16:23 | |
| zigo | I can dicotomy search it ... :P | 16:23 |
| clarkb | because it should catch that requests.HTTPError and ignore it after logging it, but that isn't happening | 16:23 |
| zigo | Oh ok. | 16:24 |
| zigo | Well, I'm not surprised. | 16:24 |
| zigo | Bad upstream... :P | 16:24 |
| clarkb | if that object isn't actually requests.HTTPError but isntead debian.path.requests.HTTPError or requests.somewhere.else.HTTPError we would see this I think | 16:24 |
| openstackgerrit | Merged openstack-infra/zuul-sphinx master: Update to new docs jobs https://review.openstack.org/648718 | 16:24 |
| mnaser | clarkb, zigo, mordred: I've managed to workaround it fro now in a very naughty way | 16:26 |
| *** diablo_rojo has joined #openstack-infra | 16:26 | |
| mnaser | PIP_CONFIG_FILE=/dev/null pip2 install -U pip | 16:26 |
| zigo | mnaser: Come on, tell, don't tease ! :) | 16:26 |
| zigo | Ah ... | 16:26 |
| zigo | Well, indeed, the pip.conf was the wrong bits... | 16:26 |
| clarkb | zigo: its not | 16:26 |
| zigo | Without it, it does work. | 16:26 |
| mnaser | --isolated doesn't work because /etc/pip.conf is always used | 16:26 |
| clarkb | the pip.conf triggers the broken path in the debian package | 16:26 |
| clarkb | but the pip.conf is valid | 16:27 |
| zigo | Ok. | 16:27 |
| *** slaweq has joined #openstack-infra | 16:28 | |
| mnaser | :O | 16:30 |
| *** rlandy is now known as rlandy|brb | 16:30 | |
| zigo | I'm trying to build the latest python-pip package from Buster, just to see what happens... | 16:31 |
| zigo | Same stuff ... | 16:32 |
| *** roman_g has joined #openstack-infra | 16:33 | |
| *** pgaxatte has quit IRC | 16:36 | |
| clarkb | zigo: I think we want to inspect what exception is actually being thrown at https://salsa.debian.org/python-team/modules/python-pip/blob/master/src/pip/_internal/index.py#L164-175 if you are able to put a breakpoint there or add some debugging printfs | 16:36 |
| clarkb | zigo: maybe add an except Exception as exc: #exception debugging stuff here | 16:36 |
| *** Lucas_Gray has quit IRC | 16:37 | |
| clarkb | zigo: it should be handled at line 164 but I believe it is falling through, but I don't have a good debuntu install to test on currently | 16:37 |
| *** Lucas_Gray has joined #openstack-infra | 16:38 | |
| corvus | except Exception as exc: print('Got exception %s' % repr(exc)) | 16:38 |
| corvus | putting that before line 174 should do the trick | 16:38 |
| openstackgerrit | Hervé Beraud proposed openstack-dev/pbr master: Fix parsing on egg names with dashes from git URLs https://review.openstack.org/648727 | 16:38 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: WIP - Pagure driver https://review.openstack.org/604404 | 16:39 |
| *** slaweq has quit IRC | 16:43 | |
| *** wolverineav has joined #openstack-infra | 16:44 | |
| *** fried_rice is now known as fried_rolls | 16:45 | |
| *** helenaAM has quit IRC | 16:45 | |
| * zigo got to go | 16:48 | |
| zigo | sorry, I tried... | 16:48 |
| *** rossella_s has quit IRC | 16:48 | |
| *** rlandy|brb is now known as rlandy | 16:49 | |
| *** lucasagomes has quit IRC | 16:49 | |
| *** Lucas_Gray has quit IRC | 16:50 | |
| *** bhavikdbavishi has quit IRC | 16:51 | |
| fungi | catching up from lunch scrollback, but pretty sure find-links is going away in pip or already has in newer versions | 16:51 |
| *** bhavikdbavishi has joined #openstack-infra | 16:54 | |
| fungi | the reason there are ubuntu-16.04 pip et cetera mirrored, i think, is that it was added briefly to global requirements when someone was trying to pin it in an inadvisable manner | 16:54 |
| openstackgerrit | Hervé Beraud proposed openstack-dev/pbr master: Fix parsing on egg names with dashes from git URLs https://review.openstack.org/648727 | 16:54 |
| mnaser | infra-root: held vm 158.69.65.235 can be dropped | 16:54 |
| *** jpich has quit IRC | 16:55 | |
| *** gfidente is now known as gfidente|afk | 16:55 | |
| *** rpittau is now known as rpittau|afk | 16:57 | |
| *** ykarel has joined #openstack-infra | 16:59 | |
| clarkb | I'm spinning up an ubuntu container now to do the package edit ninja style | 17:00 |
| *** iurygregory has quit IRC | 17:02 | |
| clarkb | Got exception HTTPError(u'404 Client Error: Not Found for url: http://mirror.dfw.rax.openstack.org/wheel/ubuntu-18.04-x86_64/pip/',) confirmed it is falling through | 17:02 |
| *** ykarel has quit IRC | 17:03 | |
| fungi | and now i'm caught up on scrollback. exciting developments! | 17:04 |
| clarkb | (<class 'pip._vendor.requests.exceptions.HTTPError'>, HTTPError(u'404 Client Error: Not Found for url: http://mirror.dfw.rax.openstack.org/wheel/ubuntu-18.04-x86_64/pip/',), <traceback object at 0x7fae017bb200>) | 17:05 |
| fungi | thanks mnaser, i'm releasing that held node now | 17:05 |
| clarkb | is the full exc_info | 17:05 |
| clarkb | ok I've fixed it | 17:06 |
| clarkb | it was a pathing issue | 17:06 |
| *** dtantsur is now known as dtantsur|afk | 17:06 | |
| clarkb | the fix is actually in pip itself | 17:07 |
| clarkb | which means I've got to sort out their PR process again :/ I'll work on that | 17:07 |
| fungi | you can actually just fork the repo on salsa.debian.org and then create a merge request to the original with your commit | 17:13 |
| fungi | it's (unfortunately) eerily similar to a github workflow | 17:13 |
| clarkb | fungi: the fix ahs to go to pip upstream | 17:14 |
| fungi | oh | 17:14 |
| clarkb | I'll get a PR up shortly and share it here :) | 17:14 |
| fungi | well then, it's eerily similar to a salsa (gitlab) workflow ;) | 17:14 |
| *** owalsh has quit IRC | 17:15 | |
| *** owalsh has joined #openstack-infra | 17:17 | |
| clarkb | https://github.com/pypa/pip/pull/6367 | 17:24 |
| clarkb | mnaser: odyssey4me zigo ^ fyi | 17:24 |
| clarkb | it is possible that debuntu may want to carry that patch in their packaging if upstream doesn't grab it quickly | 17:24 |
| clarkb | note the failing tests on that PR seem to be related to pytest failing to load a plugin | 17:27 |
| clarkb | I don't think it is related to my PR | 17:27 |
| corvus | yeah, i was about to say apparently all your.... rocketships? crashed..? | 17:28 |
| clarkb | the plugin it failed to load is rerunfailures plugin | 17:29 |
| corvus | you can fix that by pre-loading the "irony" plugin | 17:29 |
| clarkb | its a good thing I wasn't drinking tea just now | 17:29 |
| clarkb | it would be all over my monitor | 17:29 |
| clarkb | I'm checking master right now to see if this is "expected" | 17:31 |
| clarkb | yes master has the same issue | 17:31 |
| corvus | clarkb: https://github.com/pypa/pip/pull/6365 and https://github.com/pypa/pip/pull/6366 are relevant | 17:33 |
| clarkb | ah the plugin requires a newer version of pytest | 17:33 |
| clarkb | yup hopefully they get that fixed and we can merge other fixes like mine :) | 17:34 |
| corvus | clarkb: so all you need to do is just Depends-On: #6365 | 17:34 |
| clarkb | corvus: if only all CI worked like zuul | 17:34 |
| fungi | s/worked like/were replaced by/ | 17:34 |
| clarkb | in any case my PR should fix this should debuntu pull it in at some point | 17:34 |
| clarkb | and now I can feel like I did my good deed of the day fixing a 3 year old bug that both sides rejected as not their problem | 17:35 |
| *** jpena is now known as jpena|off | 17:37 | |
| AJaeger | config-core, there's quite a backlog of reviews, could you spend some time reviewing, please? | 17:43 |
| *** slaweq has joined #openstack-infra | 17:44 | |
| AJaeger | also, please review this infra-manual change https://review.openstack.org/#/c/630981/ | 17:44 |
| AJaeger | config-core, I'm conferencing next week - please don't depend on me ;) | 17:45 |
| *** jistr is now known as jistr|off | 17:45 | |
| *** jistr|off is now known as jistr | 17:45 | |
| *** psachin has quit IRC | 17:47 | |
| *** kjackal has joined #openstack-infra | 17:48 | |
| *** finucannot is now known as stephenfin | 17:53 | |
| *** yolanda has quit IRC | 17:53 | |
| clarkb | fungi: is there a tldr on the opensuse problem? | 17:54 |
| clarkb | AJaeger: have fun in tenesee | 17:54 |
| clarkb | (also I can't type or spell) | 17:54 |
| clarkb | AJaeger: and ya I expect today is mostly going to be a day of reviews for me. | 17:55 |
| *** wolverineav has quit IRC | 18:00 | |
| *** jamesmcarthur has quit IRC | 18:01 | |
| *** ramishra has quit IRC | 18:01 | |
| *** wolverineav has joined #openstack-infra | 18:02 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: WIP: Fix for image build leaks https://review.openstack.org/647599 | 18:02 |
| *** jamesmcarthur has joined #openstack-infra | 18:03 | |
| *** kjackal has quit IRC | 18:07 | |
| *** kjackal has joined #openstack-infra | 18:07 | |
| *** sshnaidm|off has quit IRC | 18:08 | |
| *** trown|lunch is now known as trown | 18:08 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Match tag items against containing branches https://review.openstack.org/578557 | 18:10 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Use implied branch matcher for implied branches https://review.openstack.org/640272 | 18:10 |
| fungi | clarkb: no real tl;dr because there were several problems | 18:10 |
| fungi | biggest was that the master suse package repositories were, at one point, published empty | 18:11 |
| fungi | so we faithfully mirrored that | 18:11 |
| fungi | and as a result the base install carried a version of unbound with outdated/expired root signing keys so was filtering out important dns responses | 18:12 |
| *** jamesmcarthur_ has joined #openstack-infra | 18:12 | |
| *** adriancz has quit IRC | 18:13 | |
| fungi | they got the packages back in place, but had some temp directories in the tumbleweed package trees which were not world-readable so rsync was trying to descend into them and failed | 18:13 |
| fungi | because all the suse mirrors share a volume, we weren't vos releasing the mirrors which were repaired | 18:13 |
| fungi | so we merged some (partly broken) temporary workarounds to ignore rsync errors for tumbleweed | 18:14 |
| *** jamesmcarthur has quit IRC | 18:14 | |
| fungi | once the mirrors were back on track, image builds for 42.3 were failing (not sure why, i lacked sufficient time to dig into the logs) but eventually completed later on | 18:15 |
| *** iurygregory has joined #openstack-infra | 18:17 | |
| openstackgerrit | James E. Blair proposed openstack-infra/project-config master: Complete zuul-sphinx removal from openstack tenant https://review.openstack.org/648752 | 18:18 |
| *** jamesmcarthur_ has quit IRC | 18:20 | |
| *** jamesmcarthur has joined #openstack-infra | 18:21 | |
| *** sshnaidm|off has joined #openstack-infra | 18:21 | |
| *** jamesmcarthur has quit IRC | 18:26 | |
| *** electrofelix has quit IRC | 18:29 | |
| *** roman_g has quit IRC | 18:34 | |
| clarkb | fungi: opensuse is expected to work at this point though? (maybe tumbleweed is still broken?) | 18:34 |
| *** slaweq has quit IRC | 18:36 | |
| fungi | i think they all should work | 18:38 |
| *** pcaruana has quit IRC | 18:38 | |
| fungi | tumbleweed is probably just lacking some unreadable temporary directories which yast or whatever they use these days won't be looking for anyway | 18:39 |
| clarkb | zypper and yast2 :P | 18:39 |
| * fungi shows his age apparently | 18:39 | |
| clarkb | I just did a big zypper update since my computer was off while out of town. I should reboot for that at some point | 18:40 |
| *** kmalloc is now known as needscoffee | 18:41 | |
| fungi | oh, and not long ago i approved the change to turn the opensuse jobs back on for openstack-zuul-jobs | 18:42 |
| clarkb | thanks! | 18:43 |
| fungi | if you want a simple but exciting review to ease your way back off the island, check out 648752 | 18:45 |
| *** raukadah has quit IRC | 18:45 | |
| clarkb | will do. I'm almost through the two devstack stacks I was asked to review | 18:47 |
| *** wolverineav has quit IRC | 18:47 | |
| *** wolverineav has joined #openstack-infra | 18:49 | |
| clarkb | fungi: is that the first repo to use the new zuul tenant? | 18:50 |
| clarkb | the new zuul "zuul" tenant :) | 18:50 |
| *** wolverineav has quit IRC | 18:50 | |
| *** wolverineav has joined #openstack-infra | 18:53 | |
| *** tbachman has joined #openstack-infra | 18:57 | |
| fungi | sort of | 18:57 |
| fungi | depends on what you mean "in" | 18:57 |
| fungi | http://zuul.opendev.org/t/zuul/projects | 18:57 |
| fungi | it will be the first repo *only* in the "zuul" zuul tenant once that change merges | 18:58 |
| openstackgerrit | Merged openstack-infra/infra-manual master: Improve the details about settings i18n for new repos https://review.openstack.org/630981 | 18:58 |
| fungi | there are some config and job projects which are in "zuul" but also in other tenants | 18:58 |
| *** wolverineav has quit IRC | 18:58 | |
| openstackgerrit | Merged openstack-infra/project-config master: Complete zuul-sphinx removal from openstack tenant https://review.openstack.org/648752 | 19:00 |
| corvus | i think the next steps there are to add tag docs jobs (zuul-sphinx never had them, but other projects do, so let's just add them to zuul-sphinx to work out the issues), then pypi publication | 19:09 |
| corvus | i think we can rework most of the pypi jobs to be in zuul-jobs now. dhellmann wanted to keep the playbooks together, and since the secrets required that some of them be in project-config, all of them are. however, the new pass-to-parent feature should let us keep the secrets in p-c while the playbooks are in zuul-jobs. | 19:11 |
| corvus | so i'll take a stab at making zuul-jobs / opendev versions of those and using those for the zuul tenant. if that works, we can backport to openstack. it should make them easier to maintain. | 19:12 |
| corvus | of course, we can backport to openstack *after* the release. :) | 19:12 |
| *** fried_rolls is now known as fried_rice | 19:14 | |
| *** gmann_afk is now known as gmann | 19:19 | |
| AJaeger | clarkb: thanks, will do my best ;) | 19:19 |
| *** dpawlik has joined #openstack-infra | 19:20 | |
| AJaeger | corvus: could you look at the errors in https://review.openstack.org/643525 - -2 due to " The project "openstack-infra/zuul-sphinx" was not found." | 19:21 |
| AJaeger | , please? | 19:21 |
| AJaeger | so, right now openstack-infra/project-config is broken ;( | 19:21 |
| AJaeger | ah, I see you merged something already, corvus ... thanks! | 19:22 |
| corvus | AJaeger: it should be fixed as of 19:00, i rechecked that change | 19:22 |
| AJaeger | corvus: yeah, found it in backscroll now after more carefull reading , sorry | 19:23 |
| *** sthussey has joined #openstack-infra | 19:33 | |
| *** tbachman has quit IRC | 19:34 | |
| fungi | have the electrician working on some switches on the same circuit as my cable modem, so may be offline for a little while | 19:37 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove unneeded storyboard-webclient post job https://review.openstack.org/643525 | 19:41 |
| openstackgerrit | Merged openstack-infra/project-config master: Set variables for storyboard-webclient release job https://review.openstack.org/643545 | 19:41 |
| *** bhavikdbavishi has quit IRC | 19:42 | |
| *** tosky has joined #openstack-infra | 19:48 | |
| *** tbachman has joined #openstack-infra | 19:54 | |
| *** wolverineav has joined #openstack-infra | 19:57 | |
| *** wolverineav has quit IRC | 19:57 | |
| *** wolverineav has joined #openstack-infra | 19:57 | |
| *** dpawlik has quit IRC | 19:57 | |
| *** slaweq has joined #openstack-infra | 19:58 | |
| *** yolanda has joined #openstack-infra | 19:58 | |
| *** wolverineav has quit IRC | 20:01 | |
| *** wolverineav has joined #openstack-infra | 20:02 | |
| *** kgiusti has quit IRC | 20:04 | |
| *** slaweq has quit IRC | 20:05 | |
| *** wolverineav has quit IRC | 20:07 | |
| *** dpawlik has joined #openstack-infra | 20:09 | |
| *** slaweq has joined #openstack-infra | 20:11 | |
| *** dpawlik has quit IRC | 20:14 | |
| *** slaweq has quit IRC | 20:16 | |
| *** wolverineav has joined #openstack-infra | 20:17 | |
| *** jtomasek has joined #openstack-infra | 20:17 | |
| *** owalsh has quit IRC | 20:25 | |
| *** wolverineav has quit IRC | 20:26 | |
| *** jtomasek has quit IRC | 20:26 | |
| *** wolverineav has joined #openstack-infra | 20:27 | |
| clarkb | corvus: I unWIPed https://review.openstack.org/#/c/644428/1 if you want to give that a review now (handles the lack of metadata in zuul return) | 20:30 |
| *** wolverineav has quit IRC | 20:32 | |
| *** wolverineav has joined #openstack-infra | 20:32 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Allow user to set ara_report_executable https://review.openstack.org/648773 | 20:32 |
| *** wolverineav has quit IRC | 20:34 | |
| openstackgerrit | James E. Blair proposed opendev/base-jobs master: Add opendev-publish-tox-docs https://review.openstack.org/648775 | 20:34 |
| *** wolverineav has joined #openstack-infra | 20:35 | |
| corvus | clarkb: thanks! we're likely to start running into that soon (opendev and zuul-sphinx are now running docs jobs which produce artifacts with no metadata) | 20:35 |
| *** raissa has joined #openstack-infra | 20:37 | |
| *** mrhillsman is now known as mrhillsman_away4 | 20:40 | |
| openstackgerrit | James E. Blair proposed opendev/base-jobs master: Add opendev-publish-tox-docs https://review.openstack.org/648775 | 20:41 |
| *** mrhillsman_away4 is now known as mrhillsman_wkend | 20:41 | |
| openstackgerrit | James E. Blair proposed opendev/base-jobs master: Add opendev-publish-tox-docs https://review.openstack.org/648775 | 20:43 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Handle no metadata in zuul artifact items https://review.openstack.org/644428 | 20:43 |
| *** jamesmcarthur has joined #openstack-infra | 20:45 | |
| *** iurygregory has quit IRC | 20:45 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-sphinx master: Publish docs on release https://review.openstack.org/648779 | 20:57 |
| *** jcoufal has quit IRC | 20:58 | |
| *** trown is now known as trown|outtypewww | 21:00 | |
| openstackgerrit | Kendall Nelson proposed openstack-infra/irc-meetings master: First Contact SIG update https://review.openstack.org/648782 | 21:05 |
| *** owalsh has joined #openstack-infra | 21:06 | |
| *** wolverineav has quit IRC | 21:08 | |
| *** wolverineav has joined #openstack-infra | 21:10 | |
| *** slaweq has joined #openstack-infra | 21:11 | |
| *** wolverineav has quit IRC | 21:12 | |
| *** wolverineav has joined #openstack-infra | 21:15 | |
| dmsimard | Should the post pipeline be sending comments back to gerrit on status ? https://opendev.org/openstack-infra/project-config/src/branch/master/zuul.d/pipelines.yaml#L112 | 21:15 |
| *** slaweq has quit IRC | 21:15 | |
| dmsimard | For context, https://review.openstack.org/#/c/648445/ has triggered a post job http://zuul.openstack.org/builds?job_name=ara-upload-git-mirror | 21:16 |
| clarkb | prometheanfire: tonyb: requirements type question. If neutron imports code added in neutron-lib 1.25.0 (code doesn't exist in earlier releases) does requirements global-requirements need to set the req to >=1.25.0 ? | 21:16 |
| dmsimard | the job failed but I'm in the process of troubleshooting it | 21:16 |
| clarkb | dmsimard: no post jobs are not sent back to gerrit as they don't directly map to a change (could be tags etc) | 21:17 |
| dmsimard | anyway, Zuul didn't comment on gerrit about the failure | 21:17 |
| dmsimard | clarkb: what if there is a change, though ? | 21:17 |
| clarkb | dmsimard: the event type isn't a change which is the issue | 21:17 |
| dmsimard | it's from the merge ? | 21:18 |
| clarkb | dmsimard: the promote pipeline which does use change merged events could comment back (but I'm not sure it does) | 21:18 |
| clarkb | dmsimard: ya but the gerrit event is "this ref updated" there is no change info | 21:18 |
| clarkb | and the ref may not be tied to a change | 21:18 |
| clarkb | its a weird spot for zuul to try and infer things | 21:18 |
| dmsimard | hmmm | 21:18 |
| dmsimard | zuul knows if it's going to be merging something, it knows the ref and could trigger post from there instead of the event that comes afterwards ? | 21:19 |
| * dmsimard hand waving | 21:19 | |
| clarkb | dmsimard: if zuul was updated to do the merges then yes (its been on the raod map for a while but still not done yet) | 21:19 |
| *** owalsh_ has joined #openstack-infra | 21:19 | |
| dmsimard | oh so it's actually gerrit that merges ? | 21:19 |
| clarkb | yes | 21:20 |
| clarkb | zuul pushes the gerrit "submit" button | 21:20 |
| clarkb | (same thing with github driver too fwiw) | 21:20 |
| *** wolverineav has quit IRC | 21:20 | |
| dmsimard | the comment Zuul leaves is "Change has been successfully merged by Zuul" haha | 21:21 |
| clarkb | gerrit creates that comment | 21:21 |
| dmsimard | anyway, it makes sense | 21:21 |
| clarkb | if I clicked the button it would say Change has been successfully merged by Clark Boylan | 21:21 |
| *** owalsh has quit IRC | 21:21 | |
| dmsimard | so when it clicks the button | 21:21 |
| dmsimard | it knows which change it's clicking the button on | 21:21 |
| clarkb | yes, but not what the resulting ref will be | 21:22 |
| dmsimard | because the merge isn't done yet | 21:22 |
| dmsimard | I get it now | 21:22 |
| dmsimard | :D | 21:22 |
| clarkb | (because timestamp is part of the sha1 calculation) | 21:22 |
| clarkb | in the fast forward case it wouldn't change, but in the merge case there are things to lookup | 21:22 |
| clarkb | but as you say if zuul started to do the merges this could potentially change | 21:23 |
| *** owalsh has joined #openstack-infra | 21:23 | |
| clarkb | and that has been planned for a while, just needs to be done | 21:23 |
| dmsimard | so then no more button clicking ? | 21:23 |
| clarkb | yup, zuul would merge locally then (force?) push into the git repo | 21:24 |
| *** owalsh_ has quit IRC | 21:24 | |
| mordred | clarkb: I'm marvelling at your pip PR | 21:26 |
| clarkb | mordred: its a good one | 21:28 |
| pabelanger | dmsimard: clarkb: promote pipeline will report back to gerrit on success / failure, am using that today on github also | 21:29 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Replace openstack.org git:// URLs with https:// https://review.openstack.org/646855 | 21:30 |
| *** wolverineav has joined #openstack-infra | 21:31 | |
| *** jamesmcarthur has quit IRC | 21:31 | |
| *** jamesmcarthur has joined #openstack-infra | 21:32 | |
| dmsimard | anyway, what I was trying to troubleshoot is why it seems my secret isn't making it to my job.. I'm a bit confused | 21:33 |
| dmsimard | but I think it would be because the secret can't be used until after the merge and yet post runs after so the secret wasn't available yet ? it's seeking in an untrusted path http://logs.openstack.org/4f/4f96908789b93d1bfc00047e2e3b39d85ab76e3d/post/ara-upload-git-mirror/f8d0260/ara-report/result/0ffa8903-4dc2-4bd7-834b-0514b07899b8/ | 21:33 |
| pabelanger | where is git_mirror_credentials defined? | 21:39 |
| *** josephrsandoval has joined #openstack-infra | 21:39 | |
| pabelanger | post pipline, should have access to a secret | 21:40 |
| dmsimard | pabelanger: https://opendev.org/openstack/ara/commit/4f96908789b93d1bfc00047e2e3b39d85ab76e3d | 21:40 |
| dmsimard | maybe I have the names and stuff mixed up | 21:40 |
| dmsimard | I'll re-read docs :D | 21:40 |
| zigo | clarkb: I'll try to get this patch into Debian ASAP, before Buster is frozen. | 21:42 |
| zigo | Thanks for it. | 21:42 |
| zigo | clarkb: We can try to get it into Ubuntu Bionic as well, maybe. | 21:42 |
| zigo | jamespage: coreycb: ^ | 21:42 |
| zigo | The patch is: https://github.com/pypa/pip/pull/6367/commits/f8292a304deebcf0e4cda2e40caa226c70030f11 | 21:42 |
| *** jamesmcarthur has quit IRC | 21:43 | |
| *** jamesmcarthur has joined #openstack-infra | 21:43 | |
| pabelanger | dmsimard: I think you have a syntax issue, I believe you want: http://paste.openstack.org/show/748610/ | 21:43 |
| pabelanger | https://zuul-ci.org/docs/zuul/user/config.html#attr-job.secrets | 21:44 |
| dmsimard | oh I had it backwards ? | 21:44 |
| pabelanger | dmsimard: no, secrets is a list | 21:44 |
| dmsimard | oh that makes more sense | 21:44 |
| dmsimard | interesting that it didn't trip zuul syntax error | 21:45 |
| dmsimard | I didn't provide a list | 21:45 |
| pabelanger | yah, not sure why | 21:45 |
| *** owalsh has quit IRC | 21:45 | |
| dmsimard | pabelanger: I'll merge a fix, thanks :D | 21:48 |
| *** yamamoto has joined #openstack-infra | 21:49 | |
| *** jamesmcarthur has quit IRC | 21:49 | |
| *** jamesmcarthur has joined #openstack-infra | 21:50 | |
| *** josephrsandoval has quit IRC | 21:50 | |
| mnaser | anyone ever seen something where wheels don't get cached so pip installs in the gate are always pulling down requirements every single time? | 21:50 |
| mnaser | I have a job here that's taking quite sometime and im pretty sure it would be much faster if it just cached them instead of redownloading them | 21:51 |
| *** yamamoto has quit IRC | 21:53 | |
| pabelanger | mnaser: possible that our wheels mirrors don't have said wheel? | 21:54 |
| pabelanger | however, I think we now proxy pypi, don't we? | 21:54 |
| corvus | dmsimard, pabelanger: 'secrets' is, like many zuul job attrs, a singleton-or-list. | 21:54 |
| tbachman | I have an infra-related question — this gerrit: https://review.openstack.org/#/c/647475/ passed on the 25th, but then when we went to merge yesterday, it’s as though the artifacts themselves weren’t there. Did something significant change recently wrt the way buidls are being done the last couple of days? | 21:55 |
| pabelanger | ah, cool. didn't realize that | 21:55 |
| tbachman | the closest thing I could find was a change from git to https, but my first attempt to change those in our repos didn’t seem to have any effect | 21:56 |
| * tbachman will accept any response, including “go away you infra n00b” | 21:56 | |
| corvus | tbachman: this looks like an error: http://logs.openstack.org/75/647475/1/check/openstack-tox-pep8/6c806c4/job-output.txt.gz#_2019-03-29_00_17_47_983080 | 21:57 |
| tbachman | right - but what’s weird is we have no idea why that would have disappeared. Its as though the packages themselves aren’t present | 21:57 |
| tbachman | which is what led me to the git vs. https | 21:58 |
| tbachman | our ocata/pike builds were okay - it was only starting with queens | 21:58 |
| pabelanger | dmsimard: Oh, I think you need to set pass-to-parent: true, right? because it is the parent that is using the secret, not the job in ara | 21:59 |
| mnaser | pabelanger: the logs show the wheel being downloaded only enough | 22:00 |
| pabelanger | mnaser: have a log I can look at? | 22:00 |
| mnaser | pabelanger: http://logs.openstack.org/10/648710/1/check/openstack-ansible-deploy-aio_metal-ubuntu-bionic/74f0461/logs/ara-report/ if you open tasks under setup-openstack.yml and sort by duration, you'll see a lot of long "python_venv_build : Install python packages into the venv" | 22:01 |
| mnaser | when you click on it to view the in detail log, you see it's downloading it | 22:01 |
| mnaser | now if you click on a bunch you'll notice for an example PyMySQL is always being downloaded | 22:01 |
| *** rlandy has quit IRC | 22:02 | |
| pabelanger | mnaser: Hmm, this doesn't look right | 22:03 |
| pabelanger | The repository located at mirror.gra1.ovh.openstack.org is not a trusted or secure host and is being ignored. If this repository is available via HTTPS we recommend you use HTTPS instead, otherwise you may silence this warning and allow it anyway with '--trusted-host mirror.gra1.ovh.openstack.org' | 22:03 |
| mnaser | pabelanger: oh that's interesting, I wonder if our usage of --trusted-host is overriding the pip.conf | 22:06 |
| mnaser | http://logs.openstack.org/10/648710/1/check/openstack-ansible-deploy-aio_metal-ubuntu-bionic/74f0461/logs/etc/host/pip.conf.txt.gz | 22:07 |
| mnaser | I mean it's there. | 22:07 |
| dmsimard | pabelanger: ah, it might be that too. I wanted to try using it without because I was paranoid about the wording used | 22:07 |
| pabelanger | mnaser: not sure, maybe clarkb knows more | 22:10 |
| *** slaweq has joined #openstack-infra | 22:11 | |
| *** owalsh has joined #openstack-infra | 22:15 | |
| *** slaweq has quit IRC | 22:16 | |
| prometheanfire | clarkb: neutron itself needs to update it's minimum in requirements.txt | 22:19 |
| *** jamesmcarthur has quit IRC | 22:19 | |
| prometheanfire | clarkb: global-reqs in the reqs project doesn't have minimums | 22:20 |
| clarkb | prometheanfire: thanks | 22:21 |
| clarkb | trustedhosts should mean its ok to do http instead of https to that host | 22:21 |
| *** raissa has quit IRC | 22:24 | |
| *** ijw has joined #openstack-infra | 22:31 | |
| *** ijw has left #openstack-infra | 22:31 | |
| *** ijw has joined #openstack-infra | 22:31 | |
| ijw | #join openstack-infra | 22:32 |
| ijw | grr | 22:32 |
| *** ijw has left #openstack-infra | 22:32 | |
| *** ijw has joined #openstack-infra | 22:32 | |
| *** rkukura_ has joined #openstack-infra | 22:36 | |
| ijw | Apologies for asking, but can someone +A https://review.openstack.org/#/c/648553/ ? It removes a legacy check job for networking-vpp that has a fault in it anyway, and I was planning to move it over to .zuul.yaml as I fix it, but the sooner it's gone the less time it wastes. | 22:36 |
| *** rkukura has quit IRC | 22:38 | |
| *** rkukura_ is now known as rkukura | 22:38 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient master: Stars should be next to the name of the entity https://review.openstack.org/643153 | 22:41 |
| clarkb | ijw: done, is there a change to openstack-infra/openstack-zuul-jobs to remove the job definition from that repo? | 22:44 |
| *** wolverineav has quit IRC | 22:51 | |
| rpioso | diablo_rojo: Thanks for the email about the final PTG schedule. It appears the Baremetal SIG will be dynamically scheduled during the PTG. Is my understanding correct? Is holding it during the Forum still being considered? My present travel arrangements are for attending only the PTG. | 22:53 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy-networking-vpp-smoke https://review.openstack.org/648553 | 22:55 |
| diablo_rojo | rpioso, I am not sure what they have planned for the Forum if anything. You would want to talk to hogepodge about that. | 22:57 |
| hogepodge | I was hoping we would get a space, but I think we were too late. | 22:58 |
| hogepodge | rpioso: but you can plan on us doing something during the PTG | 22:58 |
| rpioso | diablo_rojo, hogepodge: Thank you! Hope you have an enjoyable weekend :) | 22:59 |
| *** gfidente|afk is now known as gfidente | 22:59 | |
| *** wolverineav has joined #openstack-infra | 23:01 | |
| *** sthussey has quit IRC | 23:01 | |
| openstackgerrit | Merged openstack-infra/storyboard-webclient master: Don't create a tarball in the post job https://review.openstack.org/647706 | 23:02 |
| ijw | clarkb: https://review.openstack.org/#/c/648549/ (just kicked its recheck) | 23:03 |
| *** aaronsheffield has quit IRC | 23:03 | |
| clarkb | ijw: looks like it needs a rebase? | 23:04 |
| ijw | Will deal, give me 5 | 23:04 |
| clarkb | no rush. And thanks | 23:04 |
| ijw | I saw yesterday someone was rewriting URLs all over, so it's a fair bet that caught it | 23:04 |
| diablo_rojo | rpioso, you too! | 23:05 |
| clarkb | ya we are trying to move away from the git protocol urls in favor of http(s) so that we can switch to gitea as our git server | 23:05 |
| openstackgerrit | Ian Wells proposed openstack-infra/openstack-zuul-jobs master: Remove legacy-networking-vpp-smoke https://review.openstack.org/648549 | 23:07 |
| ijw | No, I saw and it entirely makes sense; it was just my luck to pick the one moment when a broad-brush change came along (in a file I was deleting, no less). Anyway, merge conflict resolved and Zuul is thinking | 23:10 |
| clarkb | +2 thanks again | 23:13 |
| *** gfidente has quit IRC | 23:13 | |
| *** diablo_rojo has quit IRC | 23:14 | |
| *** rh-jelabarre has quit IRC | 23:34 | |
| *** wolverineav has quit IRC | 23:45 | |
| *** whoami-rajat has quit IRC | 23:47 | |
| *** dklyle has quit IRC | 23:48 | |
| *** david-lyle has joined #openstack-infra | 23:48 | |
| *** tbachman has quit IRC | 23:50 | |
| *** needscoffee is now known as kmalloc | 23:53 | |
| tonyb | clarkb: neutron need to set it's minimum in it's requirements to >=1.25.0 and the matching lower-constraint | 23:53 |
| clarkb | tonyb: yup it looks like they've done that. Reading requirements more some entries do have lower bounds is that just legacy info? | 23:54 |
| tonyb | clarkb: We shoudln't be tracking lowers globally anymore | 23:55 |
| tonyb | clarkb: I'll lookinto it on Monday if you leave me a pointer | 23:55 |
| clarkb | oh you are right I think I read a != as a >= | 23:55 |
| tonyb | ahh cool | 23:56 |
| tonyb | [tony@thor requirements]$ git grep '>=' global-requirements.txt | 23:56 |
| tonyb | global-requirements.txt:Django<2.1;python_version>='3.0' # BSD | 23:56 |
| tonyb | global-requirements.txt:dnspython3!=1.13.0,!=1.14.0;python_version>='3.0' # http://www.dnspython.org/LICENSE | 23:56 |
| tonyb | global-requirements.txt:mypy;python_version>='3.4' # MIT | 23:56 |
| tonyb | Oh no that's in the marker my bad | 23:56 |
| tonyb | nothign to see here | 23:56 |
| * tonyb goes to the park | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!