| pabelanger | Yah, I never knew about cmd until last week or so. Not even documented in docs | 00:00 |
|---|---|---|
| fungi | clarkb: jeblair: so we should convert whatever existing "shell" tasks we find to "cmd" tasks? | 00:01 |
| clarkb | fungi: no, it is still a shell task but you use the cmd: key to specify the script | 00:01 |
| clarkb | fungi: check out taht example I linked earlier in nodepool | 00:01 |
| *** wolverineav has joined #openstack-infra | 00:01 | |
| pabelanger | ya | 00:01 |
| pabelanger | command is likey the task you are thinking off | 00:02 |
| pabelanger | of* | 00:02 |
| fungi | right, so there's a command task and a cmd arg to the shell task? | 00:02 |
| clarkb | yes | 00:02 |
| * fungi just wants to make sure tilde expansion works normally in the tag-releases job | 00:03 | |
| *** andreas_s has joined #openstack-infra | 00:03 | |
| fungi | whatever the right solution is, happy to use it | 00:03 |
| openstackgerrit | Merged openstack-infra/project-config master: Use bash for the release/tag playbook https://review.openstack.org/514491 | 00:03 |
| *** rtjure has joined #openstack-infra | 00:03 | |
| fungi | though i guess that change ended up merging as-is anyway | 00:03 |
| *** ijw has joined #openstack-infra | 00:04 | |
| *** ijw has quit IRC | 00:04 | |
| fungi | we already had a role using the executable arg in project-config, so i can just fix them both in a followup | 00:04 |
| *** ijw has joined #openstack-infra | 00:04 | |
| *** wolverineav has quit IRC | 00:05 | |
| *** gyee has quit IRC | 00:06 | |
| *** rtjure has quit IRC | 00:08 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config master: Use shell.cmd with shell.executable https://review.openstack.org/514492 | 00:10 |
| fungi | clarkb: jeblair: pabelanger: like that ^ ? | 00:11 |
| *** dingyichen has joined #openstack-infra | 00:12 | |
| *** andreas_s has quit IRC | 00:12 | |
| clarkb | ya | 00:12 |
| *** rtjure has joined #openstack-infra | 00:13 | |
| *** rwsu has joined #openstack-infra | 00:13 | |
| *** verdurin has quit IRC | 00:13 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: WIP: experiment with late-binding inheritance https://review.openstack.org/511352 | 00:13 |
| *** rlandy is now known as rlandy|bbl | 00:15 | |
| *** rtjure has quit IRC | 00:17 | |
| *** xarses has quit IRC | 00:19 | |
| *** verdurin has joined #openstack-infra | 00:21 | |
| *** rtjure has joined #openstack-infra | 00:22 | |
| *** bobh has quit IRC | 00:23 | |
| *** slaweq has quit IRC | 00:24 | |
| *** rtjure has quit IRC | 00:27 | |
| openstackgerrit | Merged openstack-infra/project-config master: Use shell.cmd with shell.executable https://review.openstack.org/514492 | 00:28 |
| *** andreas_s has joined #openstack-infra | 00:30 | |
| *** slaweq has joined #openstack-infra | 00:31 | |
| *** aeng has quit IRC | 00:31 | |
| *** rtjure has joined #openstack-infra | 00:33 | |
| *** pahuang has quit IRC | 00:34 | |
| *** sree has joined #openstack-infra | 00:37 | |
| *** rtjure has quit IRC | 00:38 | |
| *** andreas_s has quit IRC | 00:39 | |
| fungi | okay, gotten as far as i can testing release job fixes tonight, starting to work on executor restarts now | 00:41 |
| *** sree has quit IRC | 00:42 | |
| *** rtjure has joined #openstack-infra | 00:43 | |
| *** thorst has joined #openstack-infra | 00:43 | |
| *** huanxie has joined #openstack-infra | 00:44 | |
| *** thorst has quit IRC | 00:44 | |
| *** aeng has joined #openstack-infra | 00:44 | |
| *** rtjure has quit IRC | 00:47 | |
| *** pahuang has joined #openstack-infra | 00:51 | |
| *** rtjure has joined #openstack-infra | 00:53 | |
| *** camunoz has quit IRC | 00:57 | |
| *** LindaWang has joined #openstack-infra | 00:58 | |
| *** rtjure has quit IRC | 00:58 | |
| *** huanxie has quit IRC | 01:01 | |
| *** kiennt26 has joined #openstack-infra | 01:02 | |
| *** dave-mccowan has joined #openstack-infra | 01:02 | |
| *** slaweq has quit IRC | 01:02 | |
| *** slaweq has joined #openstack-infra | 01:06 | |
| *** sbezverk has quit IRC | 01:06 | |
| *** rtjure has joined #openstack-infra | 01:07 | |
| *** sbezverk has joined #openstack-infra | 01:07 | |
| *** pahuang has quit IRC | 01:09 | |
| *** aeng has quit IRC | 01:09 | |
| *** aeng has joined #openstack-infra | 01:10 | |
| *** rtjure has quit IRC | 01:12 | |
| *** mriedem_away has quit IRC | 01:13 | |
| *** daidv has joined #openstack-infra | 01:16 | |
| openstackgerrit | Merged openstack-infra/project-config master: Use write-root-marker role https://review.openstack.org/513942 | 01:16 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy jobs in Tooz https://review.openstack.org/513297 | 01:16 |
| *** cuongnv has joined #openstack-infra | 01:17 | |
| *** rtjure has joined #openstack-infra | 01:18 | |
| *** fanzhang has quit IRC | 01:18 | |
| *** andreas_s has joined #openstack-infra | 01:20 | |
| *** rtjure has quit IRC | 01:22 | |
| *** pahuang has joined #openstack-infra | 01:22 | |
| *** fanzhang has joined #openstack-infra | 01:23 | |
| openstackgerrit | Hirofumi Ichihara proposed openstack-infra/project-config master: neutron-dynamic-routing: Use releasenote and publish job template https://review.openstack.org/514495 | 01:23 |
| *** andreas_s has quit IRC | 01:25 | |
| *** bobh has joined #openstack-infra | 01:25 | |
| *** Apoorva_ has joined #openstack-infra | 01:27 | |
| *** rtjure has joined #openstack-infra | 01:27 | |
| *** bobh has quit IRC | 01:28 | |
| *** Apoorva has quit IRC | 01:30 | |
| *** namnh has joined #openstack-infra | 01:31 | |
| *** Apoorva_ has quit IRC | 01:32 | |
| *** rtjure has quit IRC | 01:32 | |
| *** thorst has joined #openstack-infra | 01:33 | |
| *** ihrachys has quit IRC | 01:34 | |
| *** pramodrj07 has quit IRC | 01:34 | |
| *** MasterOfBugs has quit IRC | 01:34 | |
| *** salv-orlando has joined #openstack-infra | 01:36 | |
| *** rtjure has joined #openstack-infra | 01:37 | |
| openstackgerrit | Hiroaki Kobayashi proposed openstack-infra/project-config master: Add documentation publish jobs to Blazar https://review.openstack.org/511176 | 01:38 |
| *** slaweq has quit IRC | 01:38 | |
| *** thorst has quit IRC | 01:39 | |
| smcginnis | fungi: Just read scrollback. Thanks for figuring all that out. | 01:39 |
| *** salv-orlando has quit IRC | 01:40 | |
| *** Qiming has quit IRC | 01:41 | |
| *** liujiong has joined #openstack-infra | 01:41 | |
| *** rtjure has quit IRC | 01:42 | |
| *** markvoelker has joined #openstack-infra | 01:43 | |
| *** slaweq has joined #openstack-infra | 01:45 | |
| fungi | #status log all zuul executors have been restarted to pick up the latest bubblewrap bindmount addition | 01:45 |
| openstackstatus | fungi: finished logging | 01:45 |
| fungi | smcginnis: you bet. hoping this is the last nail in that coffin | 01:46 |
| smcginnis | fungi: I think I'll send one more through to see how it goes. Unless you think we should take the time to set up a test release. | 01:47 |
| *** rtjure has joined #openstack-infra | 01:47 | |
| fungi | your call, but lmk how things go | 01:47 |
| smcginnis | Will do. Thanks again. | 01:47 |
| fungi | the test release was more so that we could confirm whether lp comments are working | 01:47 |
| *** andreas_s has joined #openstack-infra | 01:48 | |
| *** slaweq has quit IRC | 01:49 | |
| *** rtjure has quit IRC | 01:51 | |
| *** andreas_s has quit IRC | 01:52 | |
| *** hongbin has joined #openstack-infra | 01:56 | |
| *** psachin has joined #openstack-infra | 01:58 | |
| *** jamesmcarthur has joined #openstack-infra | 01:58 | |
| *** rtjure has joined #openstack-infra | 01:58 | |
| *** Qiming has joined #openstack-infra | 02:01 | |
| *** liujiong has quit IRC | 02:01 | |
| *** annp has joined #openstack-infra | 02:02 | |
| *** liujiong has joined #openstack-infra | 02:03 | |
| *** rtjure has quit IRC | 02:03 | |
| *** dhinesh has quit IRC | 02:03 | |
| *** andreas_s has joined #openstack-infra | 02:05 | |
| *** rlandy|bbl is now known as rlandy | 02:06 | |
| *** rtjure has joined #openstack-infra | 02:08 | |
| *** wolverineav has joined #openstack-infra | 02:12 | |
| *** rtjure has quit IRC | 02:13 | |
| *** andreas_s has quit IRC | 02:14 | |
| *** wolverineav has quit IRC | 02:17 | |
| *** Qiming has quit IRC | 02:19 | |
| *** rtjure has joined #openstack-infra | 02:20 | |
| *** slaweq has joined #openstack-infra | 02:22 | |
| *** rtjure has quit IRC | 02:25 | |
| *** verdurin has quit IRC | 02:26 | |
| *** andreas_s has joined #openstack-infra | 02:28 | |
| *** rtjure has joined #openstack-infra | 02:29 | |
| *** gouthamr has quit IRC | 02:33 | |
| *** rtjure has quit IRC | 02:35 | |
| *** jamesmcarthur has quit IRC | 02:35 | |
| *** andreas_s has quit IRC | 02:37 | |
| *** jamesmcarthur has joined #openstack-infra | 02:37 | |
| *** kzaitsev_pi has quit IRC | 02:39 | |
| *** verdurin has joined #openstack-infra | 02:40 | |
| *** rtjure has joined #openstack-infra | 02:41 | |
| *** yamahata has quit IRC | 02:41 | |
| *** calebb has quit IRC | 02:41 | |
| *** iyamahat has quit IRC | 02:42 | |
| *** jamesmcarthur has quit IRC | 02:44 | |
| *** rtjure has quit IRC | 02:46 | |
| *** kzaitsev_pi has joined #openstack-infra | 02:46 | |
| *** wolverineav has joined #openstack-infra | 02:49 | |
| *** rtjure has joined #openstack-infra | 02:51 | |
| *** calebb has joined #openstack-infra | 02:51 | |
| *** andreas_s has joined #openstack-infra | 02:51 | |
| *** wolverineav has quit IRC | 02:53 | |
| *** rtjure has quit IRC | 02:55 | |
| *** ijw has quit IRC | 02:55 | |
| *** slaweq has quit IRC | 02:56 | |
| *** slaweq has joined #openstack-infra | 03:03 | |
| *** rtjure has joined #openstack-infra | 03:04 | |
| *** andreas_s has quit IRC | 03:04 | |
| *** jamesmcarthur has joined #openstack-infra | 03:05 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 03:09 |
| *** rtjure has quit IRC | 03:09 | |
| *** ijw has joined #openstack-infra | 03:12 | |
| *** witek has quit IRC | 03:14 | |
| *** rtjure has joined #openstack-infra | 03:14 | |
| *** witek has joined #openstack-infra | 03:15 | |
| *** ijw has quit IRC | 03:19 | |
| *** rtjure has quit IRC | 03:19 | |
| *** wolverineav has joined #openstack-infra | 03:23 | |
| *** andreas_s has joined #openstack-infra | 03:23 | |
| *** rlandy has quit IRC | 03:24 | |
| *** andreas_s has quit IRC | 03:27 | |
| *** pahuang has quit IRC | 03:28 | |
| *** wolverineav has quit IRC | 03:28 | |
| *** dhinesh has joined #openstack-infra | 03:29 | |
| *** Qiming has joined #openstack-infra | 03:29 | |
| *** rtjure has joined #openstack-infra | 03:30 | |
| *** jamesmcarthur has quit IRC | 03:31 | |
| *** jamesmcarthur has joined #openstack-infra | 03:31 | |
| *** armaan has quit IRC | 03:33 | |
| *** Qiming has quit IRC | 03:33 | |
| *** rtjure has quit IRC | 03:35 | |
| *** slaweq has quit IRC | 03:35 | |
| *** jamesmcarthur has quit IRC | 03:36 | |
| *** salv-orlando has joined #openstack-infra | 03:37 | |
| *** rtjure has joined #openstack-infra | 03:39 | |
| *** mat128 has quit IRC | 03:39 | |
| openstackgerrit | Tovin Seven proposed openstack-infra/project-config master: Remove legacy pyeclib job https://review.openstack.org/514512 | 03:40 |
| openstackgerrit | Tovin Seven proposed openstack-infra/openstack-zuul-jobs master: Remove legacy pyeclib job https://review.openstack.org/514513 | 03:41 |
| *** salv-orlando has quit IRC | 03:42 | |
| *** slaweq has joined #openstack-infra | 03:43 | |
| *** rtjure has quit IRC | 03:44 | |
| *** hongbin has quit IRC | 03:47 | |
| *** dave-mccowan has quit IRC | 03:47 | |
| *** ijw has joined #openstack-infra | 03:48 | |
| *** liujiong has quit IRC | 03:49 | |
| *** rtjure has joined #openstack-infra | 03:49 | |
| *** gouthamr has joined #openstack-infra | 03:49 | |
| openstackgerrit | Kien Nguyen proposed openstack-infra/openstack-zuul-jobs master: Remove legacy pyeclib job https://review.openstack.org/514513 | 03:50 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 03:50 |
| *** andreas_s has joined #openstack-infra | 03:50 | |
| *** pahuang has joined #openstack-infra | 03:51 | |
| *** alex_xu has quit IRC | 03:52 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 03:52 |
| openstackgerrit | Kien Nguyen proposed openstack-infra/project-config master: Remove legacy pyeclib job https://review.openstack.org/514512 | 03:53 |
| *** rtjure has quit IRC | 03:54 | |
| *** alex_xu has joined #openstack-infra | 03:54 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 03:54 |
| *** andreas_s has quit IRC | 03:55 | |
| *** wolverineav has joined #openstack-infra | 03:58 | |
| *** liujiong has joined #openstack-infra | 03:58 | |
| *** rtjure has joined #openstack-infra | 03:59 | |
| *** dhinesh_ has joined #openstack-infra | 04:00 | |
| *** sree has joined #openstack-infra | 04:01 | |
| *** dhinesh has quit IRC | 04:01 | |
| *** wolverineav has quit IRC | 04:02 | |
| *** gouthamr has quit IRC | 04:04 | |
| *** rtjure has quit IRC | 04:04 | |
| *** stakeda has joined #openstack-infra | 04:04 | |
| *** claudiub|2 has joined #openstack-infra | 04:05 | |
| *** armax has quit IRC | 04:05 | |
| *** rtjure has joined #openstack-infra | 04:08 | |
| *** rtjure has quit IRC | 04:13 | |
| *** armaan has joined #openstack-infra | 04:14 | |
| *** slaweq has quit IRC | 04:16 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 04:17 |
| *** rtjure has joined #openstack-infra | 04:18 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 04:19 |
| *** slaweq has joined #openstack-infra | 04:20 | |
| *** udesale has joined #openstack-infra | 04:22 | |
| *** rtjure has quit IRC | 04:22 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 04:26 |
| *** rtjure has joined #openstack-infra | 04:28 | |
| *** armaan has quit IRC | 04:29 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy oslo.versionedobjects job https://review.openstack.org/511783 | 04:29 |
| *** ijw has quit IRC | 04:31 | |
| *** rtjure has quit IRC | 04:32 | |
| *** Qiming has joined #openstack-infra | 04:33 | |
| *** adisky_ has quit IRC | 04:35 | |
| *** rtjure has joined #openstack-infra | 04:37 | |
| *** rtjure has quit IRC | 04:42 | |
| *** bhavik1 has joined #openstack-infra | 04:42 | |
| *** iyamahat has joined #openstack-infra | 04:43 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 04:43 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 04:47 |
| *** rtjure has joined #openstack-infra | 04:50 | |
| *** kiennt26 has quit IRC | 04:51 | |
| *** slaweq has quit IRC | 04:53 | |
| *** mat128 has joined #openstack-infra | 04:54 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 04:55 |
| *** rtjure has quit IRC | 04:55 | |
| *** janki has joined #openstack-infra | 04:55 | |
| ianw | i wonder why "git review" is updating the timestamp on .zuul.d/jobs.yaml when it posts it. i've not noticed that before ... it's weird, my editor then tells me the file's been modified | 04:56 |
| ianw | http://paste.openstack.org/show/624418/ | 04:57 |
| *** slaweq has joined #openstack-infra | 04:57 | |
| *** claudiub|2 has quit IRC | 04:59 | |
| *** rtjure has joined #openstack-infra | 05:00 | |
| *** Qiming has quit IRC | 05:02 | |
| *** Qiming has joined #openstack-infra | 05:03 | |
| *** ykarel has joined #openstack-infra | 05:04 | |
| *** ykarel_ has joined #openstack-infra | 05:04 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 05:04 |
| *** rtjure has quit IRC | 05:05 | |
| *** ykarel_ has quit IRC | 05:05 | |
| *** salv-orlando has joined #openstack-infra | 05:05 | |
| SamYaple | ianw: is there some sort of commit hook tom foolery happening? | 05:06 |
| ianw | 26855 unlink(".zuul.d/jobs.yaml") = 0 | 05:07 |
| *** Qiming has quit IRC | 05:07 | |
| ianw | that process being "git", "cherry-pick", "4b2ef72a25e06aac2c0ebb508f5dd7228890d049" | 05:07 |
| SamYaple | did you move the git directory after you cloned it by chance? | 05:08 |
| *** ethfci has joined #openstack-infra | 05:09 | |
| *** wolverineav has joined #openstack-infra | 05:10 | |
| ianw | ahh, i think it's an artifact of pushing a branch that a bit out of sync with origin/master | 05:12 |
| *** udesale has quit IRC | 05:12 | |
| *** Qiming has joined #openstack-infra | 05:12 | |
| *** udesale has joined #openstack-infra | 05:13 | |
| *** udesale has quit IRC | 05:13 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [DNM] start at templating out the zuulv3 jobs https://review.openstack.org/512450 | 05:13 |
| SamYaple | i suppose that makes sense | 05:13 |
| ianw | eh, i dunno, i'm not sure i've ever noticed this before | 05:14 |
| ianw | maybe i just always close the file in my editor before git review usually | 05:14 |
| *** rtjure has joined #openstack-infra | 05:14 | |
| *** wolverineav has quit IRC | 05:14 | |
| *** udesale has joined #openstack-infra | 05:17 | |
| *** rosmaita has quit IRC | 05:17 | |
| *** rtjure has quit IRC | 05:19 | |
| *** Qiming has quit IRC | 05:20 | |
| *** dmellado has quit IRC | 05:21 | |
| *** Qiming has joined #openstack-infra | 05:21 | |
| *** dmellado has joined #openstack-infra | 05:22 | |
| *** janki has quit IRC | 05:23 | |
| *** janki has joined #openstack-infra | 05:23 | |
| *** zzzeek has quit IRC | 05:24 | |
| *** Qiming_ has joined #openstack-infra | 05:24 | |
| *** zzzeek has joined #openstack-infra | 05:26 | |
| *** rtjure has joined #openstack-infra | 05:26 | |
| *** Qiming has quit IRC | 05:27 | |
| *** jtomasek has quit IRC | 05:27 | |
| *** janki has quit IRC | 05:28 | |
| *** janki has joined #openstack-infra | 05:30 | |
| *** rtjure has quit IRC | 05:31 | |
| *** bhavik1 has quit IRC | 05:31 | |
| *** rtjure has joined #openstack-infra | 05:37 | |
| *** dmellado has quit IRC | 05:37 | |
| *** dmellado has joined #openstack-infra | 05:40 | |
| *** rtjure has quit IRC | 05:41 | |
| *** dhajare has joined #openstack-infra | 05:42 | |
| *** xarses has joined #openstack-infra | 05:43 | |
| *** markvoelker has quit IRC | 05:43 | |
| *** markvoelker has joined #openstack-infra | 05:44 | |
| *** florianf has joined #openstack-infra | 05:44 | |
| *** yolanda has joined #openstack-infra | 05:44 | |
| *** gk_ has joined #openstack-infra | 05:44 | |
| *** xarses_ has quit IRC | 05:45 | |
| *** rtjure has joined #openstack-infra | 05:46 | |
| *** cshastri has joined #openstack-infra | 05:47 | |
| *** kjackal_ has quit IRC | 05:48 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Change to install a package in 'proliant-tools' https://review.openstack.org/501568 | 05:48 |
| *** markvoelker has quit IRC | 05:48 | |
| gk_ | Hi everyone, I am facing an issue in our third party CI to connect to gerrit. In the zuul's log, I have the exception "SSHException: No existing session". Besides, I used the following command in the same machine: " ssh -p 29418 username@review.openstack.org" and it is working. I wrote a small script by using the library paramiko to connect to gerrit and it failed with a connection timeout. The CI was working well since last Sat | 05:49 |
| *** wolverineav has joined #openstack-infra | 05:49 | |
| gk_ | Is anyone has any ideas for this issue? Thank you in advance. | 05:49 |
| openstackgerrit | Rabi Mishra proposed openstack-infra/project-config master: Make legacy-grenade-dsvm-heat voting for heat https://review.openstack.org/514519 | 05:50 |
| *** xarses_ has joined #openstack-infra | 05:51 | |
| *** xarses has quit IRC | 05:51 | |
| *** rtjure has quit IRC | 05:51 | |
| *** wolverineav has quit IRC | 05:53 | |
| *** rtjure has joined #openstack-infra | 05:57 | |
| *** rtjure has quit IRC | 06:02 | |
| *** slaweq has quit IRC | 06:03 | |
| *** yamahata has joined #openstack-infra | 06:05 | |
| *** coolsvap has joined #openstack-infra | 06:06 | |
| *** ankkumar has joined #openstack-infra | 06:08 | |
| *** slaweq has joined #openstack-infra | 06:08 | |
| *** Qiming_ has quit IRC | 06:10 | |
| *** rtjure has joined #openstack-infra | 06:10 | |
| *** rtjure has quit IRC | 06:15 | |
| tonyb | AJaeger: I've updated the lists of repos/branches to tag and close. And added the request/details to the meeting agenda for tomorrow | 06:16 |
| tonyb | AJaeger: I wot be there but I hope it's preety self explanatory | 06:16 |
| *** ykarel has quit IRC | 06:17 | |
| *** spectr has quit IRC | 06:20 | |
| *** spectr has joined #openstack-infra | 06:21 | |
| *** spectr has quit IRC | 06:22 | |
| *** andreas_s has joined #openstack-infra | 06:23 | |
| *** spectr has joined #openstack-infra | 06:24 | |
| *** rtjure has joined #openstack-infra | 06:24 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul-jobs master: Silence ansible-lint https://review.openstack.org/514526 | 06:29 |
| *** rtjure has quit IRC | 06:29 | |
| *** ociuhandu has joined #openstack-infra | 06:31 | |
| *** xarses has joined #openstack-infra | 06:32 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Dont install python-virtualenv for py3k in deb https://review.openstack.org/513168 | 06:33 |
| *** rtjure has joined #openstack-infra | 06:34 | |
| *** xarses_ has quit IRC | 06:34 | |
| *** claudiub|2 has joined #openstack-infra | 06:34 | |
| *** fanzhang has quit IRC | 06:35 | |
| *** ykarel has joined #openstack-infra | 06:35 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/zuul-jobs master: Silence ansible-lint https://review.openstack.org/514526 | 06:37 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Create openstack-zuul-jobs-linters jobs https://review.openstack.org/512715 | 06:37 |
| *** rtjure has quit IRC | 06:38 | |
| AJaeger | tonyb: thanks! | 06:39 |
| tonyb | AJaeger: np, thanks for the prod :) | 06:39 |
| *** fanzhang has joined #openstack-infra | 06:39 | |
| *** yolanda has quit IRC | 06:40 | |
| *** slaweq has quit IRC | 06:41 | |
| *** jbadiapa_ has joined #openstack-infra | 06:41 | |
| *** jbadiapa has quit IRC | 06:44 | |
| *** markvoelker has joined #openstack-infra | 06:44 | |
| *** rtjure has joined #openstack-infra | 06:46 | |
| *** Goneri has joined #openstack-infra | 06:46 | |
| *** slaweq_ has joined #openstack-infra | 06:46 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove jenkins Frozen tests https://review.openstack.org/513910 | 06:50 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove zuul v2 and jjb content https://review.openstack.org/507180 | 06:50 |
| *** rtjure has quit IRC | 06:50 | |
| *** ociuhandu has quit IRC | 06:53 | |
| *** slaweq has joined #openstack-infra | 06:55 | |
| *** liujiong has quit IRC | 06:56 | |
| *** rtjure has joined #openstack-infra | 06:56 | |
| AJaeger | project-config-core, the two changes above get rid of v2 config ^ | 06:57 |
| *** rtjure has quit IRC | 07:01 | |
| *** pcaruana has joined #openstack-infra | 07:02 | |
| *** rtjure has joined #openstack-infra | 07:07 | |
| *** tesseract has joined #openstack-infra | 07:11 | |
| *** kiennt26 has joined #openstack-infra | 07:12 | |
| *** rtjure has quit IRC | 07:12 | |
| *** annp has quit IRC | 07:13 | |
| *** rossella_s has joined #openstack-infra | 07:14 | |
| *** gildub has quit IRC | 07:16 | |
| *** rtjure has joined #openstack-infra | 07:17 | |
| *** markvoelker has quit IRC | 07:18 | |
| *** slaweq_ has quit IRC | 07:19 | |
| *** Qiming has joined #openstack-infra | 07:20 | |
| *** rtjure has quit IRC | 07:21 | |
| *** dhinesh_ has quit IRC | 07:23 | |
| *** Qiming has quit IRC | 07:25 | |
| *** ccamacho has joined #openstack-infra | 07:25 | |
| *** rtjure has joined #openstack-infra | 07:26 | |
| *** eumel8 has joined #openstack-infra | 07:28 | |
| AJaeger | infra-root, readthedocs progress at http://logs.openstack.org/8f/8f14c4bb474cbbdd003a7b365a047852d09ce832/post/trigger-readthedocs/10b37ac/ara/result/dcd0adea-a240-46bb-9c4a-6f0fd5c48d5e/ - but now it complains that status code is 302 and says "fail". | 07:29 |
| eumel8 | morning | 07:29 |
| *** slaweq_ has joined #openstack-infra | 07:30 | |
| *** rtjure has quit IRC | 07:31 | |
| *** gildub has joined #openstack-infra | 07:32 | |
| *** jaosorior has quit IRC | 07:33 | |
| *** Hal has joined #openstack-infra | 07:35 | |
| *** Hal is now known as Guest69866 | 07:35 | |
| *** iyamahat has quit IRC | 07:35 | |
| *** rtjure has joined #openstack-infra | 07:37 | |
| *** rtjure has quit IRC | 07:42 | |
| *** ykarel is now known as ykarel|lunch | 07:42 | |
| *** amoralej|off is now known as amoralej | 07:42 | |
| *** hashar has joined #openstack-infra | 07:42 | |
| *** yolanda has joined #openstack-infra | 07:46 | |
| *** jpena|off is now known as jpena | 07:47 | |
| *** spectr has quit IRC | 07:48 | |
| *** rtjure has joined #openstack-infra | 07:48 | |
| *** adisky_ has joined #openstack-infra | 07:48 | |
| *** spectr has joined #openstack-infra | 07:48 | |
| *** xinni9e has joined #openstack-infra | 07:52 | |
| *** xinni9e has quit IRC | 07:52 | |
| *** rtjure has quit IRC | 07:53 | |
| *** xinni9e has joined #openstack-infra | 07:53 | |
| *** xinni9e has quit IRC | 07:55 | |
| *** jaosorior has joined #openstack-infra | 07:57 | |
| *** rtjure has joined #openstack-infra | 07:57 | |
| *** sree has quit IRC | 07:57 | |
| *** sree has joined #openstack-infra | 07:58 | |
| *** dbecker has joined #openstack-infra | 07:59 | |
| *** xinni9e has joined #openstack-infra | 07:59 | |
| *** rtjure has quit IRC | 08:02 | |
| *** sree has quit IRC | 08:02 | |
| *** slaweq_ has quit IRC | 08:02 | |
| *** rtjure has joined #openstack-infra | 08:07 | |
| *** jpich has joined #openstack-infra | 08:07 | |
| *** LindaWang has quit IRC | 08:11 | |
| *** LindaWang has joined #openstack-infra | 08:11 | |
| *** slaweq_ has joined #openstack-infra | 08:11 | |
| *** rtjure has quit IRC | 08:12 | |
| *** dtantsur|afk is now known as dtantsur | 08:12 | |
| *** pgadiya has joined #openstack-infra | 08:14 | |
| *** markvoelker has joined #openstack-infra | 08:15 | |
| *** Goneri has quit IRC | 08:16 | |
| *** rtjure has joined #openstack-infra | 08:17 | |
| *** ralonsoh has joined #openstack-infra | 08:20 | |
| *** namnh has quit IRC | 08:21 | |
| *** rtjure has quit IRC | 08:21 | |
| openstackgerrit | Kien Nguyen proposed openstack-infra/zuul-jobs master: [DNM] Fix generic process-test-results role https://review.openstack.org/514559 | 08:23 |
| openstackgerrit | Adrian Czarnecki proposed openstack-infra/project-config master: Remove legacy job for monasca https://review.openstack.org/513722 | 08:23 |
| *** rtjure has joined #openstack-infra | 08:26 | |
| openstackgerrit | Adrian Czarnecki proposed openstack-infra/project-config master: Remove legacy job for monasca https://review.openstack.org/513722 | 08:31 |
| *** tosky has joined #openstack-infra | 08:31 | |
| *** rtjure has quit IRC | 08:31 | |
| *** lucas-afk is now known as lucasagomes | 08:32 | |
| *** yamamoto has quit IRC | 08:32 | |
| *** priteau has joined #openstack-infra | 08:33 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack-infra/tripleo-ci master: Match console and job-output logs for getthelogs https://review.openstack.org/514560 | 08:33 |
| openstackgerrit | Adrian Czarnecki proposed openstack-infra/openstack-zuul-jobs master: Remove monasca legacy job https://review.openstack.org/514251 | 08:34 |
| *** ociuhandu has joined #openstack-infra | 08:34 | |
| *** rtjure has joined #openstack-infra | 08:36 | |
| *** derekh has joined #openstack-infra | 08:37 | |
| lucasagomes | AJaeger, hi there, I'm seem a "ERROR No valid playbook found" in the stable/pike branch for networking-ovn should I backport the playbooks to the stable branches as well ? | 08:38 |
| *** ykarel|lunch is now known as ykarel | 08:38 | |
| *** gildub has quit IRC | 08:39 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy job from Oslo.log https://review.openstack.org/511384 | 08:40 |
| *** rtjure has quit IRC | 08:41 | |
| *** sree has joined #openstack-infra | 08:41 | |
| lucasagomes | AJaeger, here's one example: https://review.openstack.org/#/c/506561/ | 08:42 |
| *** eumel8 has quit IRC | 08:42 | |
| *** mika has quit IRC | 08:43 | |
| *** electrofelix has joined #openstack-infra | 08:44 | |
| *** ociuhandu has quit IRC | 08:44 | |
| *** jtomasek has joined #openstack-infra | 08:44 | |
| *** slaweq_ has quit IRC | 08:45 | |
| *** mika has joined #openstack-infra | 08:45 | |
| *** rtjure has joined #openstack-infra | 08:46 | |
| AJaeger | lucasagomes: https://docs.openstack.org/infra/manual/zuulv3.html#stable-branches | 08:46 |
| *** markvoelker has quit IRC | 08:48 | |
| *** hwoarang has quit IRC | 08:48 | |
| *** slaweq_ has joined #openstack-infra | 08:50 | |
| *** jtomasek has quit IRC | 08:50 | |
| *** rtjure has quit IRC | 08:51 | |
| *** hwoarang has joined #openstack-infra | 08:51 | |
| *** jtomasek has joined #openstack-infra | 08:51 | |
| lucasagomes | AJaeger, ouch, missed that sorry! | 08:52 |
| *** armaan has joined #openstack-infra | 08:52 | |
| openstackgerrit | Noam Angel proposed openstack/diskimage-builder master: cloud-init element systemctl support https://review.openstack.org/512489 | 08:55 |
| *** rtjure has joined #openstack-infra | 08:55 | |
| *** Goneri has joined #openstack-infra | 08:58 | |
| openstackgerrit | Jens Harbott (frickler) proposed openstack-infra/zuul-jobs master: Follow redirects when triggering readthedocs build https://review.openstack.org/514570 | 08:58 |
| frickler | AJaeger: ^^ should handle the readthedocs redirect | 08:58 |
| *** rtjure has quit IRC | 09:00 | |
| *** daidv has quit IRC | 09:03 | |
| *** daidv has joined #openstack-infra | 09:03 | |
| *** daidv_ has joined #openstack-infra | 09:04 | |
| *** daidv_ has quit IRC | 09:04 | |
| *** daidv has quit IRC | 09:04 | |
| *** daidv has joined #openstack-infra | 09:04 | |
| *** daidv has quit IRC | 09:05 | |
| *** daidv has joined #openstack-infra | 09:06 | |
| *** huanxie has joined #openstack-infra | 09:07 | |
| *** rtjure has joined #openstack-infra | 09:08 | |
| *** rtjure has quit IRC | 09:13 | |
| *** sdague has joined #openstack-infra | 09:16 | |
| *** gmann is now known as gmann_afk | 09:17 | |
| *** Qiming has joined #openstack-infra | 09:17 | |
| *** rtjure has joined #openstack-infra | 09:17 | |
| *** rtjure has quit IRC | 09:23 | |
| *** slaweq_ has quit IRC | 09:23 | |
| *** makowals has quit IRC | 09:23 | |
| *** pgadiya has quit IRC | 09:23 | |
| *** makowals has joined #openstack-infra | 09:24 | |
| *** pgadiya has joined #openstack-infra | 09:24 | |
| openstackgerrit | Sergey Kulanov proposed openstack-infra/jeepyb master: Ensure acl_config has absolute path https://review.openstack.org/482612 | 09:24 |
| *** slaweq_ has joined #openstack-infra | 09:26 | |
| *** rtjure has joined #openstack-infra | 09:28 | |
| *** zzzeek has quit IRC | 09:30 | |
| *** zzzeek has joined #openstack-infra | 09:32 | |
| *** yamamoto has joined #openstack-infra | 09:32 | |
| *** rtjure has quit IRC | 09:33 | |
| *** Qiming has quit IRC | 09:33 | |
| *** tosky has quit IRC | 09:38 | |
| *** rtjure has joined #openstack-infra | 09:38 | |
| *** yamamoto has quit IRC | 09:42 | |
| *** rtjure has quit IRC | 09:42 | |
| *** sambetts|afk is now known as sambetts | 09:43 | |
| *** markvoelker has joined #openstack-infra | 09:44 | |
| *** martinkopec has joined #openstack-infra | 09:45 | |
| *** Qiming has joined #openstack-infra | 09:46 | |
| *** rtjure has joined #openstack-infra | 09:48 | |
| *** Qiming has quit IRC | 09:50 | |
| frickler | I've analysed the issue displaying only partial Zuul results when tripleo is involved, see the etherpad, need advise on which solution would seem correct | 09:52 |
| *** rtjure has quit IRC | 09:53 | |
| *** xarses_ has joined #openstack-infra | 09:55 | |
| *** thorst has joined #openstack-infra | 09:56 | |
| *** xarses has quit IRC | 09:57 | |
| *** armaan has quit IRC | 09:58 | |
| *** rtjure has joined #openstack-infra | 09:58 | |
| *** slaweq_ has quit IRC | 09:58 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: test containers update https://review.openstack.org/511175 | 10:00 |
| *** thorst has quit IRC | 10:00 | |
| *** slaweq_ has joined #openstack-infra | 10:01 | |
| *** rtjure has quit IRC | 10:03 | |
| *** yamahata has quit IRC | 10:05 | |
| *** gildub has joined #openstack-infra | 10:05 | |
| *** yamamoto has joined #openstack-infra | 10:08 | |
| *** rtjure has joined #openstack-infra | 10:09 | |
| *** spectr has quit IRC | 10:10 | |
| *** spectr has joined #openstack-infra | 10:10 | |
| *** Qiming has joined #openstack-infra | 10:11 | |
| *** andreas_s has quit IRC | 10:12 | |
| *** andreas_s has joined #openstack-infra | 10:12 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Use playbook from tripleo-quickstart-extras for OVB https://review.openstack.org/513508 | 10:12 |
| *** cuongnv has quit IRC | 10:13 | |
| *** rtjure has quit IRC | 10:14 | |
| *** daidv has quit IRC | 10:14 | |
| *** markvoelker has quit IRC | 10:18 | |
| *** slaweq has quit IRC | 10:19 | |
| *** ldnunes has joined #openstack-infra | 10:20 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/openstack-zuul-jobs master: Remove legacy OSA tests https://review.openstack.org/512635 | 10:22 |
| *** rossella_s has quit IRC | 10:23 | |
| *** yamamoto has quit IRC | 10:25 | |
| *** rtjure has joined #openstack-infra | 10:25 | |
| openstackgerrit | Jens Harbott (frickler) proposed openstack-infra/system-config master: Add match for "-" in pipeline name https://review.openstack.org/514591 | 10:26 |
| frickler | ^^ after thinking about it a bit more, this is the only solution that works for existing changes, so I'm proposing this | 10:26 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config master: Remove remaining legacy openstack-ansible jobs https://review.openstack.org/514592 | 10:27 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/openstack-zuul-jobs master: Remove legacy OSA tests https://review.openstack.org/512635 | 10:28 |
| *** rtjure has quit IRC | 10:30 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/openstack-zuul-jobs master: Remove remaining legacy openstack-ansible tests https://review.openstack.org/512635 | 10:31 |
| *** slaweq has joined #openstack-infra | 10:31 | |
| *** stakeda has quit IRC | 10:32 | |
| *** slaweq_ has quit IRC | 10:35 | |
| *** rtjure has joined #openstack-infra | 10:35 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add documentation publish jobs to Blazar https://review.openstack.org/511176 | 10:38 |
| *** pbourke has quit IRC | 10:38 | |
| *** rtjure has quit IRC | 10:40 | |
| *** slaweq_ has joined #openstack-infra | 10:40 | |
| *** e0ne has joined #openstack-infra | 10:40 | |
| *** pbourke has joined #openstack-infra | 10:40 | |
| openstackgerrit | Vasyl Saienko proposed openstack-infra/project-config master: Remove legacy ironic-python-agent jobs https://review.openstack.org/514596 | 10:42 |
| *** sree has quit IRC | 10:44 | |
| *** vsaienk0 has joined #openstack-infra | 10:44 | |
| *** sree has joined #openstack-infra | 10:44 | |
| *** rtjure has joined #openstack-infra | 10:44 | |
| *** andreas_s has quit IRC | 10:45 | |
| vsaienk0 | AJaeger: could you please help with https://review.openstack.org/#/c/513380/9/zuul.d/project.yaml, we don't know if we need to move templates: section to our repo for standard jobs | 10:45 |
| *** andreas_s has joined #openstack-infra | 10:45 | |
| *** pgadiya has quit IRC | 10:46 | |
| *** aviau has quit IRC | 10:46 | |
| *** aviau has joined #openstack-infra | 10:47 | |
| odyssey4me | dmsimard oanson jamespage lbragstad we want to remove the legacy openstack-ansible cross-repo integration tests so that we can replace them with zuulv3 jobs - please note https://review.openstack.org/514592 and indicate your support | 10:47 |
| odyssey4me | oanson I see that you've already worked on a replacement for dragonflow in https://review.openstack.org/514268 - thanks! | 10:47 |
| AJaeger | vsaienk0: commented | 10:47 |
| *** sdague has quit IRC | 10:48 | |
| *** pgadiya has joined #openstack-infra | 10:49 | |
| *** sree has quit IRC | 10:49 | |
| AJaeger | odyssey4me: why not change those jobs to native v3 ones? | 10:49 |
| *** rtjure has quit IRC | 10:49 | |
| vsaienk0 | AJaeger: thanks, just pushed new version | 10:50 |
| odyssey4me | AJaeger they will be in subsequent patches, but they depend on a base job in openstack-ansible-tests which has not merged yet | 10:50 |
| odyssey4me | we need to get rid of the legacy tests first as they're in the way | 10:50 |
| AJaeger | vsaienk0: LGTM, thanks | 10:51 |
| openstackgerrit | Vasyl Saienko proposed openstack-infra/project-config master: Remove legacy ironic-python-agent jobs https://review.openstack.org/514596 | 10:51 |
| AJaeger | odyssey4me: what about adding a patch on top that adds the new ones directly? Might make it easier for reviewers to see what you do? | 10:51 |
| *** sdague has joined #openstack-infra | 10:51 | |
| *** psachin has quit IRC | 10:51 | |
| odyssey4me | AJaeger as you'll see in https://review.openstack.org/514268 dragonflow is adding an in-repo test instead | 10:52 |
| odyssey4me | for keystone/ARA I'll discuss with dmsimard and lbragstad when they're online | 10:52 |
| oanson | lihi, cc ^^^^^ | 10:53 |
| *** udesale has quit IRC | 10:53 | |
| *** dtantsur is now known as dtantsur|brb | 10:53 | |
| AJaeger | odyssey4me: ok | 10:53 |
| *** psachin has joined #openstack-infra | 10:54 | |
| *** rtjure has joined #openstack-infra | 10:55 | |
| *** yolanda has quit IRC | 10:55 | |
| *** rtjure has quit IRC | 11:00 | |
| *** slaweq has quit IRC | 11:01 | |
| *** vsaienk0 has quit IRC | 11:01 | |
| *** slaweq has joined #openstack-infra | 11:01 | |
| *** Qiming has quit IRC | 11:02 | |
| *** yamamoto has joined #openstack-infra | 11:04 | |
| *** rtjure has joined #openstack-infra | 11:06 | |
| vdrok | good morning folks | 11:06 |
| *** slaweq has quit IRC | 11:06 | |
| vdrok | AJaeger: small question regarding py27 job on ironic-specs repo, we seem to have it defined in https://raw.githubusercontent.com/openstack-infra/project-config/master/zuul.d/projects.yaml, but it does not seem to run, eg on https://review.openstack.org/496481. What could be the reason? | 11:07 |
| *** rtjure has quit IRC | 11:11 | |
| *** gildub has quit IRC | 11:13 | |
| *** lucasagomes is now known as lucas-hungry | 11:14 | |
| *** andreas_s has quit IRC | 11:15 | |
| *** markvoelker has joined #openstack-infra | 11:15 | |
| *** andreas_s has joined #openstack-infra | 11:15 | |
| *** smatzek has joined #openstack-infra | 11:15 | |
| *** rtjure has joined #openstack-infra | 11:16 | |
| vdrok | nevermind, I guess I see it, py27 has irrelevant files *.rst in the job definition | 11:16 |
| *** andreas_s has quit IRC | 11:17 | |
| *** andreas_s has joined #openstack-infra | 11:18 | |
| *** rtjure has quit IRC | 11:21 | |
| *** rtjure has joined #openstack-infra | 11:26 | |
| openstackgerrit | Vladyslav Drok proposed openstack-infra/project-config master: Override irrelevant files for py27 job on ironic-specs https://review.openstack.org/514610 | 11:26 |
| vdrok | AJaeger: would this one do the trick? ^^ | 11:27 |
| *** andreas_s has quit IRC | 11:28 | |
| *** andreas_s has joined #openstack-infra | 11:28 | |
| *** huanxie has quit IRC | 11:29 | |
| *** dbecker has quit IRC | 11:30 | |
| *** Qiming has joined #openstack-infra | 11:30 | |
| *** rtjure has quit IRC | 11:30 | |
| *** vsaienk0 has joined #openstack-infra | 11:31 | |
| *** thiagolib has joined #openstack-infra | 11:32 | |
| *** Qiming has quit IRC | 11:34 | |
| Shrews | ianw: replied to your question on https://review.openstack.org/514321 | 11:35 |
| *** rtjure has joined #openstack-infra | 11:37 | |
| *** kjackal_ has joined #openstack-infra | 11:38 | |
| *** andreas_s has quit IRC | 11:38 | |
| *** Qiming has joined #openstack-infra | 11:39 | |
| *** rtjure has quit IRC | 11:42 | |
| *** andreas_s has joined #openstack-infra | 11:43 | |
| *** Qiming has quit IRC | 11:43 | |
| *** pgadiya has quit IRC | 11:43 | |
| *** adisky_ has quit IRC | 11:43 | |
| *** udesale has joined #openstack-infra | 11:46 | |
| *** andreas_s has quit IRC | 11:46 | |
| *** andreas_s has joined #openstack-infra | 11:46 | |
| *** rtjure has joined #openstack-infra | 11:47 | |
| *** markvoelker has quit IRC | 11:48 | |
| *** HTTP_____GK1wmSU has joined #openstack-infra | 11:49 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Create job-output.txt together with JobDir https://review.openstack.org/514617 | 11:50 |
| *** jpena is now known as jpena|lunch | 11:50 | |
| *** HTTP_____GK1wmSU has left #openstack-infra | 11:50 | |
| *** rtjure has quit IRC | 11:52 | |
| *** rtjure has joined #openstack-infra | 12:00 | |
| *** amoralej is now known as amoralej|lunch | 12:01 | |
| *** rossella_s has joined #openstack-infra | 12:04 | |
| *** Qiming has joined #openstack-infra | 12:04 | |
| *** rtjure has quit IRC | 12:04 | |
| *** fultonj has quit IRC | 12:04 | |
| *** thorst has joined #openstack-infra | 12:04 | |
| *** trown|outtypewww has quit IRC | 12:04 | |
| *** trown has joined #openstack-infra | 12:05 | |
| *** fultonj has joined #openstack-infra | 12:05 | |
| *** trown|brb has joined #openstack-infra | 12:07 | |
| *** trown|brb has quit IRC | 12:07 | |
| *** gcb has joined #openstack-infra | 12:07 | |
| *** Qiming has quit IRC | 12:09 | |
| *** rtjure has joined #openstack-infra | 12:09 | |
| *** Qiming has joined #openstack-infra | 12:13 | |
| *** rtjure has quit IRC | 12:14 | |
| *** lucas-hungry is now known as lucasagomes | 12:15 | |
| AJaeger | vdrok: I'm confused, do you want to run it - or not? | 12:16 |
| *** dprince has joined #openstack-infra | 12:16 | |
| vdrok | AJaeger: yes, we want to run py27 on .rst files | 12:16 |
| vdrok | I've also done https://review.openstack.org/#/c/514626/1, not sure what is the working/best way | 12:16 |
| *** slaweq_ has quit IRC | 12:17 | |
| *** slaweq has joined #openstack-infra | 12:18 | |
| *** slaweq_ has joined #openstack-infra | 12:18 | |
| AJaeger | vdrok: discuss with the team later on, please. I have no time right now. Just quickly: 514610 creates a variant, so that merges the list. You need to inherit the job - or discuss whether this is really the right approach. I would run your tests as *pep8* and not py27 - since you're linting. | 12:18 |
| vdrok | AJaeger: aha, ok. so it seems a custom inherited job is the way to go if we leave it in py27 | 12:19 |
| vdrok | thank you | 12:19 |
| openstackgerrit | boden proposed openstack-infra/project-config master: add neutron to vmware-nsx releasenotes jobs https://review.openstack.org/514630 | 12:19 |
| *** Qiming has quit IRC | 12:20 | |
| *** rtjure has joined #openstack-infra | 12:21 | |
| *** dhinesh has joined #openstack-infra | 12:25 | |
| *** rtjure has quit IRC | 12:25 | |
| *** dpawlik has quit IRC | 12:25 | |
| *** danpawlik has joined #openstack-infra | 12:26 | |
| *** dhinesh has quit IRC | 12:29 | |
| *** Qiming has joined #openstack-infra | 12:30 | |
| openstackgerrit | Eric Fried proposed openstack-infra/project-config master: Add stables to patchbot for nova-powervm https://review.openstack.org/514633 | 12:30 |
| *** rlandy has joined #openstack-infra | 12:30 | |
| *** dave-mccowan has joined #openstack-infra | 12:30 | |
| *** rtjure has joined #openstack-infra | 12:31 | |
| *** rosmaita has joined #openstack-infra | 12:32 | |
| openstackgerrit | Eric Fried proposed openstack-infra/project-config master: Add stables to patchbot for *-powervm https://review.openstack.org/514633 | 12:33 |
| *** dbecker has joined #openstack-infra | 12:34 | |
| *** slaweq has quit IRC | 12:35 | |
| *** slaweq has joined #openstack-infra | 12:35 | |
| *** ankkumar has quit IRC | 12:36 | |
| *** rtjure has quit IRC | 12:36 | |
| *** dtantsur|brb is now known as dtantsur | 12:38 | |
| smcginnis | Hmm, watching the stream, there were log gearman errors on this: http://logs.openstack.org/42/514442/1/gate/openstack-tox-py35/739cade/job-output.txt.gz | 12:39 |
| smcginnis | But not in the actual logs. | 12:39 |
| smcginnis | And doesn't appear to have things like the local.conf and any other logs/artifacts. | 12:39 |
| *** LindaWang has quit IRC | 12:40 | |
| *** slaweq has quit IRC | 12:40 | |
| *** sree has joined #openstack-infra | 12:41 | |
| *** rtjure has joined #openstack-infra | 12:42 | |
| *** sbezverk has quit IRC | 12:45 | |
| *** sree has quit IRC | 12:46 | |
| smcginnis | Caught it on another one: http://paste.openstack.org/show/624464/ | 12:46 |
| smcginnis | Cinder link previously is probably actually fine. That was just a unit test run, so not really any other artifacts to collect I guess. | 12:47 |
| *** rtjure has quit IRC | 12:47 | |
| dmsimard | odyssey4me: my ara senses are tingling | 12:47 |
| odyssey4me | dmsimard yeah, https://review.openstack.org/514592 - do you want to do an in-repo job definition for the cross-repo test? | 12:48 |
| odyssey4me | dragonflow has one going to replace theirs: https://review.openstack.org/514268 | 12:48 |
| odyssey4me | the commit message has another example | 12:48 |
| pabelanger | smcginnis: is that cause the job to fail? | 12:50 |
| smcginnis | pabelanger: No, job was fine. Just didn't seem like we were getting all the logs. | 12:51 |
| smcginnis | pabelanger: But that might have been fine for that job, so maybe false alarm. | 12:51 |
| smcginnis | Still interesting seeing the errors in the stream. | 12:52 |
| pabelanger | Yah, we still need to debug the logstash issue with gearman | 12:52 |
| *** slaweq has joined #openstack-infra | 12:52 | |
| *** rtjure has joined #openstack-infra | 12:53 | |
| *** benj_ has quit IRC | 12:54 | |
| *** esberglu has joined #openstack-infra | 12:55 | |
| *** yolanda has joined #openstack-infra | 12:56 | |
| *** sree has joined #openstack-infra | 12:57 | |
| *** rtjure has quit IRC | 12:58 | |
| *** mriedem has joined #openstack-infra | 12:58 | |
| *** dhill_ has quit IRC | 12:58 | |
| *** bh526r has joined #openstack-infra | 13:00 | |
| *** sree has quit IRC | 13:00 | |
| *** sree has joined #openstack-infra | 13:01 | |
| dmsimard | odyssey4me: feel free to remove the legacy jobs from ara, they haven't been working and they're non-voting | 13:01 |
| dmsimard | odyssey4me: we can work something out later, no rush | 13:01 |
| dmsimard | odyssey4me: I suspect ara could just re-use a job defined in osa-tests or something | 13:01 |
| *** bobh has joined #openstack-infra | 13:01 | |
| odyssey4me | dmsimard if you can just note that in the review I'd appreciate it | 13:01 |
| *** amoralej|lunch is now known as amoralej|mtg | 13:03 | |
| *** rtjure has joined #openstack-infra | 13:03 | |
| *** edmondsw has joined #openstack-infra | 13:04 | |
| dmsimard | odyssey4me: +2 | 13:04 |
| *** armaan has joined #openstack-infra | 13:04 | |
| dhellmann | we seem to have solved the same problem in multiple ways. The release-openstack-python job checks out neutron and horizon as required-projects and then we have project templates for different jobs that each have those dependencies (publish-to-pypi-neutron and publish-to-pypi-horizon). I see the benefit of the requirements-specific templates but having extra jobs complicates the release validation, so I wonder how many | 13:05 |
| dhellmann | of those we think we're going to have? | 13:05 |
| ykarel | is there some issue going on with gate jobs? | 13:06 |
| ykarel | https://review.openstack.org/#/c/514261/ | 13:06 |
| Jeffrey4l | hi, i found this string "# BEGIN ANSIBLE GENERATED HOSTS" in /etc/hosts, how it is generated? | 13:06 |
| dhellmann | mordred, fungi, pabelanger, smcginnis : thoughts on having dependency-specific jobs? ^^ | 13:07 |
| ykarel | few minutes ago i saw jobs passed, now its showing jobs in queued state, i saw this twice for same patch: https://review.openstack.org/#/c/514261/, any idea? | 13:07 |
| *** rtjure has quit IRC | 13:08 | |
| AJaeger | dhellmann: we can remove those again and say we'll have those in the base... | 13:09 |
| dhellmann | AJaeger : yeah, I can propose that patch. I'm trying to get a sense of what direction folks were heading when they added those templates. | 13:10 |
| pabelanger | ykarel: looks like a gate reset, becasue gate is a dependent pipeline manager (https://docs.openstack.org/infra/zuul/feature/zuulv3/user/config.html#value-pipeline.manager.dependent) if the previous patchset job fails, all patches that follow are reparented and jobs run again | 13:10 |
| odyssey4me | thanks dmsimard | 13:10 |
| AJaeger | dhellmann: we didn't release the pattern initially ... | 13:10 |
| pabelanger | ykarel: looks like the gate pipeline for tripleo change queue has reset a few times, so possible a job is broken and needs to be fixed | 13:11 |
| *** jcoufal has joined #openstack-infra | 13:11 | |
| pabelanger | Jeffrey4l: which job are you looking at? | 13:11 |
| Jeffrey4l | http://logs.openstack.org/17/514517/3/check/kolla-ansible-oraclelinux-source/75bbed4/job-output.txt.gz | 13:12 |
| pabelanger | Jeffrey4l: http://git.openstack.org/cgit/openstack/kolla-ansible/tree/ansible/roles/baremetal/tasks/pre-install.yml#n19 | 13:12 |
| *** slaweq has quit IRC | 13:12 | |
| pabelanger | looks to be the role | 13:12 |
| Jeffrey4l | pabelanger, i try to update the /etc/hosts file, this is added. | 13:12 |
| Jeffrey4l | aha, kolla did. thanks. | 13:13 |
| *** slaweq has joined #openstack-infra | 13:13 | |
| *** hichihara has joined #openstack-infra | 13:14 | |
| ykarel | pabelanger, Thanks. Is there a way to know that jobs are rerunning due to a gate reset | 13:14 |
| *** niraj_singh has joined #openstack-infra | 13:16 | |
| pabelanger | ykarel: you can usually tell by how long a job has been queued into a pipeline, right now tripleo change queue is up to 14hr15min. Should never be more then 3hr, if things are working correctly | 13:16 |
| ykarel | pabelanger, Ok, Thanks | 13:16 |
| *** rtjure has joined #openstack-infra | 13:16 | |
| *** slaweq has quit IRC | 13:17 | |
| *** wolverineav has joined #openstack-infra | 13:18 | |
| *** adrian has quit IRC | 13:18 | |
| niraj_singh | is there any documentation about the jenkins jobs or when we should use which jobs like python-job, pypi-jobs etc | 13:18 |
| openstackgerrit | Major Hayden proposed openstack/gertty master: Hide Zuul comments https://review.openstack.org/514662 | 13:19 |
| *** kiennt26_ has joined #openstack-infra | 13:19 | |
| AJaeger | dhellmann: release -> realize how widespread the problem is | 13:20 |
| dhellmann | AJaeger : I'm not sure how to interpret that in terms of what you think we should be doing now vs. what has already been done. | 13:20 |
| *** rtjure has quit IRC | 13:21 | |
| *** slaweq has joined #openstack-infra | 13:21 | |
| AJaeger | dhellmann: what I wanted to say but was interrupted: I'm fine with removing those -horizon/-neutron templates. When we did those initially, we didn't realize the size of the problem. We know more now ;) | 13:21 |
| *** baoli has joined #openstack-infra | 13:21 | |
| dhellmann | ok | 13:22 |
| AJaeger | niraj_singh: the PTI - https://governance.openstack.org/tc/reference/project-testing-interface.html | 13:22 |
| *** LindaWang has joined #openstack-infra | 13:22 | |
| niraj_singh | Ajaeger: Thanks | 13:23 |
| frickler | AJaeger: looks like a new error in translation post job to me: http://logs.openstack.org/db/dbaecbda46111594224a6b8aba16576291b5a36c/post/upstream-translation-update/32a5d9c/job-output.txt.gz#_2017-10-24_13_06_36_066722 | 13:24 |
| *** slaweq_ has quit IRC | 13:24 | |
| *** gmann_afk is now known as gmann | 13:24 | |
| *** ykarel is now known as ykarel|away | 13:25 | |
| AJaeger | frickler: http://logs.openstack.org/db/dbaecbda46111594224a6b8aba16576291b5a36c/post/upstream-translation-update/32a5d9c/job-output.txt.gz#_2017-10-24_13_06_34_732223 - we query the server whether there'S stable/pike translation set up for neutron | 13:26 |
| AJaeger | And then the server tells us: No, not - and then end. | 13:26 |
| AJaeger | In that case we exit 0. But have no files produced | 13:26 |
| AJaeger | So, everything fine - but we need to teach ansible to tread this as "Success". Any ideas? | 13:27 |
| AJaeger | frickler: we should skip "Collect translation source files" in this case - so, perhaps check whether the directory exists? | 13:27 |
| *** tpsilva has joined #openstack-infra | 13:28 | |
| *** rtjure has joined #openstack-infra | 13:29 | |
| *** ykarel|away has quit IRC | 13:29 | |
| frickler | AJaeger: yeah, something along that path, I can try to come up with a patch if you're busy | 13:29 |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: DO NOT MERGE: Use rdoproject-based settings for containers https://review.openstack.org/514668 | 13:29 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: paunch: fix multinode gate job https://review.openstack.org/514671 | 13:30 |
| AJaeger | frickler: would be great if you can tackle it... | 13:30 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul feature/zuulv3: Add log streaming logging and exception handling https://review.openstack.org/513811 | 13:30 |
| *** slaweq_ has joined #openstack-infra | 13:31 | |
| dhellmann | AJaeger : in the process of composing an email about this, I think I'm convincing myself that having separate job templates will be the right approach, if we move everything into the releases repo where it's easier to manage | 13:31 |
| *** jpena|lunch is now known as jpena | 13:33 | |
| AJaeger | dhellmann: that's a policy question - where should those life. Let's discuss in a broader round here how that "architecture" could look like. Yes, definitely an option. | 13:33 |
| *** felipemonteiro_ has joined #openstack-infra | 13:33 | |
| dhellmann | AJaeger : yeah, I'm starting a ML thread | 13:34 |
| *** rtjure has quit IRC | 13:34 | |
| *** felipemonteiro__ has joined #openstack-infra | 13:38 | |
| *** rtjure has joined #openstack-infra | 13:38 | |
| *** felipemonteiro_ has quit IRC | 13:42 | |
| dhellmann | AJaeger : http://lists.openstack.org/pipermail/openstack-dev/2017-October/123926.html | 13:43 |
| *** rtjure has quit IRC | 13:43 | |
| AJaeger | project-config-core ^ | 13:45 |
| *** armax has joined #openstack-infra | 13:46 | |
| openstackgerrit | Bogdan Dobrelya proposed openstack-infra/tripleo-ci master: Match job-output as console logs for getthelogs https://review.openstack.org/514560 | 13:47 |
| openstackgerrit | Dan Prince proposed openstack-infra/openstack-zuul-jobs master: Make tripleo undercloud-containers voting https://review.openstack.org/513163 | 13:49 |
| *** rtjure has joined #openstack-infra | 13:50 | |
| *** dhajare has quit IRC | 13:50 | |
| *** coolsvap has quit IRC | 13:53 | |
| *** sbezverk has joined #openstack-infra | 13:54 | |
| *** rtjure has quit IRC | 13:54 | |
| *** bexelbie_ has joined #openstack-infra | 13:55 | |
| *** yamahata has joined #openstack-infra | 13:59 | |
| *** sbezverk has quit IRC | 13:59 | |
| *** eharney has joined #openstack-infra | 13:59 | |
| *** rtjure has joined #openstack-infra | 14:00 | |
| *** yamamoto has quit IRC | 14:00 | |
| *** iyamahat has joined #openstack-infra | 14:01 | |
| *** yamamoto has joined #openstack-infra | 14:02 | |
| *** Guest99060 is now known as med_ | 14:03 | |
| *** med_ has quit IRC | 14:03 | |
| *** med_ has joined #openstack-infra | 14:03 | |
| *** med_ is now known as medberry | 14:03 | |
| *** rtjure has quit IRC | 14:05 | |
| fungi | dhellmann: i'm wary of ending up with everyone installing horizon and neutron just so a small minority don't have to declare they need them... the separate jobs seem more like a wart we should live with until those projects can fix themselves to be able to generate sdists/wheels without requiring the source trees for the services to which they're acting as plugins/extensions? | 14:05 |
| dhellmann | fungi : they don't "install" them unless the job explicitly says to, right? | 14:05 |
| dhellmann | right now the job just clones those repos, and something run by tox installs them | 14:06 |
| fungi | dhellmann: well, tox-siblings will explicitly install them in any tox env once it's back in action, but that aside it still results in zuul needing to calculate extra horizon and neutron refs for those builds and push the source for them onto the nodes. all that is overhead | 14:07 |
| dhellmann | oh | 14:07 |
| rosmaita | i know you're all busy, but when someone has a chance, i'm seeing these failures in python-glanceclient functional tests ... i think it's a problem on the infra side, but it could be on my side ... anyway, if someone could take a quick glance at http://logs.openstack.org/91/507291/1/check/legacy-glanceclient-dsvm-functional/c66678f/job-output.txt.gz#_2017-10-24_13_45_59_430372 i'd appreciate it | 14:07 |
| dhellmann | fungi : I think I'm still suffering from not understanding how this stuff works. Could you reply on the thread with what you just said? | 14:07 |
| fungi | dhellmann: and neutron and horizon aren't exactly small git repos, as sizes compare | 14:07 |
| fungi | dhellmann: sure, i haven't gotten to reading through e-mail yet today so haven't seen the thread, but will when i reach it | 14:08 |
| hwoarang | mordred: hello. did you have a chance to look at https://review.openstack.org/#/c/513706/ again? i did a recheck and it appears that the run phase still runs from the stable/newton | 14:08 |
| *** gmann is now known as gmann_afk | 14:08 | |
| hwoarang | i can't tell if it's something that we do wrong or it's something wrong with zuulv3 | 14:08 |
| dhellmann | fungi : ok, thanks. The new world order under v3 has a *lot* of differences that I'm going to have to internalize before answers to these questions are in any way obvious | 14:08 |
| *** rtjure has joined #openstack-infra | 14:09 | |
| fungi | dhellmann: at any rate, unless _i'm_ overlooking something, we should reengineer those plugin projects to be packageable without installing other stuff (the pti will soon mandate that if mordred's proposed adjustments pass rollcall) | 14:10 |
| fungi | and also to be able to build documentation/release notes without relying on tox installing stuff for them | 14:11 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: paunch: fix multinode gate job https://review.openstack.org/514671 | 14:11 |
| dhellmann | fungi : I agree with fixing the projects as a long term solution. | 14:11 |
| fungi | so if the extra job definitions are seen as a stop-gap/workaround, not too terrible in my mind | 14:12 |
| fungi | maybe fodder for a rocky goal or something | 14:12 |
| *** kiennt26_ has quit IRC | 14:13 | |
| dmsimard | rosmaita: that's a known issue, let me dig a bit | 14:13 |
| *** kiennt26 has quit IRC | 14:14 | |
| *** rtjure has quit IRC | 14:14 | |
| rosmaita | dmsimard: no hurry, ty | 14:14 |
| dmsimard | rosmaita: the likely fix is https://review.openstack.org/#/c/512194/1, however I don't think we've got to the bottom of it | 14:14 |
| *** shardy has joined #openstack-infra | 14:15 | |
| *** medberry is now known as med_ | 14:15 | |
| *** rbrndt has joined #openstack-infra | 14:15 | |
| dmsimard | rosmaita: have you been hitting that issue consistently ? | 14:16 |
| rosmaita | dmsimard: i think so, but i'll go check some other patches | 14:18 |
| *** hongbin has joined #openstack-infra | 14:18 | |
| dmsimard | rosmaita: if you are hitting it consistently, (I don't know if you're familiar with Ansible/Zuul v3) it would help if you could *force* that patch in -- like, add a task before your playbook that patches that file with that content, just to see if that resolves it | 14:19 |
| *** hemna_ has joined #openstack-infra | 14:19 | |
| openstackgerrit | Andreas Florath proposed openstack/diskimage-builder master: [WIP] Introduce DIB_USE_CACHE and use it in zypper-minimal https://review.openstack.org/514689 | 14:19 |
| *** rtjure has joined #openstack-infra | 14:20 | |
| *** udesale has quit IRC | 14:21 | |
| dmsimard | infra: is there a way to get nodepool to just boot a "static" image rather than have it build, upload and use images built out of nodepool-builder ? | 14:23 |
| rosmaita | dmsimard: (a) i do seem to be hitting it consistently, (b) not familiar with zuul v3 yet (though i guess now is the time to start) | 14:23 |
| dmsimard | suppose I have images that are built differently, not with DIB, that I'd like to use -- and that converting that to DIB would suck | 14:23 |
| fungi | dmsimard: until support gets added to nodepool for booting specifically named images in a provider, you could likely write a fake nodepool builder which handles build requests in zk and returns the ids of the predefined images? | 14:24 |
| jeblair | hwoarang: hi! i'm aware of that problem and am looking into it as a zuul bug. i don't have a suggestion for what to do at the moment. | 14:25 |
| *** rtjure has quit IRC | 14:25 | |
| jeblair | fungi, dmsimard: nodepool supports booting from cloud images | 14:25 |
| fungi | oh! when did that land? | 14:25 |
| fungi | i guess that's just in the feature/zuulv3 branch at this stage | 14:25 |
| fungi | i keep forgetting that support made it into v3 | 14:25 |
| dmsimard | The closest I've seen around that was with snapshot-based images, where nodepool would pick up a provider image, do his thing and then snapshot that | 14:25 |
| dmsimard | I'm not sure how to get it to use a not-DIB image though | 14:26 |
| jeblair | couple months ago. tobiash added it. | 14:26 |
| dmsimard | man tobiash rocks, tobiash++ | 14:27 |
| hwoarang | jeblair: ah ok thanks for letting me know. please tell me if you find anything | 14:27 |
| jeblair | dmsimard: what are you using to build images? | 14:27 |
| dmsimard | jeblair: those would be, for example, tripleo undercloud images | 14:27 |
| jeblair | dmsimard: what does tripleo use to build images these days? | 14:29 |
| jeblair | dmsimard: https://docs.openstack.org/infra/nodepool/feature/zuulv3/configuration.html#cloud-images | 14:29 |
| fungi | dmsimard: i think the relevant docs are https://docs.openstack.org/infra/nodepool/feature/zuulv3/configuration.html#provider-cloud-images | 14:29 |
| *** eharney has quit IRC | 14:29 | |
| jeblair | fungi: both of those links, in fact :) | 14:29 |
| dmsimard | jeblair: there's some amount of DIB involved, but the images are built out of the TripleO/RDO CI promotion pipeline | 14:29 |
| fungi | yeah, we raced ;) | 14:29 |
| *** rtjure has joined #openstack-infra | 14:29 | |
| jeblair | fungi: they're different though -- those are both halves of the equation :) | 14:30 |
| fungi | yup | 14:30 |
| *** psachin has quit IRC | 14:30 | |
| jeblair | dmsimard: gotcha | 14:30 |
| dmsimard | jeblair: the hard thing is that we don't necessarily want that image to be re-built by nodepool every day like it typically would, it'd be updated by the CI promotion process. | 14:30 |
| dmsimard | jeblair: the images are built first, then thrown like 20 jobs at it to test it, if it's green, it's promoted and we use it in different places | 14:30 |
| dmsimard | using non-tested images built by nodepool would be awful :D | 14:31 |
| jeblair | dmsimard: i think the nodepool config changes needed to support that can be automated. | 14:31 |
| *** cshastri has quit IRC | 14:31 | |
| jeblair | dmsimard: images can be specified by glance name or id. | 14:32 |
| dmsimard | jeblair, fungi: thanks for the help, I guess https://docs.openstack.org/infra/nodepool/ is not zuul/v3 branch so that's why I couldn't find it | 14:32 |
| fungi | indeed. looking forward to when we reach the point where we can merge back to master | 14:32 |
| jeblair | dmsimard: yep | 14:32 |
| dmsimard | jeblair: yup that sounds great, thanks | 14:32 |
| *** rtjure has quit IRC | 14:34 | |
| *** hamzy has quit IRC | 14:35 | |
| AJaeger | clarkb, fungi, jeblair : Ready to remove zuulv2 from project-config? we first need https://review.openstack.org/513910 to merge - and then recheck and approve https://review.openstack.org/507180 | 14:37 |
| *** slaweq_ has quit IRC | 14:37 | |
| thingee | Hi infra, I've been trying to get the contributor guide post jobs to go. The last review I pushed through https://review.openstack.org/#/c/514461/ showed up on status zuul under post jobs but I never saw anything in the console log http://zuulv3.openstack.org/static/stream.html?uuid=65d05dbee99c42fa889bb4196b563d60&logfile=console.log | 14:38 |
| *** rtjure has joined #openstack-infra | 14:39 | |
| AJaeger | thingee: the logs are up - and as I said earlier today on #openstack-docs: Looks like you hit the few minutes our CI was broken | 14:39 |
| jeblair | thingee: there's a broken console log server somewhere, so there's a 10 or 20% chance a job will land on it | 14:39 |
| AJaeger | thingee: the log files are available | 14:40 |
| AJaeger | thingee: let'S merge https://review.openstack.org/514545 - and test again | 14:40 |
| *** slaweq_ has joined #openstack-infra | 14:40 | |
| thingee | AJaeger, jeblair thanks for the information | 14:41 |
| thingee | AJaeger: ok we'll see if this one works :) | 14:41 |
| *** gouthamr has joined #openstack-infra | 14:42 | |
| *** xarses has joined #openstack-infra | 14:42 | |
| AJaeger | speaking about publishing, project-config-core, please review https://review.openstack.org/#/c/514048/ and https://review.openstack.org/#/c/514053/ | 14:42 |
| *** rtjure has quit IRC | 14:44 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove project-team-guide-publish https://review.openstack.org/514054 | 14:44 |
| *** eharney has joined #openstack-infra | 14:44 | |
| hwoarang | AJaeger: proposal bot works fine for us now :) | 14:46 |
| hwoarang | thank you all for fixingit | 14:46 |
| hwoarang | *fixing it | 14:46 |
| *** slaweq has quit IRC | 14:46 | |
| AJaeger | hwoarang: team work as usual - thanks for your help as well. Glad to hear it works! | 14:47 |
| frickler | AJaeger: do you have a patch to test https://review.openstack.org/#/c/514048/ with? I'm fine with merging it but would feel better if we could test it afterwards | 14:47 |
| *** rtjure has joined #openstack-infra | 14:49 | |
| AJaeger | frickler: no, don't have - but this is similar to all the other ones, so hope it works ;) | 14:49 |
| *** sree has quit IRC | 14:50 | |
| *** tosky has joined #openstack-infra | 14:50 | |
| AJaeger | frickler: most of the converted publish jobs were broken, mordred and I have tested the general framework heavily now... | 14:51 |
| *** rtjure has quit IRC | 14:53 | |
| *** armaan has quit IRC | 14:55 | |
| *** spectr has quit IRC | 15:00 | |
| *** rtjure has joined #openstack-infra | 15:00 | |
| *** ccamacho has quit IRC | 15:02 | |
| johnsom | Looking for a second core review on https://review.openstack.org/#/c/513781/ to remove the duplicate (migrated) zuul jobs. Thanks. | 15:02 |
| *** hemna_ has quit IRC | 15:02 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Use native v3 publishing for project-team-guide https://review.openstack.org/514053 | 15:03 |
| clarkb | AJaeger: jeblair re johnsom's change, should those pipeline chnges be in octavia too? | 15:03 |
| *** salv-orlando has quit IRC | 15:03 | |
| *** salv-orlando has joined #openstack-infra | 15:04 | |
| AJaeger | clarkb: yes - but those other repos have not done the merge yet. So, I think it's fine for now. | 15:05 |
| *** rtjure has quit IRC | 15:05 | |
| johnsom | Thanks folks. If there is a change that we will need in the near future you can give me a pointer and I will try to track it as well. | 15:06 |
| dhellmann | we're still seeing py35 jobs running for patches in the openstack/releases repo even though those patches only change data files. Can someone help with figuring out why that is the case? http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/projects.yaml#n26858 | 15:06 |
| dhellmann | https://review.openstack.org/#/c/512792/ is an example of a patch running check jobs I think it shouldn't need to | 15:07 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Build native v3 install-guide docs https://review.openstack.org/514048 | 15:07 |
| *** salv-orlando has quit IRC | 15:08 | |
| *** dbecker has quit IRC | 15:09 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Migrate legacy jobs https://review.openstack.org/512637 | 15:10 |
| *** hichihara has quit IRC | 15:10 | |
| *** rtjure has joined #openstack-infra | 15:10 | |
| *** slaweq_ has quit IRC | 15:13 | |
| *** rtjure has quit IRC | 15:15 | |
| *** bexelbie_ has quit IRC | 15:16 | |
| openstackgerrit | Merged openstack-infra/project-config master: paunch: fix multinode gate job https://review.openstack.org/514671 | 15:18 |
| *** tosky has quit IRC | 15:19 | |
| sshnaidm | hi, we have a problem with a job that is configured to run on rh1 cloud, but it runs on rax cloud - https://github.com/openstack-infra/project-config/blob/78cc1f28045395b8b6f749a29a866a7a960fa2d9/jenkins/jobs/projects.yaml#L13107 | 15:19 |
| *** rtjure has joined #openstack-infra | 15:20 | |
| sshnaidm | ansible host is on rax cloud: http://logs.openstack.org/42/507642/4/experimental-tripleo/legacy-tripleo-ci-centos-7-ovb-ha-tempest-oooq/123ea18/zuul-info/host-info.primary.yaml | 15:20 |
| sshnaidm | pabelanger, ^^ | 15:20 |
| *** slaweq has joined #openstack-infra | 15:20 | |
| dmsimard | sshnaidm: that's a jenkins v2 definition.. | 15:20 |
| pabelanger | sshnaidm: you should be using tripelo-centos-7 as your label | 15:20 |
| pabelanger | that image is pinned to tripleo-test-cloud-rh1 | 15:21 |
| dhellmann | fungi, jeblair : I'm seeing 221 errors when we try to send release announcement emails. Did our mail sending setup change as part of the zuul upgrade, too? http://logs.openstack.org/45/453cf8167a15f697679658f7b7eba358eba853d4/release/oslo.config-announce-release/5d3e9eb/console.html | 15:21 |
| dmsimard | https://github.com/openstack-infra/openstack-zuul-jobs/blob/master/zuul.d/zuul-legacy-jobs.yaml#L10498 | 15:21 |
| *** hamzy has joined #openstack-infra | 15:21 | |
| sshnaidm | pabelanger, I do, take a look at first link | 15:21 |
| fungi | dhellmann: i've seen announcements go to the release announcements list. are you getting those errors consistently? | 15:22 |
| dhellmann | fungi : none of the oslo releases we just did were announced | 15:22 |
| pabelanger | sshnaidm: that is JJB, for zuulv2 | 15:22 |
| *** LindaWang has quit IRC | 15:22 | |
| pabelanger | sshnaidm: need to look at zuulv3 job | 15:22 |
| *** mlavalle has joined #openstack-infra | 15:22 | |
| fungi | dhellmann: weird, but a bunch of others did. wondering if there's something breaking on the listserv instead. taking a look when i get off the conference call i'm in | 15:23 |
| dhellmann | fungi : although a neutron-lib release from earlier did go through | 15:23 |
| *** ccamacho has joined #openstack-infra | 15:23 | |
| dhellmann | I've only see the 1 email work today | 15:23 |
| clarkb | maybe they ere going into moderation | 15:23 |
| pabelanger | http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/zuul-legacy-jobs.yaml#n10498 | 15:23 |
| pabelanger | that is using legacy-centos-7, which is correct | 15:24 |
| pabelanger | let me see what is going on | 15:24 |
| *** rtjure has quit IRC | 15:24 | |
| pabelanger | ah | 15:24 |
| pabelanger | I see the issue | 15:24 |
| mlavalle | fungi, pabelanger clarkb : question, do you have an example of a project legacy job that has been migrated? | 15:24 |
| *** wolverineav has quit IRC | 15:25 | |
| pabelanger | http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/nodesets.yaml#n116 is setup to use label: centos-7, which won't work for OVB | 15:25 |
| *** wolverineav has joined #openstack-infra | 15:25 | |
| pabelanger | sshnaidm: ^so we need to switch them to tripleo-centos-7 nodeset on all OVB jobs | 15:25 |
| *** dhinesh has joined #openstack-infra | 15:25 | |
| frickler | fungi: dhellmann: the log looks clean to me, the 221 comes after send-mail sends "quit". this would be the msg-id to track on the listserv: reply: b'250 OK id=1dzkMZ-0007Dt-Gu\r\n' | 15:26 |
| sshnaidm | pabelanger, I don't understand, we have tripleo-centos-7 in all vb jobs, all jobs work fine besides this one | 15:26 |
| dhellmann | clarkb : I'm working on getting logged in to check that. | 15:26 |
| jlvillal | A couple of review requests: For Gerrit patch pages: Add result_TIMED_OUT & result_RETRY_LIMIT as failure colors https://review.openstack.org/#/c/513444/ | 15:26 |
| pabelanger | sshnaidm: which other job? | 15:26 |
| *** hemna_ has joined #openstack-infra | 15:26 | |
| jlvillal | And: Support the log level "CRITICAL" https://review.openstack.org/#/c/509670/ | 15:26 |
| dhellmann | fungi : hrm, ok. so where's the email? | 15:26 |
| sshnaidm | pabelanger, from my first link: https://github.com/openstack-infra/project-config/blob/78cc1f28045395b8b6f749a29a866a7a960fa2d9/jenkins/jobs/projects.yaml#L13107 | 15:27 |
| sshnaidm | pabelanger, its name is legacy-tripleo-ci-centos-7-ovb-ha-tempest-oooq | 15:27 |
| sshnaidm | pabelanger, and it runs in experimental-tripleo | 15:27 |
| pabelanger | sshnaidm: no, you are looking in the wrong folder, anything in jenkins/jobs is about to get deleted, you should be using openstack-zuul-jobs/zuul.d now | 15:27 |
| AJaeger | Let'S remove those files ;) we first need https://review.openstack.org/513910 to merge - and then recheck and approve https://review.openstack.org/507180 | 15:28 |
| pabelanger | sshnaidm: can you post a log to working OVB job | 15:28 |
| sshnaidm | pabelanger, before migration to zuul it worked fine | 15:28 |
| pabelanger | sshnaidm: yes, there is a regression in zuulv3 jobs, is what I am saying | 15:28 |
| sshnaidm | pabelanger, http://logs.openstack.org/38/512638/2/check-tripleo/legacy-tripleo-ci-centos-7-ovb-ha-oooq/af0e14a/job-output.txt.gz | 15:28 |
| pabelanger | sshnaidm: let me post a patch | 15:28 |
| *** tosky has joined #openstack-infra | 15:28 | |
| *** dhajare has joined #openstack-infra | 15:29 | |
| pabelanger | yah, I see the issue | 15:30 |
| *** dhinesh has quit IRC | 15:30 | |
| fungi | dhellmann: still on a conference call, but i'll look at the listserv when it's over | 15:30 |
| *** rtjure has joined #openstack-infra | 15:30 | |
| dhellmann | fungi , clarkb : the moderator queue for the list is empty, so that's not it | 15:30 |
| dhellmann | fungi : ack, thanks | 15:30 |
| AJaeger | pabelanger: what do you think about https://review.openstack.org/514526 to silence some ansible-lint things? | 15:31 |
| *** slaweq_ has joined #openstack-infra | 15:31 | |
| * AJaeger will be back later | 15:31 | |
| sshnaidm | pabelanger, nodeset: legacy-tripleo-centos-7 ? | 15:32 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: Run legacy-tripleo-ci-centos-7-ovb-ha-tempest-oooq on tripleo-rh1 https://review.openstack.org/514718 | 15:32 |
| pabelanger | sshnaidm: ^ | 15:32 |
| pabelanger | yah | 15:32 |
| sshnaidm | pabelanger, it was originally with timeout 210 mins | 15:33 |
| sshnaidm | pabelanger, 12600 secs | 15:33 |
| *** rossella_s has quit IRC | 15:33 | |
| *** Goneri has quit IRC | 15:34 | |
| pabelanger | we've cap everything at 10800 for zuulv3 | 15:34 |
| *** rtjure has quit IRC | 15:35 | |
| sshnaidm | pabelanger, can we do exception for this experimental job? | 15:35 |
| clarkb | dhellmann: re the irrelevant files I've just double checked the regex against those file paths and it seems like they match. Will have to dig in more | 15:35 |
| dhellmann | clarkb : thanks. I'm stumped on that one. | 15:35 |
| *** slaweq_ has quit IRC | 15:36 | |
| pabelanger | sshnaidm: have to discuss it at weekly meeting, but currently job should likely be refactored to address length of time. A 3.5 hour jobs in pipelines, doesn't really scale well | 15:36 |
| clarkb | dhellmann: I wonder if it is because the base openstack-tox-py35 job has its own irrelevant files list and that is winning over this variants list | 15:36 |
| clarkb | jeblair: ^ | 15:36 |
| sshnaidm | pabelanger, ack | 15:36 |
| jeblair | clarkb: i'm a couple problems deep at this point. can we put it on the etherpad for later? | 15:37 |
| dhellmann | clarkb : maybe? I'm not sure of the precedence order there. | 15:37 |
| clarkb | jeblair: ya | 15:37 |
| dhellmann | yeah, this is a low priority one | 15:37 |
| sshnaidm | pabelanger, so from now I need to look at openstack-zuul-jobs/ zuul.d/zuul-legacy-jobs.yaml ? | 15:37 |
| pabelanger | sshnaidm: right, until those jobs are migrated into tripleo repos | 15:38 |
| dhellmann | clarkb , fungi : another email job failed with "too many concurrent smtp connections", so maybe that's related to the other failure? http://logs.openstack.org/b6/b6800279e4a73afd485548810dd99a35fc757d30/release/announce-release/39645c7/job-output.txt.gz | 15:39 |
| dhellmann | I'll put that on the etherpad, too | 15:39 |
| *** ijw has joined #openstack-infra | 15:39 | |
| *** yamamoto has quit IRC | 15:40 | |
| *** rtjure has joined #openstack-infra | 15:41 | |
| fungi | dhellmann: yeah, my gut says we're seeing a volume issue on the listserv, likely entirely unrelated to release jobs. we send one e-mail per job right? | 15:42 |
| dhellmann | fungi : yes, just 1 per release | 15:42 |
| *** mummer has joined #openstack-infra | 15:43 | |
| *** e0ne has quit IRC | 15:43 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Migrate legacy jobs for feature/zuulv3 branch https://review.openstack.org/513766 | 15:44 |
| fungi | the 221 from http://logs.openstack.org/45/453cf8167a15f697679658f7b7eba358eba853d4/release/oslo.config-announce-release/5d3e9eb/console.html should be normal behavior. that's the smtp code for "Closing connection (Mail received successfully, Goodbye)." | 15:44 |
| dhellmann | fungi : I noticed I didn't receive email from a wiki update ttx made earlier, too, so maybe that's related as well | 15:44 |
| dhellmann | oh, my google search showed an error message for 221 related to authorization | 15:44 |
| *** dhill_ has joined #openstack-infra | 15:44 | |
| dhellmann | I guess I shouldn't trust things I find on the internet :-) | 15:44 |
| clarkb | fungi: ya thats why I wondered if they were ending up in moderation, but could be separate lists issue having them stuck in limbo? | 15:45 |
| * clarkb breakfasts then is back to help the issues list/debugging/meeting prep | 15:45 | |
| *** rtjure has quit IRC | 15:46 | |
| fungi | dhellmann: technically rfc 2821 defines the 221 code as "Service closing transmission channel" but it's not necessarily considered an error condition (especially when followed by a 250 "Requested mail action okay, completed" as part of normal session termination) | 15:47 |
| fungi | er, especially when _following_ | 15:47 |
| dhellmann | makes sense | 15:47 |
| *** hemna_ has quit IRC | 15:48 | |
| fungi | dhellmann: is this the message you were looking for? seems to match that job log: http://lists.openstack.org/pipermail/release-announce/2017-October/002772.html | 15:49 |
| fungi | that's a log and message from 20 days ago though | 15:49 |
| *** felipemonteiro__ has quit IRC | 15:49 | |
| dhellmann | fungi : yeah, that's an old one | 15:49 |
| fungi | were you linking it as a pre-v3 example then? | 15:50 |
| dhellmann | let me find a thing that was tagged | 15:50 |
| fungi | looked like you were asking why it didn't go through to the list | 15:50 |
| dhellmann | castellan 0.14.1 | 15:51 |
| *** ccamacho has quit IRC | 15:51 | |
| fungi | thanks, i'll go find the job log | 15:51 |
| *** slaweq has quit IRC | 15:51 | |
| dhellmann | fungi : oslo.config 4.13.2 was the tag | 15:51 |
| *** slaweq has joined #openstack-infra | 15:51 | |
| dhellmann | so I got the wrong one | 15:51 |
| dhellmann | and I don't see the 4.13.2 tag on the oslo.config repo | 15:52 |
| dhellmann | I didn't notice those numbers didn't match | 15:52 |
| melwitt | does anyone know if the newer zuul jobs are available in graphite.o.o anywhere? I have a graph link that is no longer showing new failure rates because it's referring to old jenkins jobs | 15:52 |
| fungi | okay, so looking at castellan 0.14.1 seems to be http://logs.openstack.org/8d/8dd77e745f1cd0fb82609d35e8c5a37edeba7440/release/announce-release/a4fedec/ | 15:52 |
| *** rtjure has joined #openstack-infra | 15:52 | |
| *** caphrim007 has quit IRC | 15:53 | |
| fungi | dhellmann: yep, the castellan tag hit a 421 "Too many concurrent SMTP connections; please try again later." | 15:53 |
| *** yolanda has quit IRC | 15:54 | |
| *** hashar is now known as hasharAway | 15:54 | |
| *** caphrim007 has joined #openstack-infra | 15:54 | |
| clarkb | melwitt: they should be now, but I think some of the paths may have changed. Let me find examples or docs on that | 15:54 |
| fungi | dhellmann: i have a feeling there's a flood hitting the listserv right now, probably getting rejected as spam to owner addresses | 15:54 |
| dhellmann | yeah | 15:54 |
| *** Guest69866 has quit IRC | 15:54 | |
| dhellmann | so the next thing to figure out is why the oslo tag job didn't actually tag all of the releases | 15:55 |
| openstackgerrit | Jens Harbott (frickler) proposed openstack-infra/project-config master: Use docs tox environment for contributor-guide publishing https://review.openstack.org/514726 | 15:55 |
| dhellmann | http://logs.openstack.org/7d/7d3a7e81f9a4d4abb1bec7d5f405ab06ba397f08/release-post/tag-releases/9abec8b/job-output.txt.gz shows it tagging castellan then stopping | 15:55 |
| frickler | project-config-core: ^^ small fix for thingee ;) | 15:56 |
| clarkb | melwitt: check out https://review.openstack.org/#/c/513950/ a big difference seems to be that things are scoped to zuul tenant now | 15:56 |
| *** andreas_s has quit IRC | 15:56 | |
| * med_ giggles a bit about "listserv" as he hasn't heard that term in a long time | 15:56 | |
| *** scottda_ has joined #openstack-infra | 15:56 | |
| fungi | med_: i'd rather we made them into newsgroups | 15:57 |
| fungi | but the times they are a' changing | 15:57 |
| clarkb | frickler: thingee we've pushed back on that in the past to help make sure some version of "normal upstraem sphinx doc generation tooling" works without special sauce | 15:57 |
| clarkb | frickler: thingee in our case that means we run python setup.py build_sphinx in a plain virtualenv | 15:57 |
| *** rtjure has quit IRC | 15:57 | |
| *** ihrachys has joined #openstack-infra | 15:58 | |
| fungi | frickler: also, see https://review.openstack.org/508694 which merged today | 15:58 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: remove some debugging for release git clone https://review.openstack.org/514729 | 15:58 |
| pabelanger | frickler: clarkb: Right, i don't think tox -edocs proper for PTI | 15:59 |
| *** nunchuck has quit IRC | 15:59 | |
| clarkb | frickler: thingee I see hte bug here is no env is specified at all in the past, as an alternative can you use the playbook on line 156 of https://review.openstack.org/#/c/514726/1/zuul.d/jobs.yaml ? I'll leave a comment on the change | 15:59 |
| pabelanger | We have existing jobs that are doing build_sphinx, and should be easy to use that | 15:59 |
| fungi | tox -edocs wasn't what the pti mandated before, and now the pti mandates no tox at all | 15:59 |
| *** yamahata has quit IRC | 16:00 | |
| fungi | should be made to work with `sphinx-build -b html doc/source doc/build` outside a virtualenv as soon as feasible | 16:00 |
| *** jogo has quit IRC | 16:00 | |
| pabelanger | tox_extra_args: -vv python setup.py build_sphinx | 16:00 |
| *** jeblair has quit IRC | 16:00 | |
| melwitt | thanks clarkb | 16:01 |
| *** jeblair has joined #openstack-infra | 16:01 | |
| *** rtjure has joined #openstack-infra | 16:02 | |
| *** iyamahat has quit IRC | 16:02 | |
| openstackgerrit | Edgar Magana proposed openstack-infra/irc-meetings master: Include a new time for UC IRC meetings https://review.openstack.org/513863 | 16:04 |
| fungi | dhellmann: does this DELIVERABLES= indicate the list of deliverables was exhausted after castellan? http://logs.openstack.org/7d/7d3a7e81f9a4d4abb1bec7d5f405ab06ba397f08/release-post/tag-releases/9abec8b/job-output.txt.gz#_2017-10-24_15_02_04_709450 | 16:04 |
| *** salv-orlando has joined #openstack-infra | 16:05 | |
| frickler | clarkb: fungi: is there a fallback when the project is lacking doc/requirements.txt ? | 16:05 |
| *** amoralej|mtg is now known as amoralej | 16:05 | |
| dhellmann | fungi : that variable can be set as an argument to the script when its run by hand. when it is not set, list_deliverable_changes figures out which deliverables have been included in the commit | 16:05 |
| dhellmann | maybe I should update the script to print out what work it is going to do before it tries to do it | 16:05 |
| clarkb | frickler: the job runs tox -e venv -- python setup.py build_sphinx, I don't think that requires doc/requirements.txt | 16:06 |
| fungi | frickler: i believe the reference implementation backing the change to the pti has not merged yet. in the meantime just add a venv testenv to tox.ini in the project | 16:06 |
| dhellmann | running release_from_yaml.sh locally against that commit in the releases repo does not reproduce the problem (it seems to loop over all of the items) | 16:06 |
| *** rtjure has quit IRC | 16:07 | |
| fungi | dhellmann: and i guess the job log doesn't contain the whole of the output from list_deliverable_changes.py there so hard to know what the job was asked to iterate on besides that first line | 16:07 |
| *** andreas_s has joined #openstack-infra | 16:07 | |
| dhellmann | yeah, it prints the line one at a time | 16:08 |
| dhellmann | I'll update the job to dump all of the data once and then actually do the tagging | 16:08 |
| openstackgerrit | Jens Harbott (frickler) proposed openstack-infra/project-config master: Use the python-docs playbook for contributor-guide publishing https://review.openstack.org/514726 | 16:09 |
| *** salv-orlando has quit IRC | 16:09 | |
| frickler | ^^ o.k., tested locally and seems to work fine this way | 16:09 |
| smatzek | The python-troveclient check and gate have aparently been broken since v3 started doing checks. I've been focused on trove-master and am just now turning an eye to that project. It appears that openstack-zuul-jobs/playbooks/legacy is missing a job for this jenkins job [1]. python-saharaclient does have a similar legacy tempest job. Is the fix here to add a job in playbooks/legacy using sarahaclient's tempest job and the original j | 16:10 |
| smatzek | enkins job as reference, and then update zuul.d/projects.yaml? [1] https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/trove.yaml#L210 | 16:10 |
| *** dtantsur is now known as dtantsur|afk | 16:10 | |
| *** andreas_s has quit IRC | 16:12 | |
| fungi | dhellmann: on the too many concurrent sessions error, it looks like we restrict the mta on lists.o.o to allow no more than 100 inbound smtp connections at a time but we seem to be floating around twice that many concurrent connection attempts | 16:12 |
| clarkb | dmsimard: pabelanger left a comment on https://review.openstack.org/#/c/514489/2 now that I have had a bit of time to think on it | 16:12 |
| clarkb | dmsimard: pabelanger can you let me know what you think? | 16:12 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: have release job show what info it is working with https://review.openstack.org/514733 | 16:12 |
| clarkb | fungi: and generally that is ok beacuse most smtp clients will retry later ya? | 16:12 |
| *** jogo has joined #openstack-infra | 16:12 | |
| *** jogo has joined #openstack-infra | 16:12 | |
| fungi | clarkb: correct | 16:13 |
| *** rtjure has joined #openstack-infra | 16:13 | |
| dhellmann | yeah, we don't retry for these messages because the node gets deleted | 16:13 |
| dmsimard | clarkb: yeah I haven't had time to get back to it yet, will look. | 16:13 |
| clarkb | smatzek: give me a minute to dig in and see what is going on | 16:13 |
| dhellmann | fungi : the patch ^^ has some more details. Can we merge that and re-run tag releases for 7d3a7e81f9a4d4abb1bec7d5f405ab06ba397f08? | 16:13 |
| fungi | if you tail the exim mainlog you'll see we're sustaining several attempts per second to deliver spam from random clients with random @qq.com e-mail addresses mostly all addressed to openstack-announce-owner@ and openstack-docs-owner@ | 16:14 |
| smatzek | clarkb: thanks. a bit more info. A review that is failing: https://review.openstack.org/#/c/500033/, a review where jenkins did the check/gate https://review.openstack.org/#/c/510264/. The gate-tempest jobs are different. | 16:15 |
| fungi | and the spammers are (unsurprisingly) not very courteous about disconnections once we emit an error | 16:15 |
| clarkb | smatzek: legacy-tempest-dsvm-neutron-src is the job that it got replaced with I think | 16:16 |
| clarkb | smatzek: as part of lib-forward-testing | 16:16 |
| dhellmann | fungi : I need to step away for a few minutes. brb | 16:16 |
| clarkb | smatzek: the problem likely being that that new job doesn't install the trove devstack plugin | 16:16 |
| fungi | k | 16:16 |
| *** jascott1 has joined #openstack-infra | 16:17 | |
| smatzek | clarkb: it doesn't, per the log looking I did this morning | 16:17 |
| *** rtjure has quit IRC | 16:18 | |
| smatzek | python-saharaclient has a job similar to what trove had in jenkins that does the plugin enable, https://github.com/openstack-infra/openstack-zuul-jobs/tree/master/playbooks/legacy/tempest-dsvm-neutron-src-python-saharaclient | 16:18 |
| smatzek | so I'm thinking we likely need a job like that for troveclient | 16:18 |
| *** andreas_s has joined #openstack-infra | 16:19 | |
| robcresswell | o/ How do we run legacy jobs against EOL tags? We've had success use override-branch for existing stable branches, but it doesn't work for tags (probably to be expected) | 16:19 |
| clarkb | smatzek: ya i think making one like legacy-tempest-dsvm-neutron-src-python-saharaclient for troveclient is the most straigtforward way to address that | 16:19 |
| robcresswell | Referring to https://review.openstack.org/#/c/514314/1 mainly :) | 16:19 |
| clarkb | robcresswell: I don't think we've ever run jobs against EOL tags? | 16:19 |
| clarkb | robcresswell: since you can't push changes to EOL'd branches | 16:19 |
| smatzek | and then update zuul.d/projects.yaml for python-troveclient ? | 16:19 |
| smatzek | if so, I'll put some reviews up. | 16:20 |
| robcresswell | clarkb: No, but you can run software that depends on EOL branches | 16:20 |
| robcresswell | Or, supports EOL branches I should say | 16:20 |
| clarkb | robcresswell: gotcha so its changes to valid branches that then install EOL'd software? | 16:20 |
| clarkb | smatzek: yup exactly that | 16:20 |
| robcresswell | clarkb: Yeah. networking-cisco supports Mitaka, Newton, Ocata (and we're fixing Pike atm) | 16:21 |
| robcresswell | clarkb: We were running on them all before but the v3 migration broke us. Sam fixed the other tests with https://review.openstack.org/#/c/513339/ but we're stuck on the EOL test. | 16:21 |
| *** lucasagomes is now known as lucas-afk | 16:22 | |
| *** tesseract has quit IRC | 16:22 | |
| *** rtjure has joined #openstack-infra | 16:22 | |
| *** andreas_s has quit IRC | 16:23 | |
| *** iyamahat has joined #openstack-infra | 16:25 | |
| *** pcaruana has quit IRC | 16:25 | |
| clarkb | robcresswell: a really quick read of the zuul code implementing override-branches shows that it only examines the list of branches from what I can tell (its looking at git directly and saying is this thing a branch) | 16:26 |
| clarkb | robcresswell: so I don't think override-branches is going to do what you want | 16:26 |
| clarkb | robcresswell: instead you will likely need to add a playbook task that checks out the desired ref in the test repos in your job | 16:26 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Fix branch ordering on dynamic reconfiguration https://review.openstack.org/514744 | 16:27 |
| robcresswell | clarkb: Good to know, thankyou :) | 16:27 |
| *** rtjure has quit IRC | 16:27 | |
| clarkb | robcresswell: https://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/executor/server.py?h=feature/zuulv3#n755 is where it examines the branches fwiw | 16:27 |
| jeblair | hwoarang: ^ i think 514744 is a fix for the bug you saw. mnaser you may have run into this too. | 16:27 |
| *** Apoorva has joined #openstack-infra | 16:28 | |
| jeblair | hwoarang, mnaser: it's non-deterministic; could do something different each run. | 16:28 |
| clarkb | jeblair: see robcresswell's thing we might considering a more relaxed override directive that operates on any valid ref | 16:29 |
| jeblair | fungi, clarkb, dhellmann: now that we have secrets, we can have the job smtp auth and get better treatment | 16:29 |
| openstackgerrit | Merged openstack-infra/project-config master: have release job show what info it is working with https://review.openstack.org/514733 | 16:29 |
| jeblair | clarkb, robcresswell: ah this is the ref issue? | 16:29 |
| dhellmann | jeblair : oh, that would be good | 16:29 |
| clarkb | jeblair: basically they want override-branch to operate on tags but https://git.openstack.org/cgit/openstack-infra/zuul/tree/zuul/executor/server.py?h=feature/zuulv3#n755 seems to force it to only work on branches? | 16:30 |
| fungi | jeblair: oh, excellent idea | 16:30 |
| robcresswell | jeblair: Its just how branch overrides are currently handled; we have requirements for tags as well as branches | 16:30 |
| jeblair | clarkb, robcresswell: yeah, i remember reviewing a change like this yesterday | 16:30 |
| jeblair | i guess it didn't work | 16:30 |
| fungi | dhellmann: ready for me to reenqueue 7d3a7e81f9a4d4abb1bec7d5f405ab06ba397f08 into release-post? | 16:31 |
| clarkb | override-ref maybe in addition to override-branch? or maybe we just make backward incompatbile changes really quicklly while we can | 16:31 |
| jeblair | clarkb: i'm going to feel free to make backwards incompatible changes until we release it. :) | 16:31 |
| robcresswell | I dont know if you're familiar with NPM but they have what they call "commit-ish" refs (please dont scream, I swear its useful) that lets you do things like specific commits, branches, or tags :) | 16:32 |
| robcresswell | https://docs.npmjs.com/files/package.json#git-urls-as-dependencies That sort of thing. | 16:33 |
| clarkb | robcresswell: ya that would be a generic ref | 16:33 |
| clarkb | robcresswell: basically anything that git knows about in its refs | 16:33 |
| *** shardy has quit IRC | 16:33 | |
| robcresswell | Yup | 16:33 |
| robcresswell | Just thought that approach may be valuable. I'd imagine newton-eol will cause others issues soon enough | 16:33 |
| robcresswell | Perhaps we're just the only people "lucky" enough to be attempting to support Mitaka on upstream infra :) | 16:34 |
| clarkb | part of me doesn't want to encourage the idea that we are supporting EOL releases but ignoring that I can see this being useful with stuff like tempest release tags | 16:34 |
| jeblair | yeah, i mean, we're having this conversation because openstack-infra literally does not support that software any more :) | 16:34 |
| robcresswell | I can understand that | 16:34 |
| sambetts | robcresswell: you mean insane enough ;) | 16:34 |
| robcresswell | Ha, yeah | 16:35 |
| *** jascott1 has quit IRC | 16:35 | |
| jeblair | otoh, wearing my zuul maintainer hat, i am unconstrained by that policy and am willing to consider making the software do what its users want. | 16:35 |
| ianychoi | Hello! I am not still sure whether here is a right place or not, but infra team can help add one launchpad member in one launchpad core team? I would like to add Frank (current I18n PTL) on https://launchpad.net/~openstack-i18n-core but it is difficult to communicate with the owner of the team. | 16:35 |
| robcresswell | jeblair: Yeah, its a weird one because its basically a regression test | 16:35 |
| *** dhinesh has joined #openstack-infra | 16:35 | |
| robcresswell | Well, thats exactly what it is. | 16:36 |
| *** rtjure has joined #openstack-infra | 16:36 | |
| robcresswell | We're not adding new content, just making sure existing fixes / features dont break old stuff, which happens to be Mitaka while we try and shift away from it. | 16:36 |
| clarkb | ianychoi: unfortunately I don't think there is much we can do. You might be able to get the launchpad admins to help somehow, but easiest thing is likely having daisy update someone else to have administrator access | 16:36 |
| jeblair | so yeah, i'm oppen to adding support for this in zuul. i'm not sure whether we should support it in 'override-branch' or 'override-ref'. | 16:36 |
| ianychoi | clarkb, thanks a lot for your quick reply! Great comment :) | 16:37 |
| *** janki has quit IRC | 16:37 | |
| clarkb | jeblair: I'm tempted to say moving everything to override-ref and removing branch makes the most sense (but is possibly the most work too) | 16:37 |
| jeblair | i think the question is: would a user ever be surprised that 'override-branch: some-tag-name' worked instead of falling back. | 16:37 |
| sambetts | I assumed it would work because that was the bahviour in devstack-gate if you set the override branch | 16:37 |
| clarkb | sambetts: ya I think devstack-gate was just doing a direct checkout $name | 16:38 |
| jeblair | i think it was more accidental there -- it wasn't very smart | 16:38 |
| clarkb | sambetts: but zuul is now listing the branches and checking the override branch is in that list | 16:38 |
| clarkb | jeblair: exactly | 16:38 |
| sambetts | yeah, just wondering if I would have made the assumtion it would work if I didn't have prior knowledge of how devstack-gate operated | 16:39 |
| *** vsaienk0 has quit IRC | 16:39 | |
| jeblair | clarkb: yeah, i'm inclined to think the option name should actually describe what it does. so if we support both, it should be override-ref. | 16:39 |
| jeblair | clarkb: that moves the question to: should we also keep override-branch? | 16:39 |
| jeblair | this may be worth an infra ml post. | 16:40 |
| jeblair | see what other people want out of it. | 16:40 |
| clarkb | jeblair: In the past we've said don't have tags and branches with the same name because it causes all sorts of problems | 16:40 |
| fungi | does override-branch still retain fallback mechanics that a literal non-branch ref might lack? | 16:40 |
| clarkb | jeblair: potentially if you had override-ref and override-branch you could support ^ | 16:40 |
| *** jpich has quit IRC | 16:40 | |
| jeblair | fungi: i think they would both fallback in the same manner | 16:40 |
| clarkb | jeblair: but that sounds like a headache to test and debug and is such a corner case I'm not sure it is useful | 16:41 |
| jeblair | clarkb: true. i wouldn't want to be in that position, but someone else might. :) | 16:41 |
| *** yamamoto has joined #openstack-infra | 16:41 | |
| sambetts | I would suggest that make people do ref/tags for tags to prevent clashes with branch names | 16:41 |
| *** rtjure has quit IRC | 16:41 | |
| sambetts | the code already does that for the zuul ref | 16:41 |
| *** ramishra has quit IRC | 16:41 | |
| sambetts | but it depends if it'll be a pain for users | 16:42 |
| fungi | just thinking through how to, e.g., allow a project who has retained their stable/mitaka branch to test against mitaka-eol tags of some projects but stable/mitaka branches of other projects who have also not eol'd | 16:42 |
| sambetts | set the per-project override ? | 16:42 |
| jeblair | sambetts: yeah, i don't think i want to have 'refs/heads' or whatever in the config language. and i've been in a position where branches lacked 'refs/heads' but tags had 'refs/tags' and that's annoying. :) | 16:42 |
| openstackgerrit | Merged openstack/python-jenkins master: Sanitize url in get_info() function https://review.openstack.org/504536 | 16:43 |
| sambetts | jeblair: yeah I thought that might be annoying | 16:43 |
| smcginnis | fungi: FWIW, we do have an issue in the driverfixes branches that distros want to run unit tests on those changes but there is requirements branch to get the right constraints from. | 16:43 |
| *** yamahata has joined #openstack-infra | 16:43 | |
| jeblair | fungi: oh interesting. so have 'override-branch: mitaka; override-ref:mitaka-eol' and count on some kind of precedence to get you one or the other of those? | 16:43 |
| *** vsaienk0 has joined #openstack-infra | 16:43 | |
| *** Swami has joined #openstack-infra | 16:43 | |
| fungi | yeah, it's fuzzy | 16:43 |
| clarkb | fungi: updated etherpad with the idea of using authenticated smtp to get around connection limits | 16:44 |
| jeblair | fungi: i probably would have accidentally implemented it that way anyway. | 16:44 |
| sambetts | regex? | 16:44 |
| fungi | smcginnis's driverfixes/mitaka vs mitaka-eol tag in requirements is likely a better concrete example | 16:44 |
| jeblair | keep in mind that you can specify per-project overrides | 16:44 |
| clarkb | jeblair: fungi you could still implement it that way with project specific overrides using just override-ref | 16:44 |
| clarkb | ya that | 16:44 |
| jeblair | so while it's a bit more work to keep the job up to date, it is possible (and more explicit) | 16:45 |
| fungi | ahh, okay. i just hadn't happened across an example of that yet, but i guess you can do that for each required-projects entry which needs it? | 16:45 |
| jeblair | fungi: yep | 16:45 |
| jeblair | so when a req-project drops their branch and goes eol, you could (if we implement override-ref) switch it then | 16:45 |
| *** rtjure has joined #openstack-infra | 16:45 | |
| *** andreas_s has joined #openstack-infra | 16:46 | |
| jeblair | sambetts, robcresswell: in the mean time, is there an opportunity for you to run some shell or ansible at the start of your job to check out the eol tags? | 16:46 |
| fungi | noting that zuul can't (and i guess shouldn't expect to) merge any proposed changed with an override-ref while it could on an override-branch | 16:46 |
| jeblair | sambetts, robcresswell: they will be there in the repo. | 16:46 |
| *** hemna_ has joined #openstack-infra | 16:47 | |
| *** martinkopec has quit IRC | 16:47 | |
| jeblair | sambetts, robcresswell: this might be an interim step you could take until zuul supports the tag checkout. | 16:47 |
| sambetts | jeblair: thats kind of how we used to do it with the zuul-cloner, but we had logic to prevent it reinstalling again if its already installed | 16:47 |
| robcresswell | jeblair: I think so, that's what we were discussing anyway | 16:47 |
| sambetts | so we can try to fix that in the meantime | 16:47 |
| robcresswell | I just wanted to clarify / raise it upstream before we charged ahead | 16:48 |
| sambetts | yeah we were hoping to avoid hacks if it was possible ;) | 16:48 |
| robcresswell | It didn't seem like too unlikely a scenario for others either, given the glacial pace of some upgrades, so it seemed worth an open discussion | 16:48 |
| * sambetts has a habit of overengineering things when he could just set a config option | 16:48 | |
| jeblair | robcresswell, sambetts: at the best, i'd give this a couple days for us to fix it, but i think it might be behind some other fixes, so it may be looking more like a week or so. | 16:48 |
| *** dhinesh has quit IRC | 16:49 | |
| robcresswell | jeblair: Understood, really appreciate you looking into it | 16:49 |
| sambetts | Yeah thanks! | 16:49 |
| jeblair | robcresswell: thanks for bringing it up! | 16:49 |
| robcresswell | If we can help with testing or more info on use case, please ping. | 16:49 |
| robcresswell | jeblair: No problem :) | 16:49 |
| *** trown is now known as trown|lunch | 16:50 | |
| *** rtjure has quit IRC | 16:50 | |
| *** pvaneck has joined #openstack-infra | 16:51 | |
| *** dhinesh has joined #openstack-infra | 16:51 | |
| jeblair | robcresswell, sambetts: oh one question -- to clarify, was it devstack-gate that supported checking out tags before, or zuul-cloner, or both? | 16:51 |
| *** rkukura has joined #openstack-infra | 16:51 | |
| sambetts | jeblair: both, https://github.com/openstack/networking-cisco/blob/master/tools/tox_install.sh#L35 is how we used to do it in our tox env before ansible started doing it | 16:52 |
| *** derekh has quit IRC | 16:52 | |
| jeblair | sambetts: thx | 16:52 |
| sambetts | jeblair: and in our third party CI we set the override branch | 16:52 |
| *** jascott1 has joined #openstack-infra | 16:53 | |
| sambetts | jeblair: and I think someone must of fixed the EOL tag thing in devstack-gate because I recently rebuilt our third party CI after a longish down time and had to add https://review.openstack.org/#/c/512245/ to make it work like it used to | 16:55 |
| sambetts | and https://review.openstack.org/#/c/510946 | 16:55 |
| *** rtjure has joined #openstack-infra | 16:55 | |
| sambetts | very straight forward patches but nothing is really built with EOL tags in mind :/ | 16:55 |
| sambetts | or just tags in general | 16:55 |
| sambetts | so even pinning to a release tag doesn't work | 16:56 |
| *** jtomasek has quit IRC | 16:56 | |
| fungi | dhellmann: http://logs.openstack.org/7d/7d3a7e81f9a4d4abb1bec7d5f405ab06ba397f08/release-post/tag-releases/a7a85ec/job-output.txt.gz#_2017-10-24_16_52_46_560403 | 16:56 |
| *** jascott1 has quit IRC | 16:57 | |
| *** yamamoto has quit IRC | 16:57 | |
| fungi | all i can figure is that the while loop read to the end of the output rather than line-by-line | 16:59 |
| *** andreas_s has quit IRC | 16:59 | |
| *** jascott1 has joined #openstack-infra | 16:59 | |
| *** rtjure has quit IRC | 17:00 | |
| smcginnis | fungi: Trying to catch up between sessions. Is the issue it didn't read in all the projects correctly, so not all got processed? | 17:00 |
| *** baoli has quit IRC | 17:00 | |
| *** sambetts is now known as sambetts|afk | 17:00 | |
| sambetts|afk | night all o/ thanks jeblair | 17:00 |
| *** tosky is now known as re | 17:00 | |
| fungi | smcginnis: from the log, it looks like it only does one pass through that while loop, yes | 17:00 |
| openstackgerrit | Vladyslav Drok proposed openstack-infra/project-config master: Run pep8 job instead of py27 https://review.openstack.org/514610 | 17:00 |
| *** re is now known as tosky | 17:00 | |
| *** felipemonteiro_ has joined #openstack-infra | 17:01 | |
| jeblair | clarkb: if we go with the idea of just having one option, we could call it 'override-checkout'. | 17:01 |
| *** baoli has joined #openstack-infra | 17:01 | |
| *** jascott1 has quit IRC | 17:02 | |
| *** felipemonteiro__ has joined #openstack-infra | 17:02 | |
| *** dhajare has quit IRC | 17:02 | |
| inc0 | hey, why run.yml is not being run here? | 17:03 |
| inc0 | https://review.openstack.org/#/c/513855/5 | 17:03 |
| *** jascott1 has joined #openstack-infra | 17:03 | |
| *** andreas_s has joined #openstack-infra | 17:04 | |
| clarkb | jeblair: ++ that seems clear what it does | 17:04 |
| clarkb | "checkout this name" | 17:04 |
| pabelanger | inc0: chances are you have a syntax error in your playbook. Let me check on executor | 17:04 |
| *** dhill_ has quit IRC | 17:06 | |
| *** felipemonteiro_ has quit IRC | 17:06 | |
| inc0 | ahh I think I got it | 17:06 |
| inc0 | yeah pabelanger it was syntax error, sorry | 17:06 |
| openstackgerrit | Samuel Matzek proposed openstack-infra/openstack-zuul-jobs master: Add legacy tempest troveclient job https://review.openstack.org/514768 | 17:06 |
| inc0 | I was so focused on .zuul.yml | 17:06 |
| pabelanger | inc0: the plan is to bubble those errors up to jobs, pretty sure we have open issue on zuulv3-issues etherpad, but do you mind confirming and maybe add it if missing | 17:07 |
| inc0 | will do, thanks pabelanger | 17:08 |
| *** andreas_s has quit IRC | 17:08 | |
| dhellmann | fungi, smcginnis : I wonder if this is a bash behavioral change between versions? is there some way to tell what version is/was present on that node? | 17:09 |
| *** vsaienk0 has quit IRC | 17:09 | |
| *** Apoorva_ has joined #openstack-infra | 17:10 | |
| fungi | dhellmann: looking at the log even more closely, it seems the while did prompt a subsequent read but decided there was no remaining data? http://logs.openstack.org/7d/7d3a7e81f9a4d4abb1bec7d5f405ab06ba397f08/release-post/tag-releases/a7a85ec/job-output.txt.gz#_2017-10-24_16_52_49_695727 | 17:10 |
| *** rtjure has joined #openstack-infra | 17:11 | |
| *** dhill_ has joined #openstack-infra | 17:11 | |
| dhellmann | fungi : yeah, that's the eof behavior, I think? maybe I should use a temporary file there. I wonder if the python script is dying for some reason. | 17:11 |
| *** vsaienk0 has joined #openstack-infra | 17:11 | |
| haleyb | clarkb: ping, regarding zuul stats for neutron | 17:11 |
| clarkb | haleyb: hey | 17:11 |
| *** Apoorva_ has quit IRC | 17:12 | |
| clarkb | haleyb: the stats are at paths like stats.zuul.tenant.openstack.pipeline.check.project.git_openstack_org.openstack_neutron.master.job.legacy-neutron-dsvm-functional.FAILURE now | 17:12 |
| fungi | dhellmann: it'll be the current version of bash in ubuntu xenial *taking into account security updates), so 4.3-14ubuntu1.2, while the old signing01.ci node was running trusty and had bash 4.3-7ubuntu1.7 | 17:12 |
| smcginnis | fungi, dhellmann: Kind of looks like there is no newline between those separate deliverables/ | 17:12 |
| *** Apoorva has quit IRC | 17:12 | |
| clarkb | haleyb: basically they are scoped to tenants and the project is more uniquely identified and the jobs are paired to projects | 17:12 |
| *** Apoorva_ has joined #openstack-infra | 17:12 | |
| fungi | dhellmann: i wouldn't expect significant behavior changes in builtins between different package revisions of the same upstream bash release (4.3) | 17:12 |
| *** apetrich has quit IRC | 17:12 | |
| dhellmann | fungi : I have bash 4.3-14u1.2 locally so that's not it | 17:13 |
| *** apetrich has joined #openstack-infra | 17:13 | |
| dhellmann | smcginnis : the python script should be printing one per line. | 17:13 |
| dhellmann | it's so weird that this works on my local box, and worked fine upstream before. | 17:13 |
| haleyb | clarkb: so tenant and project are variables? | 17:13 |
| dhellmann | s/upstream/in ci/ | 17:13 |
| smcginnis | We have the /bin/bash in there, but this isn't another case that we need to explicitly tell ansible to use bash, is it? | 17:14 |
| clarkb | haleyb: tenant for us is always openstack but yes project can be neutron or neutron-lib etc | 17:14 |
| openstackgerrit | Samuel Matzek proposed openstack-infra/project-config master: Fix python-troveclient tempest tests https://review.openstack.org/514769 | 17:14 |
| dhellmann | smcginnis : interesting thought. this script is run directly from the task that we did that to yesterday. | 17:15 |
| *** slaweq has quit IRC | 17:15 | |
| dhellmann | or sunday, whenever it was | 17:15 |
| clarkb | haleyb: beacuse we no longer have unique job names, like gate-neutron-py27 instead it is openstack-tox-py27 used on all the projects | 17:15 |
| *** dhinesh has quit IRC | 17:15 | |
| haleyb | clarkb: is there a key somewhere? i had been looking at graphite.openstack.org and couldn't find anything | 17:16 |
| *** rtjure has quit IRC | 17:16 | |
| clarkb | haleyb: graphite.openstack.org is where I found that path above | 17:16 |
| jeblair | haleyb, clarkb: https://docs.openstack.org/infra/zuul/feature/zuulv3/admin/monitoring.html#stat-zuul.tenant.%3Ctenant%3E.pipeline.%3Cpipeline%20name%3E.project.%3Ccanonical_hostname%3E.%3Cproject%3E.%3Cbranch%3E.job.%3Cjobname%3E.%3Cresult%3E | 17:16 |
| haleyb | maybe i didn't look in the right place, and the search didn't find it | 17:16 |
| smcginnis | fungi: This is a re-run, right? http://logs.openstack.org/7d/7d3a7e81f9a4d4abb1bec7d5f405ab06ba397f08/release-post/tag-releases/a7a85ec/job-output.txt.gz#_2017-10-24_16_52_49_689018 | 17:17 |
| fungi | smcginnis: yes | 17:17 |
| *** sree has joined #openstack-infra | 17:18 | |
| haleyb | clarkb: aha, there is it! thanks. hopefully i can get our dashboard limping along again | 17:18 |
| jeblair | haleyb, clarkb: so i think that translates to something like: stats.counts.zuul.tenant.openstack.gate.project.neutron.git_openstack_org.neutron.master.job.openstack-tox-py35.SUCCESS | 17:18 |
| jeblair | nope i mesesd that up | 17:18 |
| haleyb | openstack_neutron | 17:19 |
| jeblair | haleyb, clarkb: so i think that translates to something like: stats.counts.zuul.tenant.openstack.pipeline.gate.project.neutron.git_openstack_org.neutron.master.job.openstack-tox-py35.SUCCESS | 17:19 |
| jeblair | yep | 17:19 |
| jeblair | haleyb, clarkb: so i think that translates to something like: stats.counts.zuul.tenant.openstack.pipeline.gate.project.neutron.git_openstack_org.openstack_neutron.master.job.openstack-tox-py35.SUCCESS | 17:19 |
| fungi | smcginnis: dhellmann: i'm reenqueuing it again and holding the node | 17:19 |
| dhellmann | fungi : ack. I'm looking at some other changes to that script | 17:19 |
| jeblair | haleyb, clarkb: stats.counts.zuul.tenant.openstack.pipeline.gate.project.git_openstack_org.openstack_neutron.master.job.openstack-tox-py35.SUCCESS | 17:20 |
| haleyb | jeblair: yes, i think that's it, and now that i found the folder in graphite i should be able to map all the names | 17:21 |
| *** vsaienk0 has quit IRC | 17:21 | |
| clarkb | haleyb: cool, let us know if you have any other questions | 17:21 |
| *** vsaienk0 has joined #openstack-infra | 17:22 | |
| *** sree has quit IRC | 17:22 | |
| *** rkukura has quit IRC | 17:23 | |
| *** rtjure has joined #openstack-infra | 17:23 | |
| clarkb | I'm going to pop out for a caffeine break then work on meeting prep | 17:24 |
| pabelanger | ack | 17:24 |
| thingee | clarkb: https://review.openstack.org/#/c/514726/2 plz | 17:26 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Use native v3 publishing for project-team-guide https://review.openstack.org/514053 | 17:26 |
| smcginnis | fungi: Looks like it should be ready for poking around. | 17:26 |
| fungi | smcginnis: yeah, i've already been logged in for several minutes | 17:26 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix branch ordering on dynamic reconfiguration https://review.openstack.org/514744 | 17:27 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: remove set -e from release scripts to let them accumulate errors https://review.openstack.org/514772 | 17:27 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: use a temporary file to hold the release change list https://review.openstack.org/514773 | 17:27 |
| dhellmann | fungi , smcginnis : I suspect the problem is the set -e change but I went ahead and made the file change, too ^^ | 17:27 |
| *** salv-orlando has joined #openstack-infra | 17:27 | |
| smcginnis | dhellmann: Temp file looks like a good idea. | 17:28 |
| *** rtjure has quit IRC | 17:28 | |
| thingee | clarkb: nevermind | 17:28 |
| fungi | dhellmann: awesome, that all looks like a good idea regardless | 17:29 |
| *** andreas_s has joined #openstack-infra | 17:29 | |
| *** ralonsoh has quit IRC | 17:32 | |
| openstackgerrit | Merged openstack-infra/project-config master: Use the python-docs playbook for contributor-guide publishing https://review.openstack.org/514726 | 17:32 |
| *** andreas_s has quit IRC | 17:33 | |
| *** gyee has joined #openstack-infra | 17:36 | |
| *** rtjure has joined #openstack-infra | 17:36 | |
| openstackgerrit | Merged openstack-infra/project-config master: remove set -e from release scripts to let them accumulate errors https://review.openstack.org/514772 | 17:38 |
| openstackgerrit | Merged openstack-infra/project-config master: use a temporary file to hold the release change list https://review.openstack.org/514773 | 17:38 |
| fungi | smcginnis: dhellmann: reenqueued now that those ^ merged | 17:40 |
| smcginnis | Watching | 17:40 |
| dhellmann | watching | 17:41 |
| *** rtjure has quit IRC | 17:41 | |
| * dhellmann offers the gerbils powering the jobs a carrot to run faster | 17:42 | |
| AJaeger | smatzek: you can add new jobs directly in repo and review with your team... If you need help with that, please ask - and read the zuul v3 docu at https://docs.openstack.org/infra/manual/zuulv3.html | 17:45 |
| *** jascott1 has quit IRC | 17:45 | |
| *** jascott1 has joined #openstack-infra | 17:46 | |
| *** rtjure has joined #openstack-infra | 17:47 | |
| dansmith | dhellmann: whoa, does that work? | 17:47 |
| openstackgerrit | Vladyslav Drok proposed openstack-infra/project-config master: Run pep8 and linters jobs instead of py27 https://review.openstack.org/514610 | 17:47 |
| dhellmann | dansmith : not so far :-( | 17:47 |
| dansmith | bummer | 17:47 |
| SamYaple | dhellmann: try a radioactive carrot. ive always had good success with those | 17:47 |
| AJaeger | vdrok: pep8 is the name for our linters job, no need to run both | 17:48 |
| smcginnis | fungi, dhellmann: Did it skip it completely this time? | 17:48 |
| vdrok | AJaeger: hrm, is it defined as kind of alias? I was requested to do this https://review.openstack.org/514626 | 17:48 |
| dhellmann | smcginnis : it only ran castellan again | 17:48 |
| vdrok | to separate doc8 and testr into different envs | 17:49 |
| *** jascott1 has quit IRC | 17:49 | |
| AJaeger | vdrok: 514626 does not run linters at all, you have no such environment and don't use it. | 17:49 |
| *** bh526r has quit IRC | 17:50 | |
| *** trown|lunch is now known as trown | 17:50 | |
| AJaeger | vdrok: the change is perfect, just commit message wrong. I'll comment on 514626 | 17:50 |
| smcginnis | I've noticed if you are pn a bad network, the stream sometime just skips over stuff. | 17:50 |
| dhellmann | smcginnis , fungi : so, I could rewrite release_from_yaml.sh in python | 17:50 |
| vdrok | AJaeger: ah, I did intend to add linters job as well into project config | 17:50 |
| vdrok | hit the git-review too early | 17:50 |
| AJaeger | vdrok: either pep8 or linters - please, not both. | 17:51 |
| dhellmann | smcginnis , fungi : or I could try generating a bash script instead of looping over the data | 17:51 |
| AJaeger | vdrok: that's just wasting resources | 17:51 |
| dhellmann | but otherwise I'm at a loss to see what's wrong | 17:51 |
| *** rtjure has quit IRC | 17:52 | |
| vdrok | AJaeger: okie, will comment as well, thank you :) | 17:52 |
| smcginnis | dhellmann: Let's see if fungi has any post mortem insights. | 17:53 |
| dhellmann | yeah | 17:53 |
| fungi | dhellmann: smcginnis: yeah, as best i can tell this is some strange behavior with looping read in bash... i'm going to try to see if i can reproduce manually and nail it down | 17:54 |
| *** jascott1 has joined #openstack-infra | 17:54 | |
| smcginnis | Rewrite in python doesn't sound too bad if we have to. | 17:54 |
| *** andreas_s has joined #openstack-infra | 17:56 | |
| *** nunchuck has joined #openstack-infra | 17:56 | |
| *** rtjure has joined #openstack-infra | 17:56 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove Oslo.log legacy job https://review.openstack.org/511385 | 17:56 |
| jeblair | fungi, dhellmann, smcginnis: can you point me at a build of the job you're looking at? | 17:58 |
| dhellmann | jeblair : http://logs.openstack.org/7d/7d3a7e81f9a4d4abb1bec7d5f405ab06ba397f08/release-post/tag-releases/e570884/job-output.txt.gz | 17:58 |
| fungi | jeblair: http://logs.openstack.org/7d/7d3a7e81f9a4d4abb1bec7d5f405ab06ba397f08/release-post/tag-releases/e570884/ | 17:58 |
| fungi | jeblair: attempts to use bash's built-in `read` command to iterate over lines of input seem to inexplicably stop after one iteration | 17:59 |
| *** dhinesh has joined #openstack-infra | 18:00 | |
| *** rtjure has quit IRC | 18:01 | |
| openstackgerrit | Vladyslav Drok proposed openstack-infra/project-config master: Run linters job instead of py27 https://review.openstack.org/514610 | 18:01 |
| *** tosky has quit IRC | 18:01 | |
| jeblair | fungi: that 'tee' suggests we should see something on stdout, right? | 18:01 |
| *** rbrndt has quit IRC | 18:02 | |
| *** baoli has quit IRC | 18:02 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Follow redirects when triggering readthedocs build https://review.openstack.org/514570 | 18:02 |
| fungi | jeblair: and we do | 18:02 |
| *** baoli has joined #openstack-infra | 18:03 | |
| jeblair | fungi: which line(s) is(are) output by the tee? | 18:03 |
| fungi | jeblair: oh! the second tee | 18:03 |
| jeblair | oh, there's a first :) | 18:03 |
| fungi | i don't know if that second (inner) tee is expected to show any deliverable updates on that iteration | 18:04 |
| jeblair | k, i will reorient myself :) | 18:04 |
| *** andreas_s has quit IRC | 18:04 | |
| *** iyamahat has quit IRC | 18:04 | |
| fungi | this is a reenqueued job, and it determines that the first line from the deliverable changes list doesn't need processing (because it got processed on a previous run) but doesn't proceed to loop over the next and subsequent lines | 18:05 |
| *** baoli has quit IRC | 18:05 | |
| fungi | basically the first time this ran it released castellan 0.14.1 and didn't loop over subsequent entries | 18:06 |
| *** rtjure has joined #openstack-infra | 18:06 | |
| fungi | when rerunning it decides (correctly) that castellan 0.14.1 is a no-op and skips it, but still doesn't continue to subsequent lines | 18:06 |
| *** gyee has quit IRC | 18:07 | |
| fungi | and dhellmann using a similar version of bash is unable to reproduce this behavior locally on his workstation | 18:07 |
| SamYaple | where is the code in question? | 18:07 |
| dhellmann | and this was working before all of these other changes we've made as part of the zuul upgrade | 18:07 |
| jeblair | SamYaple: project-config/jenkins/scripts/release-tools/release_from_yaml.sh | 18:07 |
| SamYaple | thanks | 18:08 |
| fungi | and the other scripts it calls are in that same directory | 18:08 |
| fungi | the playbook calling into it is http://git.openstack.org/cgit/openstack-infra/project-config/tree/playbooks/release/tag.yaml | 18:08 |
| *** ijw has quit IRC | 18:08 | |
| *** rkukura has joined #openstack-infra | 18:09 | |
| *** smatzek has quit IRC | 18:09 | |
| *** smatzek has joined #openstack-infra | 18:10 | |
| fungi | oh, also i was wrong, those two tees aren't in inner/outer loops, they're from the two scripts called independently by the playbook. the second one likely should be empty because it's for determining whether the release request included branch creation needs | 18:10 |
| fungi | but regardless, it's the first we're concerned about not iterating | 18:11 |
| *** rtjure has quit IRC | 18:11 | |
| *** iyamahat has joined #openstack-infra | 18:11 | |
| fungi | this was being done with a pipe into a while read loop previously and behaved the same as it's doing now with stdin redirection from a file to the while read loop | 18:11 |
| fungi | so pipe vs file redirection doesn't seem to have altered the behavior | 18:12 |
| *** caphrim007 has quit IRC | 18:13 | |
| *** dhajare has joined #openstack-infra | 18:13 | |
| *** jpena is now known as jpena|off | 18:13 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: WIP - Move openstack-infra/tripleo-ci jobs in-tree https://review.openstack.org/514778 | 18:13 |
| *** caphrim007 has joined #openstack-infra | 18:13 | |
| *** iyamahat has quit IRC | 18:14 | |
| *** iyamahat has joined #openstack-infra | 18:14 | |
| *** andreas_s has joined #openstack-infra | 18:14 | |
| *** amoralej is now known as amoralej|off | 18:14 | |
| *** smatzek has quit IRC | 18:14 | |
| *** caphrim007_ has joined #openstack-infra | 18:15 | |
| *** caphrim007_ has quit IRC | 18:15 | |
| *** caphrim007 has quit IRC | 18:15 | |
| fungi | basic attempts to reproduce on the node i held are working as expected instead | 18:15 |
| *** kjackal_ has quit IRC | 18:16 | |
| *** vsaienk0 has quit IRC | 18:16 | |
| fungi | but i'm just doing simple looping over a datafile, not rerunning the script | 18:16 |
| *** rtjure has joined #openstack-infra | 18:17 | |
| fungi | which... i _can_ attempt if desired | 18:17 |
| jeblair | fungi: it's idempotent, right? | 18:19 |
| jeblair | (i mean, as long as it continues to fail :) | 18:19 |
| fungi | in theory, yes | 18:19 |
| *** caphrim007 has joined #openstack-infra | 18:19 | |
| *** vsaienk0 has joined #openstack-infra | 18:20 | |
| dhellmann | fungi : the script should be idempotent, so assuming we have all the credentials in place it's safe to run it | 18:20 |
| jeblair | fungi: i'm a little worried that might work though. :) | 18:20 |
| fungi | ahh, right, we do remove the ssh and pgp credentials at the end of the job (just not the lp creds temporarily, though we should be safe to approve the revert on that now) | 18:21 |
| jeblair | maybe it would be worthwhile to try to run it via ansible from an executor; or maybe just try to reproduce something synthetic through ansible | 18:21 |
| *** rtjure has quit IRC | 18:21 | |
| fungi | so if i did try to rerun manually on the node, it would fail to tag and push stuff | 18:22 |
| jeblair | i'm going to work on the latter right now -- i'll propose a job that behaves like this and we'll see zuul run it. | 18:22 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: WIP - Move openstack-infra/tripleo-ci jobs in-tree https://review.openstack.org/514778 | 18:22 |
| jeblair | fungi: okay, maybe it's worth a try then | 18:22 |
| *** andreas_s has quit IRC | 18:23 | |
| *** Apoorva has joined #openstack-infra | 18:23 | |
| *** electrofelix has quit IRC | 18:23 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: rewrite release jobs in python https://review.openstack.org/514780 | 18:24 |
| dhellmann | fungi , jeblair , smcginnis : ^^ | 18:24 |
| jeblair | can't argue with that | 18:24 |
| *** salv-orlando has quit IRC | 18:25 | |
| dhellmann | not the prettiest python code, but ... | 18:25 |
| openstackgerrit | Sean McGinnis proposed openstack-infra/project-config master: Revert "Temporarily leave launchpad creds for release jobs" https://review.openstack.org/514781 | 18:25 |
| *** salv-orlando has joined #openstack-infra | 18:25 | |
| smcginnis | fungi: Clean up. ^ | 18:25 |
| fungi | smcginnis: oh, i approved 514438 | 18:26 |
| *** Apoorva_ has quit IRC | 18:26 | |
| fungi | thanks though! | 18:27 |
| smcginnis | fungi: Oh, was it already done? Sorry, I missed that. | 18:27 |
| fungi | i likely didn't say | 18:27 |
| smcginnis | dhellmann: I thought I saw you mention earlier that it still needed to be cleaned up. Guess I shouldn't have looked closer. :) | 18:28 |
| smcginnis | Oops, fungi ^ | 18:28 |
| smcginnis | dhellmann: Typo in that patch. | 18:28 |
| *** camunoz has joined #openstack-infra | 18:28 | |
| dhellmann | smcginnis : I meant if they *were* cleaned up the script wouldn't work | 18:28 |
| smcginnis | But I think it might be good to get that landed, then try re-enqueuing that again. | 18:28 |
| dhellmann | smcginnis : not a typo -- there's a .sh wrapper for the .py | 18:29 |
| *** baoli has joined #openstack-infra | 18:29 | |
| openstackgerrit | Merged openstack-infra/project-config master: Revert "Temporarily leave launchpad creds for release jobs" https://review.openstack.org/514438 | 18:29 |
| *** rtjure has joined #openstack-infra | 18:29 | |
| *** salv-orlando has quit IRC | 18:30 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: rewrite release jobs in python https://review.openstack.org/514780 | 18:31 |
| *** baoli has quit IRC | 18:33 | |
| *** baoli has joined #openstack-infra | 18:33 | |
| *** baoli has quit IRC | 18:33 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: avoid creating extra temporary directory in release job https://review.openstack.org/514787 | 18:34 |
| *** rtjure has quit IRC | 18:34 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: DNM: test bash script https://review.openstack.org/514788 | 18:35 |
| *** andreas_s has joined #openstack-infra | 18:37 | |
| *** tosky has joined #openstack-infra | 18:38 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: TLS everywhere: Only generate ssh key if it's not there https://review.openstack.org/514152 | 18:38 |
| *** dasanind has quit IRC | 18:38 | |
| fungi | smcginnis: dhellmann: jeblair: with the current state of release_from_yaml.sh rerunning from an interactive shell on the held node does seem to iterate over all the deliverables | 18:40 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: DNM: test bash script https://review.openstack.org/514788 | 18:40 |
| smcginnis | Hmm | 18:40 |
| jeblair | fungi: so this is starting to feel like a difference running under zuul or ansible? | 18:40 |
| fungi | it fails because it can't use the gerrit ssh api, but it continues on to subsequent entries | 18:40 |
| fungi | and yes, i have an inkling this is something to do with the caller | 18:41 |
| fungi | just not (yet) sure what | 18:41 |
| *** rtjure has joined #openstack-infra | 18:41 | |
| smcginnis | Any other env differences? | 18:41 |
| jeblair | fungi: then maybe my stub script test will show something. if not, we may want to try to run ansible like zuul | 18:41 |
| dhellmann | this script is 2 levels down from the caller, though, why would that be making a difference in how a builtin shell function works? | 18:41 |
| *** pvaneck has quit IRC | 18:41 | |
| fungi | i poured over the manpage subsection for bash's read builtin, and didn't get any good ideas as to why | 18:42 |
| *** priteau has quit IRC | 18:42 | |
| *** salv-orlando has joined #openstack-infra | 18:42 | |
| fungi | dhellmann: the outer loop which isn't continuing is being invoked from a shell task in a playbook, so only one level down from the caller depending on how you look at it | 18:42 |
| fungi | rather the script containing that while loop is called from a shell task in the playbook | 18:43 |
| dhellmann | fungi : yeah, I was thinking caller process -> shell process -> 2nd shell process | 18:43 |
| clarkb | fungi: is it running in sh or similar again? | 18:43 |
| fungi | clarkb: nope, same one we already switched to be bash | 18:43 |
| *** baoli has joined #openstack-infra | 18:43 | |
| dhellmann | clarkb : no, it's explicitly /bin/bash | 18:43 |
| dhellmann | although is /bin/bash right? maybe should be some other path? | 18:44 |
| clarkb | dhellmann: should be /bin/bash on ubuntu | 18:44 |
| dhellmann | ok | 18:44 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: DNM: test bash script https://review.openstack.org/514788 | 18:45 |
| fungi | not only is the playbook containing the shell task set explicitly to use /bin/bash as its executable, but the while loop in question is in a script with a #!/bin/bash as well and is being invoked by name, not as an argument to a shell | 18:45 |
| *** andreas_s has quit IRC | 18:45 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/openstack-zuul-jobs master: WIP - Move TripleO jobs/templates to tripleo-ci https://review.openstack.org/514792 | 18:46 |
| smcginnis | Something failing within the loop that's causing the loop to exit but not showing up as a total failure? | 18:46 |
| *** rtjure has quit IRC | 18:46 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 18:47 |
| dhellmann | smcginnis : maybe that math to accumulate errors? | 18:47 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: DNM: test bash script https://review.openstack.org/514788 | 18:48 |
| jeblair | here's something that could be relevant -- i think the entirety of the "shell" command in the playbook is placed on the bash command line | 18:49 |
| jeblair | i don't know *how* that could affect things | 18:50 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/openstack-zuul-jobs master: WIP - Move TripleO jobs/templates to tripleo-ci https://review.openstack.org/514792 | 18:51 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 18:52 |
| *** rtjure has joined #openstack-infra | 18:52 | |
| clarkb | jeblair: any chance this could also be related to cmd: vs no cmd: ? | 18:53 |
| jeblair | clarkb: i can't think of how... but it's so strange anything is possible | 18:53 |
| jeblair | my simple test case failed to reproduce the problem | 18:54 |
| jeblair | https://review.openstack.org/514788 and http://logs.openstack.org/88/514788/4/check/zuul-migrate/66533b7/ | 18:54 |
| jeblair | dhellmann, fungi: have any other suspect statements i didn't include in 514788 that you think i should add in? | 18:55 |
| *** vsaienk0 has quit IRC | 18:55 | |
| *** iyamahat has quit IRC | 18:56 | |
| *** rtjure has quit IRC | 18:56 | |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Migrate legacy jobs for feature/zuulv3 branch https://review.openstack.org/513766 | 18:56 |
| dhellmann | jeblair : looking | 18:56 |
| fungi | the only significant difference i can see is it's not reading multiple fields | 18:57 |
| dhellmann | jeblair : try reading more than one variable | 18:57 |
| dhellmann | yep | 18:57 |
| jeblair | ack | 18:57 |
| fungi | not that i have any idea why that would alter behavior | 18:57 |
| dhellmann | maybe read is actually failing because of the data? | 18:58 |
| fungi | i'm going to try reenqueuing that releases commit into release-post one more time too, just to make sure we're running with the same versions of scripts i thought we were | 18:58 |
| jeblair | i'll copy the data too | 18:58 |
| smcginnis | The earlier printed data all looked OK, but who knows. | 18:58 |
| clarkb | meeting time in #openstack-infra | 18:59 |
| clarkb | (not to interrupt debugging) | 19:00 |
| fungi | indeed it is! | 19:00 |
| clarkb | er | 19:00 |
| dhellmann | jeblair : also try passing the values you read in the loop into the 2nd script and print them there | 19:00 |
| clarkb | #openstack-meeting | 19:00 |
| *** rtjure has joined #openstack-infra | 19:01 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: DNM: test bash script https://review.openstack.org/514788 | 19:02 |
| *** e0ne has joined #openstack-infra | 19:03 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/project-config master: Remove nodepool legacy jobs from project https://review.openstack.org/514799 | 19:05 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/openstack-zuul-jobs master: Remove legacy nodepool job defs and playbooks https://review.openstack.org/514321 | 19:05 |
| *** rtjure has quit IRC | 19:06 | |
| *** vsaienk0 has joined #openstack-infra | 19:06 | |
| *** gyee has joined #openstack-infra | 19:07 | |
| *** ijw_ has joined #openstack-infra | 19:09 | |
| *** ijw_ has joined #openstack-infra | 19:09 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/project-config master: Remove nodepool legacy jobs from project https://review.openstack.org/514799 | 19:09 |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/project-config master: Fix Grafana periodic counters for neutron-lib dashboard https://review.openstack.org/514801 | 19:10 |
| *** gyee has quit IRC | 19:12 | |
| *** rbrndt has joined #openstack-infra | 19:12 | |
| *** rtjure has joined #openstack-infra | 19:13 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: WIP - Move openstack-infra/tripleo-ci jobs in-tree https://review.openstack.org/514778 | 19:14 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 19:14 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/openstack-zuul-jobs master: WIP - Move TripleO jobs/templates to tripleo-ci https://review.openstack.org/514792 | 19:14 |
| fungi | smcginnis: dhellmann: jeblair: reenqueue of that last releases repo commit still produced the proken behavior. only one pass through the while loop, and then it seems to read no data on the second iteration: http://logs.openstack.org/7d/7d3a7e81f9a4d4abb1bec7d5f405ab06ba397f08/release-post/tag-releases/2f28115/job-output.txt.gz#_2017-10-24_19_04_48_751171 | 19:15 |
| fungi | s/proken/broken/ | 19:15 |
| *** Apoorva has quit IRC | 19:15 | |
| dhellmann | fungi : are things in a state where you could make a local modification and try again? or do we need to push a patch through for that to work? | 19:16 |
| *** Apoorva has joined #openstack-infra | 19:16 | |
| *** rbrndt has quit IRC | 19:16 | |
| *** vsaienk0 has quit IRC | 19:16 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: try looping and just printing info to see if that reads everything https://review.openstack.org/514803 | 19:17 |
| dhellmann | fungi : if you can do ^^ locally that's good, otherwise we can merge it | 19:17 |
| fungi | dhellmann: we delete all the credentials at the end of the job, so no i can't easily rerun it on the held node without tweaking the job to drop the removal tasks again | 19:17 |
| dhellmann | ok | 19:17 |
| *** rtjure has quit IRC | 19:18 | |
| *** pvaneck has joined #openstack-infra | 19:18 | |
| fungi | oh, though i did exactly what you have in 514803 earlier and it looped through them all | 19:18 |
| fungi | (prefixing the release.sh call with echo) | 19:18 |
| fungi | though i did not add the rc math bit | 19:19 |
| dhellmann | weird | 19:19 |
| *** vsaienk0 has joined #openstack-infra | 19:20 | |
| *** jascott1 has quit IRC | 19:20 | |
| *** jascott1 has joined #openstack-infra | 19:21 | |
| *** slaweq has joined #openstack-infra | 19:22 | |
| *** hemna_ has quit IRC | 19:22 | |
| *** rtjure has joined #openstack-infra | 19:24 | |
| *** jascott1 has quit IRC | 19:26 | |
| *** ijw_ has quit IRC | 19:29 | |
| *** ijw has joined #openstack-infra | 19:29 | |
| *** rtjure has quit IRC | 19:29 | |
| *** vsaienk0 has quit IRC | 19:30 | |
| *** jascott1 has joined #openstack-infra | 19:33 | |
| *** rtjure has joined #openstack-infra | 19:34 | |
| *** jascott1 has quit IRC | 19:35 | |
| *** boden has joined #openstack-infra | 19:36 | |
| *** vsaienk0 has joined #openstack-infra | 19:36 | |
| *** rtjure has quit IRC | 19:39 | |
| boden | FYI.. while https://review.openstack.org/#/c/513199/ did change our failures, it doesn’t appear to address all our issues for stable branch jobs.. I added a new item to https://etherpad.openstack.org/p/zuulv3-issues | 19:40 |
| *** dhinesh has quit IRC | 19:41 | |
| tosky | what is the easier way to reference the src directory of a certain repository in a role defined in that repository? | 19:42 |
| *** rtjure has joined #openstack-infra | 19:43 | |
| *** jascott1 has joined #openstack-infra | 19:45 | |
| jeblair | tosky: probably zuul._projects[project_name].src_dir | 19:46 |
| jeblair | tosky: https://docs.openstack.org/infra/zuul/feature/zuulv3/user/jobs.html#var-zuul.projects | 19:46 |
| jeblair | tosky: note we're in the middle of a transition there, and _projects will become projects very soon | 19:46 |
| *** ijw has quit IRC | 19:46 | |
| Shrews | jeblair: is zuul.project.src_dir still valid (mentioned in those docs)? | 19:47 |
| jeblair | tosky: but also, you may want to consider passing the src location into the role, and using that variable to supply the value (so that the role is more generic) | 19:47 |
| *** felipemonteiro__ has quit IRC | 19:47 | |
| jeblair | Shrews: that's the project under test, whereas i assumed tosky meant a specific named project in a cross-repo test. but i could be wrong. | 19:47 |
| Shrews | jeblair: ah, k. i interpreted the other way. | 19:48 |
| *** jascott1 has quit IRC | 19:48 | |
| fungi | dhellmann: testing 514803 verbatim on the node i held (under an interactive bash shell as the zuul user with a copy of the tag.yaml shell task invoked explicitly under bash) does what you would expect with the new loop, but also the old loop works under my attempts anyway with or without that | 19:48 |
| jeblair | Shrews, tosky: yeah, if that's what was meant, that's totally the right way to do it. :) | 19:48 |
| *** rtjure has quit IRC | 19:48 | |
| *** caphrim007 has quit IRC | 19:48 | |
| fungi | dhellmann: so since i'm unable to manually reproduce the loop behavior, that doesn't really help explain anything | 19:49 |
| jeblair | fungi: want me to repeat that test using ansible from my workstation? | 19:49 |
| fungi | jeblair: please feel free | 19:50 |
| jeblair | fungi: ip address ... and do i need any other info? | 19:50 |
| fungi | jeblair: 149.202.168.223 is the node i held, if you want it | 19:50 |
| fungi | {{ zuul.project.canonical_name }} is git.openstack.org/openstack/releases for http://git.openstack.org/cgit/openstack-infra/project-config/plain/playbooks/release/tag.yaml | 19:51 |
| *** rockyg has joined #openstack-infra | 19:51 | |
| jeblair | cool. i'll work up an inventory with that variable in it too. | 19:51 |
| tosky | jeblair: exactly; I have a role A1 in a repository A and I want to use it inside the repository B, but I'd like to have a sane default and not have to specify the location of A | 19:51 |
| jeblair | tosky: is this specifically because you want to use a role from repo A in a job defined in repo B? | 19:52 |
| tosky | jeblair: yes; I normally use that role in A, but I have (at least one) use case in B | 19:52 |
| tosky | I can of course add a parameter to A1 that points to the sources of A, but... sane defaults | 19:52 |
| jeblair | tosky: ah, that's different and so much nicer! | 19:52 |
| jeblair | tosky: https://docs.openstack.org/infra/zuul/feature/zuulv3/user/config.html#attr-job.roles | 19:52 |
| ianw | tonyb: ping me when around, we can look at branch closures whenever you want | 19:52 |
| jeblair | tosky: add "A" as a "role" to the job in B | 19:53 |
| tosky | jeblair: that's the easy part | 19:53 |
| tosky | let me show the code | 19:53 |
| jeblair | tosky: (assuming you're talking about the ansible that zuul runs, not your own ansible invocation) | 19:53 |
| tosky | I want to use http://git.openstack.org/cgit/openstack/sahara-image-elements/tree/roles/build-sahara-images-dib/tasks/main.yaml | 19:54 |
| tosky | inside https://review.openstack.org/#/c/512058/27/playbooks/sahara-tests-scenario.yaml | 19:54 |
| tosky | but of course zuul.project.src_dir, as Shrews pointed out, is the current project (so sahara-tests, when I use the build-sahara-images-dib role inside sahara-tests) | 19:55 |
| *** rtjure has joined #openstack-infra | 19:55 | |
| tosky | back to the question: what is the cleanest way to automatically point to the location of the source of sahara-image-elements? | 19:56 |
| tosky | or do I really need to pass them as parameter to the role? | 19:56 |
| *** smatzek has joined #openstack-infra | 19:56 | |
| AJaeger | tosky: check how I've done it for openstack-manuals, let me give you some links in a minute | 19:56 |
| * tosky git pull on his checkout of openstack-manuals | 19:57 | |
| AJaeger | tosky: http://git.openstack.org/cgit/openstack/openstack-manuals/tree/.zuul.yaml#n18 | 19:57 |
| AJaeger | that defines the job | 19:58 |
| jeblair | tosky: okay, so you're all set using the role, it's just that role does something to a git checkout of a project. | 19:58 |
| AJaeger | tosky: and a template - and here I'm using it http://git.openstack.org/cgit/openstack/security-doc/tree/.zuul.yaml | 19:58 |
| *** jascott1 has joined #openstack-infra | 19:58 | |
| AJaeger | tosky: wait - you only want the role? Not the complete job? | 19:58 |
| tosky | AJaeger: what you do is fine in that context: you pass "{{ zuul.project.src_dir }}/publish-docs", and zuul.project.src_dir is the directory of the project you are working on | 19:59 |
| tosky | I don't want that | 19:59 |
| *** jascott1 has quit IRC | 19:59 | |
| *** jascott1 has joined #openstack-infra | 20:00 | |
| *** smatzek has quit IRC | 20:00 | |
| AJaeger | tosky: getting it now - sorry to mislead | 20:00 |
| tosky | np :) | 20:00 |
| jeblair | tosky: and you want the role to always build sahara-image-elements, even if this job is running on a change to a different project? | 20:00 |
| *** rtjure has quit IRC | 20:00 | |
| tosky | jeblair: exactly, I want to always run that script against sahara-image-elements, at least by default (with optional variable etc etc) | 20:00 |
| jeblair | tosky: then yeah, i think the _projects thing is the way to go | 20:01 |
| tosky | maybe it's just not possible and I need to pass the path to the sources, but just to be sure :) | 20:01 |
| jeblair | (note, will be renamed projects) | 20:01 |
| clarkb | config-core I've just approved https://review.openstack.org/#/c/513910/2 I propose we tag that change "jenkins-eol" or similar and call it a day :) | 20:01 |
| AJaeger | call it a day sounds like a great proposal here in Europe ;) | 20:02 |
| clarkb | that gives me time to eat lunch while it gates then tag it afterwards :) | 20:02 |
| tosky | jeblair: with the same index as now (full namespaced name of the repository, so git.openstack.org/openstack/sahara-image-elements in my case) ? | 20:02 |
| AJaeger | clarkb: enjoy! | 20:02 |
| jeblair | tosky: yep. | 20:03 |
| tosky | jeblair: and more important, when will _projects be renamed project? | 20:03 |
| jeblair | tosky: very soon | 20:03 |
| tosky | I guess soonish before breaking everything | 20:03 |
| jeblair | yeah, we're trying to sneak this in before we get lots of references everywhere | 20:03 |
| AJaeger | tosky: https://review.openstack.org/514117 | 20:03 |
| tosky | before people try to do sneaky things like me :) | 20:04 |
| AJaeger | tosky: and friends, check the topic:projects-dict | 20:04 |
| jeblair | tosky: well, it's more that as soon as people started using it, we realized the list form was wrong :) | 20:04 |
| jeblair | iterating over a list of projects is a minor use case. getting info on a specific project like you want is the main use case. | 20:04 |
| *** rtjure has joined #openstack-infra | 20:05 | |
| jeblair | (and the dict form makes both suitably easy) | 20:05 |
| jeblair | (the list form makes the second hard) | 20:05 |
| tosky | yep | 20:05 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove jenkins Frozen tests https://review.openstack.org/513910 | 20:05 |
| *** e0ne has quit IRC | 20:06 | |
| tosky | jeblair: another question related to the definitions for the stable branches; the guide suggest to move only part of the definitions (I don't remember now if the jobs or the templates) | 20:07 |
| AJaeger | tosky: templates | 20:07 |
| tosky | but wouldn't it make sense to just copy the entire file? | 20:07 |
| AJaeger | jobs only if you want to change them | 20:07 |
| tosky | when we will branch queens, we will copy the entire file anyway | 20:07 |
| tosky | so we could start using that pattern even now for the older branches | 20:08 |
| jeblair | tosky: yeah, the dcos may need to be updated | 20:08 |
| jeblair | tosky: what you describe is the pattern zuul is designed for | 20:08 |
| *** rcarrillocruz has quit IRC | 20:08 | |
| *** gyee has joined #openstack-infra | 20:08 | |
| *** rcarrill1 has joined #openstack-infra | 20:08 | |
| jeblair | yep. i will update those docs | 20:09 |
| tosky | ok, so I will copy the entire file(s), thanks :) | 20:09 |
| tosky | uhm, same for playbooks and roles? | 20:09 |
| jeblair | tosky: yep | 20:09 |
| tosky | ack | 20:10 |
| *** rtjure has quit IRC | 20:10 | |
| jeblair | tosky: basically, your thought around "well, that's going to happen when we branch" is the main design constraint :) | 20:10 |
| tosky | I guess it's not going to be a problem, when the in-tree jobs for a certain repository are stable, to squash the various commits into a single commit for each stable branch | 20:11 |
| tosky | instead of cherry-picking all the changes | 20:11 |
| jeblair | yeah, this is a one-time issue | 20:12 |
| *** gyee has quit IRC | 20:13 | |
| AJaeger | tosky: that's a policy question - I agree and would do a single commit as backport | 20:13 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Check that zuul.d/projects.yaml is sorted https://review.openstack.org/514061 | 20:13 |
| tosky | AJaeger, jeblair : I thought so; thanks for confirming | 20:14 |
| tosky | better be sure; probably others are doing the same but I feel like being on the edge of the change, so better ask | 20:14 |
| *** rtjure has joined #openstack-infra | 20:14 | |
| * AJaeger follows now clarkb's suggestion to call it a day and waves good night | 20:15 | |
| dhellmann | fungi , jeblair : I don't mind continuing to dig into what's going on, but I'm also inclined to just take that python rewrite and move on so we can finish the queens milestone. I have a high level of certainty that we'll encounter other bugs this week as we process releases. | 20:15 |
| tosky | bye AJaeger ! | 20:15 |
| *** eharney has quit IRC | 20:16 | |
| *** dhinesh has joined #openstack-infra | 20:16 | |
| jeblair | dhellmann, fungi: ack. can you prepare for that, but give me a few more minutes before you do so -- i'm running test invocations now on the held node and it'd be useful to have a tiny bit more debug time before things change and we can't reproduce anymore. | 20:17 |
| dhellmann | jeblair : https://review.openstack.org/514780 is ready when you are | 20:18 |
| *** dhinesh_ has joined #openstack-infra | 20:19 | |
| *** rtjure has quit IRC | 20:19 | |
| jeblair | fungi: i think it's stuck on sshing to gerrit because they key isn't there.. what did you do at this point in the script? | 20:20 |
| fungi | jeblair: i simply hit enter | 20:20 |
| *** ldnunes has quit IRC | 20:20 | |
| jeblair | ah that is harder for me :) | 20:20 |
| fungi | and allowed it to fail that step | 20:20 |
| fungi | yeah, might need some surgery on the script there on disk | 20:21 |
| jeblair | so when i kill ssh it seems to proceed. | 20:21 |
| jeblair | and continues the loop. | 20:21 |
| fungi | huh | 20:21 |
| jeblair | so it's not just "under ansible". it must be "under zuul". | 20:21 |
| jeblair | i'll try to do it again with local zuul modules | 20:22 |
| fungi | well, or something about failing the ssh client calls is causing it to be able to continue iterating | 20:22 |
| jeblair | er plugins | 20:22 |
| jeblair | fungi: good point | 20:22 |
| *** armax has quit IRC | 20:22 | |
| *** dhinesh has quit IRC | 20:22 | |
| jeblair | fungi: perhaps i should give up and lets just move with the python rewrite | 20:22 |
| jeblair | so many variables. | 20:23 |
| fungi | yeah, i've about finished reviewing the python rewrite there | 20:23 |
| jeblair | only other thing it might be worth trying is to use the script ansible module instead of shell | 20:25 |
| jeblair | but, we have a python rewrite in hand | 20:25 |
| *** rtjure has joined #openstack-infra | 20:25 | |
| dhellmann | how is that module different? | 20:25 |
| *** priteau has joined #openstack-infra | 20:26 | |
| jeblair | dhellmann: it copies the script to the remote host and runs it. so the script content is not a blob of shell cmdline args. but also, the script content doesn't get templated, so we'd need to give the src_dir as an argument (should be easy) | 20:26 |
| dhellmann | ok | 20:27 |
| jeblair | http://docs.ansible.com/ansible/latest/script_module.html | 20:27 |
| dhellmann | it seems odd that your test case didn't reproduce the behavior | 20:27 |
| jeblair | (also, the script itself is not contained in the yaml, it's in a separate file, so it's much easier to edit) | 20:27 |
| jeblair | dhellmann: i agree | 20:27 |
| *** priteau has quit IRC | 20:27 | |
| jeblair | dhellmann: there's a lot missing from that test case, but the fact that it failed suggests that it's either not related to the way ansible is running the script, or the connection is labyrinthine. | 20:28 |
| *** jcoufal has quit IRC | 20:29 | |
| jeblair | dhellmann, fungi: approved python rewrite change. though i will readily admit my +2 is basically a moral-support +2. | 20:29 |
| *** rockyg has quit IRC | 20:30 | |
| *** rtjure has quit IRC | 20:30 | |
| jeblair | fungi: i'm done on that held node | 20:30 |
| dhellmann | yeah, I expect we'll have to test it live to see if it works | 20:30 |
| fungi | thanks jeblair, will delete | 20:30 |
| dhellmann | I did run it locally with the git push disabled | 20:30 |
| fungi | i assumed you had done something along those lines | 20:30 |
| fungi | it looked a bit too polished ;) | 20:31 |
| fungi | to have been entirely untested i mean | 20:31 |
| *** jascott1 has quit IRC | 20:31 | |
| dhellmann | oh, well *most* of the script is the old list_deliverable_changes.py content that we've been using for ages | 20:31 |
| *** jascott1 has joined #openstack-infra | 20:31 | |
| dhellmann | the fact that the file was renamed and changed got a bit lost in the review | 20:31 |
| fungi | yeah, i picked up on that | 20:32 |
| dhellmann | oh, maybe because I didn't actually delete the old file | 20:32 |
| fungi | but i was wondering why... yep | 20:32 |
| *** baoli has quit IRC | 20:32 | |
| openstackgerrit | Merged openstack-infra/project-config master: rewrite release jobs in python https://review.openstack.org/514780 | 20:32 |
| *** jascott1 has quit IRC | 20:32 | |
| fungi | figured you were planning another cleanup change to delete it or something | 20:33 |
| *** jascott1 has joined #openstack-infra | 20:33 | |
| fungi | okay, that's merged, i'll do another reenqueue and then need to go check the oil in the car real quick while it runs | 20:33 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: remove unused release-tools scripts https://review.openstack.org/514816 | 20:34 |
| dhellmann | fungi : thanks, and here's the other cleanup patch ^^ | 20:34 |
| *** apetrich has quit IRC | 20:34 | |
| *** felipemonteiro_ has joined #openstack-infra | 20:34 | |
| *** rtjure has joined #openstack-infra | 20:35 | |
| *** jascott1 has quit IRC | 20:35 | |
| *** felipemonteiro__ has joined #openstack-infra | 20:35 | |
| *** kjackal_ has joined #openstack-infra | 20:36 | |
| *** apetrich has joined #openstack-infra | 20:37 | |
| *** dhajare has quit IRC | 20:37 | |
| *** bobh has quit IRC | 20:38 | |
| *** felipemonteiro_ has quit IRC | 20:39 | |
| *** rtjure has quit IRC | 20:39 | |
| tonyb | ianw: Hey timezone buddy 'sup? | 20:40 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: add more trace info to process_release_requests.py https://review.openstack.org/514818 | 20:42 |
| *** felipemonteiro__ has quit IRC | 20:42 | |
| dhellmann | fungi : it appears to have worked! | 20:42 |
| *** felipemonteiro__ has joined #openstack-infra | 20:42 | |
| fungi | huzzah!!! | 20:42 |
| fungi | maybe i should take that as a clue to have dinner before something else grabs me | 20:42 |
| dhellmann | fungi : yes, let's quit while we're ahead :-) | 20:43 |
| fungi | tonyb: any corrections/additions from the eol topic in the infra meeting? http://eavesdrop.openstack.org/meetings/infra/2017/infra.2017-10-24-19.00.log.html#l-205 | 20:44 |
| *** rtjure has joined #openstack-infra | 20:44 | |
| tonyb | fungi: nope, I re-did the lists yesterday to account for the small amounts of feedback I got (for newton EOL) | 20:44 |
| tonyb | fungi: as noted I got zip for the old numeric branches even though where possible I CCd the team in question | 20:45 |
| *** dprince has quit IRC | 20:45 | |
| tonyb | fungi, AJaeger: sorry I missed the meeting, sick kids today :( | 20:45 |
| openstackgerrit | Brian Haley proposed openstack-infra/project-config master: Fix neutron-failure-rate grafana dashboard https://review.openstack.org/509291 | 20:46 |
| fungi | tonyb: no apologies required, just wanted to make sure we had it covered adequately | 20:46 |
| tonyb | fungi: thanks. I'll wait for ianw to arriave and we can work through the lists | 20:47 |
| tonyb | fungi, ianw, AJaeger: there will be at least one more round but obviously it'll be much smaller | 20:47 |
| *** baoli has joined #openstack-infra | 20:47 | |
| *** kjackal_ has quit IRC | 20:47 | |
| *** rtjure has quit IRC | 20:49 | |
| *** slaweq__ has joined #openstack-infra | 20:50 | |
| ianw | tonyb: hey | 20:50 |
| tonyb | ianw: what do you need from me for the EOL stuff? | 20:51 |
| *** slaweq__ has quit IRC | 20:52 | |
| openstackgerrit | Brian Haley proposed openstack-infra/project-config master: Fix neutron-failure-rate grafana dashboard https://review.openstack.org/509291 | 20:52 |
| ianw | a double check that i don't end up closing every branch in existence accidentally :) | 20:52 |
| *** slaweq_ has joined #openstack-infra | 20:52 | |
| ianw | let me get some terms ready here ... | 20:53 |
| *** smatzek has joined #openstack-infra | 20:53 | |
| *** hasharAway has quit IRC | 20:53 | |
| tonyb | ianw: okay :) | 20:53 |
| tonyb | ianw: I've never actually run the script (for the obvious reasons) but jhesketh_ seems to have improved it over the last few cycles | 20:54 |
| clarkb | config-core ok 8643da6c1c246249d694fb7f7c24d51a7449df94 is in project-config now. Thinking of calling the tag jenkins-eol, anyone have a better name? | 20:54 |
| tonyb | ianw: The last time he ran it it was with the current email format which he said was much better | 20:55 |
| tonyb | ianw: Alos the way it's borken up by team makes it easy to run a few verify ; repeat | 20:55 |
| fungi | clarkb: i suggested "farewell-jenkins" since jenkins itself is not eol upstream, we're just no longer carrying configuration derived from our former use of it | 20:55 |
| tonyb | ianw: also you cna batch your time on it | 20:55 |
| * tonyb has no idea how long a run actually takes | 20:55 | |
| clarkb | fungi: that works | 20:55 |
| clarkb | config-core any opposition to farewell-jenkins? | 20:56 |
| dmsimard | oops, I need to setup config-core highlight | 20:56 |
| fungi | clarkb: plus, it's marginally less negative/offensive to the jenkins community that way... no ill will (hopefully!) | 20:56 |
| *** rtjure has joined #openstack-infra | 20:57 | |
| *** andreas_s has joined #openstack-infra | 20:57 | |
| dmsimard | I don't have a strong opinion | 20:57 |
| dmsimard | farewell-jenkins can be seen as sarcastic if you try | 20:57 |
| dmsimard | ¯\_(ツ)_/¯ | 20:57 |
| jeblair | we could not tag | 20:58 |
| *** slaweq_ has quit IRC | 20:58 | |
| *** iyamahat has joined #openstack-infra | 20:59 | |
| jeblair | or attempt to be exceedingly factual with 'before-jenkins-config-removed'. | 20:59 |
| fungi | easy enough to just say "checkout 8643da6 if you want the last known state of our jenkins config" | 20:59 |
| dmsimard | fungi: tags are readily available in cgit, github or git CLI though | 21:00 |
| ianw | tonyb: so agree http://lists.openstack.org/pipermail/openstack-dev/2017-October/123910.html is the latest list? | 21:00 |
| dmsimard | oh, here, would "zuulv2-eol" work ? | 21:00 |
| pabelanger | okay, config-core added to highlights | 21:00 |
| *** rosmaita has quit IRC | 21:00 | |
| fungi | dmsimard: so are commit ids ;) just not as discoverable | 21:01 |
| clarkb | jeblair: exceedingly factual may be a good way to go. Since it would be hard to misunderstand that | 21:01 |
| tonyb | ianw: Yup that's the most up to date information I have | 21:01 |
| tonyb | ianw: I dont' see any replies coming in from overnight so LGTM | 21:01 |
| fungi | clarkb: jeblair: "before-jenkins-config-removed" wfm | 21:02 |
| *** rtjure has quit IRC | 21:02 | |
| dmsimard | sure, +1 | 21:02 |
| dmsimard | let's not lose too much time over this :) | 21:02 |
| dmsimard | we're tagging the commit *before* the purge, right ? | 21:03 |
| *** tbarron is now known as tbarron|bbl | 21:03 | |
| clarkb | dmsimard: I am tagging 8643da6c1c246249d694fb7f7c24d51a7449df94 | 21:03 |
| clarkb | which is before the pruge | 21:03 |
| dmsimard | perfect | 21:03 |
| ianw | ok, going to add myself to bootstrappers to try out some EOL | 21:04 |
| *** iyamahat has quit IRC | 21:04 | |
| tonyb | ianw: Thanks. | 21:04 |
| dmsimard | newton EOL is happening now ? | 21:04 |
| tonyb | dmsimard: Yup | 21:05 |
| *** thorst has quit IRC | 21:06 | |
| pabelanger | we should find a way to automate purging git cache on nodepool-builder servers | 21:06 |
| pabelanger | aside from just remembering to log in an delete the directory | 21:06 |
| pabelanger | or, maybe we just purge it once a week / month | 21:06 |
| pabelanger | via crontab | 21:07 |
| clarkb | could also probably get dib itself to police that better | 21:07 |
| clarkb | at least for removing old branches | 21:07 |
| clarkb | tag pushed | 21:07 |
| *** thorst has joined #openstack-infra | 21:07 | |
| jeblair | i bet we could do something with our giant event-driven arbitrary code execution engine. | 21:07 |
| pabelanger | indeed! | 21:08 |
| *** rtjure has joined #openstack-infra | 21:08 | |
| pabelanger | looking forward to it | 21:08 |
| jeblair | we should be able to run jobs that ssh into builders on branch deletion | 21:08 |
| *** vsaienk0 has quit IRC | 21:09 | |
| pabelanger | speaking of deleting things, https://review.openstack.org/514483/ removes zuul-env from images, are we ready to land that? | 21:09 |
| ianw | tonyb: ok, one thing is it's "eol_branch.sh" (rather than "eol-branch.sh") ... want to double check barbican & python-barbican client? | 21:09 |
| tonyb | ianw: *sigh* so sorry. I'll fix that for next time. | 21:10 |
| ianw | i need to figure out how to make git not prompt for my gpg key | 21:10 |
| ianw | run under ssh-agent i guess | 21:10 |
| clarkb | pabelanger: will that break any jobs that use the full path to the z-c env in them? | 21:10 |
| clarkb | pabelanger: the z-c shim goes to the symlink location right? | 21:10 |
| ianw | i mean gpg-agent | 21:10 |
| tonyb | ianw: sorry no idea other than gpg-agent or similar | 21:11 |
| *** vsaienk0 has joined #openstack-infra | 21:11 | |
| *** thorst has quit IRC | 21:12 | |
| tonyb | ianw: barbican and python-barbicanclient LGTM, the branches are gone and the tag is there | 21:13 |
| pabelanger | clarkb: we should be safe: http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/playbooks/legacy/pre.yaml#n3 | 21:13 |
| *** rtjure has quit IRC | 21:13 | |
| tonyb | ianw: and it's all visible on git.o.o | 21:13 |
| clarkb | pabelanger: you may need to mkdir -p /usr/zuul-env/bin/ in that case | 21:15 |
| * clarkb reads the fetch role | 21:15 | |
| clarkb | oh perfect doe that already | 21:15 |
| *** andreas_s has quit IRC | 21:15 | |
| pabelanger | yah | 21:15 |
| pabelanger | so. really should be a noop when we remove it | 21:16 |
| clarkb | I've +2'd the dib element change to remove it but not approved it. But I agree should be a noop due to the z-c shim | 21:16 |
| pabelanger | jeblair: did you want to review and approve^ 514483 | 21:17 |
| *** jascott1 has joined #openstack-infra | 21:17 | |
| *** rtjure has joined #openstack-infra | 21:17 | |
| *** armax has joined #openstack-infra | 21:18 | |
| *** esberglu has quit IRC | 21:18 | |
| *** esberglu has joined #openstack-infra | 21:19 | |
| jeblair | pabelanger: i would be inclined to do that after the release is complete, just in case something unanticipated happened and we lost time due to rolling back images | 21:19 |
| *** sree has joined #openstack-infra | 21:19 | |
| jeblair | but good news, apparently we're able to release things now :) | 21:20 |
| *** ijw has joined #openstack-infra | 21:20 | |
| EmilienM | I've pushed https://review.openstack.org/#/q/topic:tripleo/migrate-to-zuulv3 - I guess I'll need some help from here | 21:22 |
| *** rtjure has quit IRC | 21:22 | |
| pabelanger | jeblair: sure, np | 21:23 |
| *** felipemonteiro_ has joined #openstack-infra | 21:23 | |
| *** esberglu has quit IRC | 21:24 | |
| *** sree has quit IRC | 21:24 | |
| pabelanger | I didn't get a chance to attends todays meeting, but now that we decided to stay on zuulv3, do we want to start the process of deleting zuulv2.5 things? EG: zm01-zm04 (updated to zuulv3) and delete our zuul-launchers | 21:24 |
| jeblair | pabelanger: ++ though i think zm01-04 are the wrong operating system? | 21:24 |
| *** andreas_s has joined #openstack-infra | 21:24 | |
| jeblair | so they may need complete replacement? i'm not sure | 21:25 |
| *** smatzek has quit IRC | 21:25 | |
| clarkb | ya I think they need replacement | 21:25 |
| pabelanger | yah, we'll needed rebuild them, they are trusty | 21:25 |
| *** ijw has quit IRC | 21:25 | |
| clarkb | also zuul.o.o (and then rename zuulv3) | 21:25 |
| clarkb | or maybe just cname? | 21:26 |
| clarkb | pabelanger: it might be worth putting up an etherpad list so that we don't forget thing and can split up the work | 21:26 |
| *** felipemonteiro__ has quit IRC | 21:26 | |
| pabelanger | Yah, let me start on that now. make sure people are okay before I go off and delete things :D | 21:26 |
| *** rtjure has joined #openstack-infra | 21:27 | |
| *** wolverineav has quit IRC | 21:28 | |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/project-config master: Fix Grafana periodic counters for neutron-lib dashboard https://review.openstack.org/514801 | 21:29 |
| *** bobh has joined #openstack-infra | 21:29 | |
| pabelanger | https://etherpad.openstack.org/p/zuulv2-removal | 21:31 |
| *** rtjure has quit IRC | 21:32 | |
| *** thorst has joined #openstack-infra | 21:32 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Include a new time for UC IRC meetings https://review.openstack.org/513863 | 21:35 |
| *** vsaienk0 has quit IRC | 21:35 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: New time for Neutron drivers meeting, odd weeks https://review.openstack.org/512856 | 21:36 |
| *** thorst has quit IRC | 21:36 | |
| *** rtjure has joined #openstack-infra | 21:37 | |
| *** iyamahat has joined #openstack-infra | 21:37 | |
| *** thorst has joined #openstack-infra | 21:38 | |
| *** esberglu has joined #openstack-infra | 21:39 | |
| pabelanger | clarkb: I think that is everything for zuulv2 removal, or everything I can think of | 21:39 |
| openstackgerrit | Ian Wienand proposed openstack-infra/release-tools master: Add some notes on eol_branch.sh https://review.openstack.org/514846 | 21:39 |
| ianw | tonyb: ^ | 21:39 |
| pabelanger | jeblair: maybe you have recommendation on how to handle zookeeper on nodepool.o.o, see etherpad^ | 21:40 |
| ianw | in short, the answer to the gpg-agent thing is to switch git to using gpg2 | 21:40 |
| clarkb | pabelanger: ya I'm not coming up with anything else right now. Also we don't really need to rebuild nb01/02 | 21:40 |
| clarkb | pabelanger: we can just have 03 and 04 for now :P | 21:40 |
| openstackgerrit | Dean Troyer proposed openstack/os-client-config master: Consume the osc-functional-devstack-tips job https://review.openstack.org/514847 | 21:40 |
| *** slaweq has quit IRC | 21:40 | |
| pabelanger | clarkb: well, we should consider moving nb03.o.o out of vexxhost, due to network bandwidth. But we do need to rebuild to switch to python3. DNS could be the same | 21:41 |
| clarkb | pabelanger: we probably need to think about a) how we make zuulv3.o.o zuul.o.o at least logically and b) how to zookeeper without a nodepool.o.o | 21:41 |
| *** rtjure has quit IRC | 21:41 | |
| pabelanger | yah | 21:41 |
| tonyb | ianw: Thanks. | 21:41 |
| *** andreas_s has quit IRC | 21:42 | |
| tonyb | ianw, fungi, clarkb: If bootstrappers membership is all that's required perhaps next time we could just add me for $short_time and lighten the load on y'all | 21:42 |
| clarkb | tonyb: ya I think that is it. But do we trust you :P | 21:43 |
| clarkb | tonyb: thats actually probably a good idea | 21:43 |
| *** thorst has quit IRC | 21:43 | |
| ianw | tonyb: give me 20 minutes, let's try it out when i get back | 21:43 |
| tonyb | clarkb: I'm preety dodgy y'all shoudl be careful ;P | 21:43 |
| clarkb | infra-root ^ thoughts on tonyb's idea? | 21:44 |
| clarkb | the biggest gotcha is be careful pushing code and/or reviewing changes while you have your privs up | 21:45 |
| tonyb | clarkb: Yeah. I'd in a batch to void that | 21:45 |
| tonyb | clarkb: could I self remove when I was done? | 21:45 |
| tonyb | clarkb: My worry is that if I mess things up it'll be a firedrill for y'all, where as if it's and an infra-root then they already have the perms and skills to fix it | 21:47 |
| *** iyamahat has quit IRC | 21:47 | |
| fungi | clarkb: i'm not opposed to making tonyb an honorary temporary project bootstrapper for long enough to run a script, at least until gerrit 2.14 gives us branch deletion acls | 21:47 |
| clarkb | tonyb: let me check if you can remove yourself | 21:47 |
| *** andreas_s has joined #openstack-infra | 21:47 | |
| clarkb | you will be able to remoev yourself | 21:47 |
| fungi | i thought project bootstrappers was administrator-owned not self-owned | 21:48 |
| * fungi double-checks | 21:48 | |
| *** rtjure has joined #openstack-infra | 21:48 | |
| fungi | huh, it is indeed self-owned | 21:48 |
| *** gcb has quit IRC | 21:48 | |
| fungi | we _might_ also consider changing that | 21:48 |
| fungi | to only be administrator-controlled | 21:49 |
| fungi | (adding/removing members of the group i mean) | 21:49 |
| fungi | okay, i'm stumped on another release automation challenge if anyone's around with a second pair of eyeballs | 21:52 |
| *** andreas_s has quit IRC | 21:52 | |
| clarkb | I'll take a look | 21:52 |
| fungi | this ref was a nova pre-release tag: http://logs.openstack.org/2b/2bf4b000dfaf5d84b37dd0ce194efb509566b22c/ | 21:53 |
| *** rtjure has quit IRC | 21:53 | |
| fungi | from earlier today | 21:53 |
| fungi | in the pre-release pipeline's publish-openstack-python-tarball job i think the python-tarball playbook from project-config should have used the sign-artifacts role from zuul-jobs | 21:54 |
| openstackgerrit | Tony Breeds proposed openstack-infra/release-tools master: Add some notes on eol_branch.sh https://review.openstack.org/514846 | 21:54 |
| fungi | mentioned as a localhost role in its post phase | 21:55 |
| tonyb | ianw: fixed a nit in ^^ | 21:55 |
| fungi | clarkb: but i don't see it mentioned in the console log | 21:57 |
| clarkb | fungi: I'm reading the playbooks for that job and not seeing the signing role used | 21:57 |
| fungi | and we don't seem to have uploaded any signature along with the build artifact either | 21:57 |
| fungi | clarkb: the job uses the python-tarball playbook and in the post phase it includes the sign-artifacts role | 21:58 |
| *** rtjure has joined #openstack-infra | 21:58 | |
| fungi | http://git.openstack.org/cgit/openstack-infra/project-config/tree/playbooks/python-tarball/post.yaml#n8 | 21:58 |
| clarkb | fungi: but the job is defined in ozj so it will use the playbook from there | 21:59 |
| clarkb | fungi: I think that may be the problem, the relative path to playbooks is off by one repo | 21:59 |
| fungi | and that causes it to be silently ignored? | 22:00 |
| clarkb | fungi: release-openstack-python uses the one in project-config | 22:00 |
| *** boden has quit IRC | 22:00 | |
| clarkb | fungi: iirc the rule is you can only run playbooks within your own repo | 22:00 |
| clarkb | so less ignored and more just adhering to ^ that rule | 22:00 |
| fungi | oh, do we have a duplicate playbook? | 22:01 |
| *** aeng has quit IRC | 22:01 | |
| fungi | indeed, we seem to have | 22:01 |
| clarkb | duplicate name/path but different contents | 22:01 |
| clarkb | and in different repos | 22:02 |
| fungi | right, i'm wondering what the intended behavior was supposed to be | 22:02 |
| openstackgerrit | Dean Troyer proposed openstack-infra/project-config master: Change osc-plugin-jobs to openstackclient-plugin-jobs https://review.openstack.org/514849 | 22:02 |
| *** rtjure has quit IRC | 22:03 | |
| clarkb | fungi: I think it worked as expected it ran the post playbook from ozj where the job is defined right? | 22:03 |
| fungi | clarkb: it's almost like the author expected the post phase for the project-config playbook to be additive with the post phase for the playbook of the same name in openstack-zuul-jobs | 22:03 |
| fungi | actually, no, becasuse they both have identical copies of the same run phase | 22:05 |
| fungi | pre phase tasks are also identical except they reverse the order of ensure-tox vs revoke-sudo | 22:05 |
| *** andreas_s has joined #openstack-infra | 22:06 | |
| fungi | so more likely this is an (incomplete) transition moving the job from one place to the other? | 22:06 |
| * fungi goes hunting in git history | 22:06 | |
| *** florianf has quit IRC | 22:07 | |
| fungi | mordred: pabelanger: do you have any insight into this? it looks like both of you were hacking on these and as best as i can tell the version in openstack-zuul-jobs may be obsolete but is also the one the jobs seem to end up using? | 22:08 |
| pabelanger | fungi: sorry, which job? | 22:09 |
| fungi | pabelanger: the python-tarball playbooks in project-config and openstack-zuul-jobs | 22:09 |
| fungi | they're mostly but not quite entirely the same | 22:09 |
| pabelanger | looking | 22:09 |
| fungi | commit dates on the project-config playbook's files are a good several weeks newer | 22:10 |
| *** andreas_s has quit IRC | 22:10 | |
| fungi | but the publish-openstack-python-tarball jobs are using the playbook from openstack-zuul-jobs because that's where the job itself is defined | 22:10 |
| pabelanger | so, it depends on which job is using the playbooks | 22:10 |
| pabelanger | for example in ozj, we have both publish-openstack-python-tarball and build-openstack-python-tarball | 22:11 |
| *** slaweq has joined #openstack-infra | 22:11 | |
| pabelanger | but release-openstack-pyhon, used the playbooks in project-config | 22:11 |
| fungi | this came about because the release team and i were surprised to discover that the sign-tarball role wasn't getting applied to releases | 22:11 |
| pabelanger | so, both playbooks should be getting used, across the 3 jobs | 22:11 |
| clarkb | pabelanger: I thought the rule was you could only use the playbooks in your repo | 22:12 |
| fungi | do we have guidance on whether publish-openstack-python-tarball or release-openstack-python should be getting used? | 22:12 |
| clarkb | pabelanger: roles can be shared but playbooks are "local" | 22:12 |
| fungi | the two seem redundant to me | 22:12 |
| pabelanger | clarkb: right, that's what we do | 22:12 |
| pabelanger | so, let me find signing role | 22:13 |
| fungi | but it appears that openstack projects have publish-openstack-python-tarball right now, which doesn't sign releases because it's using the openstack-zuul-jobs python-tarball playbook rather than the project-config python-tarball playbook | 22:13 |
| fungi | the signing role is in zuul-jobs | 22:13 |
| fungi | and included by the python-tarball playbook post phase from project-config | 22:14 |
| fungi | but not by the python-tarball playbook post phase from openstack-zuul-jobs | 22:14 |
| pabelanger | Right, because it need gpg key | 22:14 |
| *** bobh has quit IRC | 22:14 | |
| pabelanger | we don't want to add that to ozj | 22:14 |
| *** wolverineav has joined #openstack-infra | 22:15 | |
| fungi | let me start over | 22:15 |
| fungi | why are we applying release jobs which don't make artifact signatures to openstack projects? i'm trying to decide if this overlap and choice of job was intentional, or accidental | 22:16 |
| clarkb | fungi: it may be intentional to not sign pre release objects? | 22:16 |
| fungi | and i worry that we have too many similarly-named jobs and identically-named playbooks in different repos with differing behavior, leading people not to know which to use | 22:16 |
| pabelanger | release-openstack-python job does sign-artifacts right? | 22:17 |
| fungi | clarkb: we were signing pre-releases with v2 | 22:17 |
| clarkb | fungi: ok so this is a regresion, your worry seems like a likely cause in that case | 22:17 |
| fungi | pabelanger: yes, but for these tags we apparently ran publish-openstack-python-tarball instead | 22:17 |
| ianw | tonyb: ok, http://paste.openstack.org/show/624527/ is where i was up to | 22:17 |
| fungi | and it's not clear to me why we have both publish-openstack-python-tarball _and_ release-openstack-python jobs | 22:17 |
| pabelanger | fungi: and tags should also be uploaded to pypi, is that right? | 22:17 |
| *** aeng has joined #openstack-infra | 22:18 | |
| fungi | tags don't get uploaded anywhere. sdists and wheels get uploaded to pypi for some projects but not for others, so perhaps that was the intended distinction and we just missed that we still want to create signatures (which only ever get uploaded to tarballs.openstack.org anyway) for projects which don't also get uploaded to pypi? | 22:18 |
| *** baoli has quit IRC | 22:19 | |
| *** andreas_s has joined #openstack-infra | 22:19 | |
| fungi | but beyond that, comparing the yaml files for the python-tarball playbook in project-config to the ones in openstack-zuul-jobs, it seems like there's a fair amount of duplication but also some unintentional divergence | 22:20 |
| fungi | which could lead to other behavior differences as well | 22:20 |
| pabelanger | I think the original idea was, publish-openstack-python-tarballs == branch tarballs for projects, which upload to tarballs.o.o, and release-openstack-python ran when we tagged a release and uploaded to pypi | 22:20 |
| tonyb | ianw: that looks like barbican, congress and cloudkitty have been done | 22:20 |
| pabelanger | trying to confirm, this have changes a little since first writing them | 22:21 |
| fungi | pabelanger: aha, so for whatever reason we ran publish-openstack-python-tarballs on tags | 22:21 |
| *** ijw has joined #openstack-infra | 22:22 | |
| *** claudiub|2 has quit IRC | 22:22 | |
| ianw | tonyb: yep ... you want to queue up designate and i'll add you to bootstrappers and we can see if it works? | 22:22 |
| *** rtjure has joined #openstack-infra | 22:22 | |
| tonyb | ianw: okay, gimme 5 to make sure I have GPG setup correctly | 22:23 |
| fungi | pabelanger: a commonality on the missing signatures is that these were tags for projects which don't upload to pypi, so that seems like an important datapoint | 22:23 |
| fungi | (cinder, glance, keystone, nova) | 22:23 |
| pabelanger | fungi: yah, so looking at the release pipelines in projects.yaml and project-templates.yaml, if we want things to end up on pypi, we should be using the release-openstack-python jobs. Otherwise, the publish-openstack-python-tarball, is really meant to upload wheels and tarballs, to tarballs.o.o | 22:24 |
| pabelanger | and, we didn't do gpg signing on branch / master tarballs | 22:24 |
| fungi | pabelanger: ahh, so we missed that there's a third class there | 22:25 |
| pabelanger | it is possible, I was able to do the testing with openstack-dev/sandbox :) | 22:26 |
| fungi | in v3 we had jobs (or combinations thereof) which upload branch-tip tarballs/wheels, jobs which uploaded tag-based tarballs/wheels with signatures but only to tarballs.openstack.org, and jobs which uploaded tag-based tarballs/wheels with signatures to tarballs.openstack.org but also uploaded the tarballs/wheels to pypi | 22:26 |
| *** salv-orlando has quit IRC | 22:26 | |
| fungi | er, i mean in v2 | 22:26 |
| *** salv-orlando has joined #openstack-infra | 22:27 | |
| *** rtjure has quit IRC | 22:27 | |
| fungi | and it appears that the first two cases got combined, resulting in projects which don't publish their releases on pypi not getting signatures uploaded to tarballs.o.o | 22:27 |
| fungi | that explains the symptoms we were seeing | 22:28 |
| pabelanger | yah, that sounds right (well wrong). | 22:28 |
| *** ijw has quit IRC | 22:28 | |
| pabelanger | sounds like we'll either need to always sign things upload to tarballs.o.o or create this 3 job | 22:28 |
| fungi | since we did those things (build artifacts and upload them to tarballs.o.o, sign artifacts and upload the signatures to tarballs.o.o, upload artifacts to pypi) as separate jobs in v2 we were able to compose them as needed | 22:29 |
| *** edmondsw has quit IRC | 22:29 | |
| pabelanger | yah | 22:29 |
| *** edmondsw has joined #openstack-infra | 22:29 | |
| fungi | however... i guess there'd be nothing wrong (now that we've massively parallelized this) with also signing branch tip tarballs/wheels | 22:29 |
| fungi | which leaves me with the other concern... what looks like some unintentional divergence between the python-tarball playbooks in project-config and in openstack-zuul-jobs and how to ensure we keep them in better sync in the future | 22:30 |
| pabelanger | yah, either way, this new job will need to live in project-config (to access gpg secret). We could call it sign-openstack-python, then parent to publish-openstack-artifacts | 22:31 |
| *** salv-orlando has quit IRC | 22:31 | |
| fungi | wondering how we should deal with things like the differing order of revoke-sudo and ensure-tox roles in the pre phase across those playbooks | 22:32 |
| *** andreas_s has quit IRC | 22:32 | |
| fungi | or the way that the post phase in one is bespoke while in the other it reuses predefined roles | 22:33 |
| *** edmondsw has quit IRC | 22:34 | |
| fungi | they've only been around for about a month and it seems to me like the code duplication there is already taking its toll resulting in changes to one not getting made to the other | 22:34 |
| pabelanger | Yah, right now we don't really share playbooks, just roles and inherit jobs. We did talk about using a meta role, but I haven't used it much | 22:35 |
| *** rtjure has joined #openstack-infra | 22:36 | |
| pabelanger | the main issue right now, is project-config cannot parent to ozj, because of ordering. So, we just copied a lot of things into project-config to make it work. Long term, I'm hoping we come up with a better method | 22:36 |
| tonyb | ianw: ready when you are ... see /pm for deatils | 22:37 |
| *** mummer has quit IRC | 22:37 | |
| fungi | pabelanger: good to know, because my knee-jerk reaction would have been to move the rest into project-config so we could cut down on the duplication | 22:38 |
| ianw | project-core: adding tonyb to bootstrappers so we can do some EOL-ing | 22:38 |
| pabelanger | fungi: right, we could do that, but means we lose untrusted (dynamic reconfiguration) jobs, making it harder to iterate on changes. | 22:38 |
| fungi | pabelanger: since having an almost identical version of this in openstack-zuul-jobs seems like more accidents waiting to happen if we already have to have a copy of most of it in project-config | 22:39 |
| pabelanger | that's the main reason, I've liked keeping them in ozj | 22:39 |
| ianw | tonyb: ok, give it a go :) | 22:39 |
| fungi | pabelanger: the dynamic reconfiguration argument is a great one, thanks | 22:40 |
| fungi | pabelanger: that's maknig me more inclined to think that we just need to have three tarball publishing jobs instead of the two | 22:40 |
| *** rtjure has quit IRC | 22:41 | |
| *** Sukhdev has joined #openstack-infra | 22:41 | |
| * tonyb crosses fingers (and toes) | 22:41 | |
| pabelanger | fungi: yah, I also think mordred likely has some thoughts on this too. But I'm happy to move the direction everybody wants too | 22:41 |
| fungi | pabelanger: though... we probably need more descriptive names. i already couldn't tell what the difference should have been between publish-openstack-python-tarball and release-openstack-python | 22:42 |
| *** andreas_s has joined #openstack-infra | 22:42 | |
| fungi | though now knowing what the intended difference is, i can sort of see it reflected in those names | 22:42 |
| SamYaple | fungi: pulish pushest to tarball.o.o, release pushes to pypi? | 22:42 |
| fungi | SamYaple: yup, named by someone who didn't consider that we also release tarballs for service projects without putting them on pypi | 22:43 |
| *** thorst has joined #openstack-infra | 22:43 | |
| fungi | so if you _didn't_ know that service project releases don't end up on pypi, the naming convention sort of made sense | 22:43 |
| *** felipemonteiro_ has quit IRC | 22:43 | |
| tonyb | ianw: openstack/designate openstack/desginate-dashboard and openstack/python-designateclient done. | 22:44 |
| SamYaple | cool. i was just guessing so they *kinda* make sense. but i would throw pypi in the name at least | 22:44 |
| SamYaple | or something | 22:44 |
| *** felipemonteiro_ has joined #openstack-infra | 22:44 | |
| tonyb | ianw: I can pastebin the log for verification that there aren't any problems | 22:44 |
| fungi | SamYaple: yeah, to actually differentiate them we probably need to | 22:44 |
| *** slaweq has quit IRC | 22:44 | |
| tonyb | ianw: http://paste.openstack.org/show/624528/ | 22:45 |
| *** mriedem has quit IRC | 22:46 | |
| ianw | tonyb: cool, didn't seem to need to abandon any changes but i imagine that will work? i say let it run | 22:46 |
| tonyb | ianw: Let's find a project with open changes to confim that. | 22:46 |
| *** andreas_s has quit IRC | 22:46 | |
| *** mriedem has joined #openstack-infra | 22:47 | |
| *** rtjure has joined #openstack-infra | 22:48 | |
| *** slaweq has joined #openstack-infra | 22:48 | |
| *** thorst has quit IRC | 22:49 | |
| tonyb | ianw: Ahh but it's complicated as *some* projects have stable-main-core as an owner. I guess I'll run it carefully until I get to the end *or* hit a snag | 22:49 |
| tonyb | ianw: as we're iun the same TZ I'll ping you if I hit a snag | 22:49 |
| ianw | tonyb: ok, sounds good | 22:50 |
| *** rtjure has quit IRC | 22:53 | |
| *** ijw has joined #openstack-infra | 22:57 | |
| *** aeng has quit IRC | 22:57 | |
| *** rtjure has joined #openstack-infra | 22:57 | |
| *** aeng has joined #openstack-infra | 22:57 | |
| *** sdague has quit IRC | 22:59 | |
| *** andreas_s has joined #openstack-infra | 23:00 | |
| *** felipemonteiro_ has quit IRC | 23:01 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/project-config master: Add a Python release job for API services https://review.openstack.org/514855 | 23:02 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/openstack-zuul-jobs master: Add a release project-template for API services https://review.openstack.org/514856 | 23:02 |
| fungi | pabelanger: clarkb: dhellmann: first couple steps ^ | 23:02 |
| fungi | working on the change to use the new project-template/job in the projects list now | 23:02 |
| *** rtjure has quit IRC | 23:02 | |
| tonyb | Running query: status:open project:openstack/kolla branch:stable/newton | 23:03 |
| tonyb | Found commit 2b72f226496d81dae511768bba79c8cc4a3b8c4f to abandon | 23:03 |
| tonyb | error: fatal: abandon not permitted | 23:03 |
| tonyb | ianw: ^^ | 23:03 |
| * tonyb is checking ACL structure | 23:03 | |
| jeblair | that may be an implicit admin permission | 23:04 |
| tonyb | jeblair: Yeah it looks like I'd need to be an admin to get passed that as that project doesn't include bootstrappers | 23:05 |
| fungi | ahh, yep, we normally allow change owners and core reviewers to abandon changes, but i _guess_ we could add that for project bootstrappers (or release managers) in the all-projects acl | 23:05 |
| jeblair | yeah, that may be something we can add | 23:05 |
| fungi | tonyb: you're already ni release-managers, right? | 23:06 |
| fungi | s/ni/in/ | 23:06 |
| armax | fungi: hi | 23:06 |
| *** ijw has quit IRC | 23:06 | |
| fungi | armax: what's up? | 23:06 |
| tonyb | fungi: Yes but let me double check | 23:06 |
| armax | I am trying to parse your question on the mailing list about neutron and neutron-lib | 23:06 |
| armax | but I am not sure I fully understand it | 23:06 |
| armax | I figured asking here would be quicker | 23:07 |
| tonyb | fungi: Yup https://review.openstack.org/#/admin/groups/11,members | 23:07 |
| *** rtjure has joined #openstack-infra | 23:07 | |
| fungi | armax: right now we build sdist tarballs as part of the release process by doing so under a tox-built virtualenv | 23:07 |
| armax | OK | 23:07 |
| *** ijw has joined #openstack-infra | 23:07 | |
| fungi | armax: though recent changes to the project testing interface guidelines, in an effort to bring us closer to the mainstream python ecosystem, indicate we should work on not using tox but just calling `python setup.py sdist` | 23:07 |
| fungi | the theory being you shouldn't need runtime dependencies installed just to generate an sdist or build a wheel | 23:08 |
| *** andreas_s has quit IRC | 23:09 | |
| fungi | so if neutron plugins are including neutron source installs in theor tox environments just to satisfy their own installs for things like unit test runs, then odds are they're not really needed in our sdist and wheel jobs | 23:09 |
| armax | I see, so you’re saying that we may not need to include the required-projects dependency for the release job definition? | 23:10 |
| armax | assuming I understood correctly | 23:11 |
| fungi | and all the above basically goes for directly calling sphinx-build to generate html documentation out of rst content | 23:11 |
| fungi | armax: pretty much, yes | 23:11 |
| armax | neutron subprojects ends up needing neutron because of the sheer amount of direct imports they make to neutron modules | 23:11 |
| armax | which we’re trying to break, slowly overtime | 23:11 |
| *** rtjure has quit IRC | 23:11 | |
| fungi | armax: right, but when doing release and documentation activities, we in theory shouldn't need to install those projects | 23:11 |
| armax | yup | 23:12 |
| armax | I see | 23:12 |
| fungi | and so their dependencies are mostly irrelevant | 23:12 |
| armax | makes sense | 23:12 |
| *** askb has quit IRC | 23:12 | |
| ianw | jeblair / fungi : https://git.openstack.org/cgit/openstack-infra/project-config/tree/tools/normalize_acl.py#n189 <- saying bootstrappers can abandon here though? | 23:12 |
| jeblair | i guess i have two questions about that, both relating to docs: 1) what if a project needs a sphinx plugin? 2) do the imports need to work for autodoc? (and therefore all the deps be installed) | 23:12 |
| armax | I wonder why http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/project-templates.yaml#n136 was added in the first place | 23:13 |
| *** andreas_s has joined #openstack-infra | 23:13 | |
| *** hongbin has quit IRC | 23:14 | |
| fungi | armax: because right now we're still running the sdist and wheel build process under tox, and tox is installing the project and needs its dependencies | 23:14 |
| fungi | the idea being we'd drop all that and the publish-to-pypi-neutron project-template could go away at that point | 23:14 |
| jeblair | ianw: yeah, i think that means that we make sure that boostrappers can abandon, even if a project sets an exclusive acl. which suggests that we also made that the default (i haven't looked) | 23:14 |
| armax | gotcha, but if we stopped doing that, then we could get rid of that | 23:14 |
| fungi | armax: i was basically challenging dhellmann's assertion that it was blocked on the neutron-lib breakout work | 23:15 |
| fungi | i think it could happen way, way sooner instead | 23:15 |
| armax | OK, but that would be predicated on the fact that the release job would stop using tox | 23:16 |
| armax | correct? | 23:16 |
| fungi | yep | 23:16 |
| fungi | just mostnig wondering whether you knew something i didn't about a long-term need for neutron or neutron-lib in the release process for those plugins | 23:16 |
| fungi | s/mostnig/mostly/ | 23:16 |
| *** bobh has joined #openstack-infra | 23:17 | |
| armax | to my knowledge the need for neutron is tied to imports in the code, there may be dirty coupling in some scripts for controlling gate behavior | 23:17 |
| armax | but that’s far easier ot decouple | 23:18 |
| *** ijw has quit IRC | 23:18 | |
| armax | if it happened by accident | 23:18 |
| ianw | jeblair: https://review.openstack.org/#/admin/projects/All-Projects,access <- abandon does not seem to be granted here | 23:18 |
| *** rtjure has joined #openstack-infra | 23:18 | |
| *** ijw has joined #openstack-infra | 23:19 | |
| *** bobh has quit IRC | 23:19 | |
| jeblair | ianw: i'd be fine adding it there | 23:20 |
| *** sree has joined #openstack-infra | 23:20 | |
| jeblair | (should update the docs if we do) | 23:20 |
| fungi | armax: okay, cool, sounds promising then that we could drop the special neutron-specific release project-template when we switch to not using tox for building sdists and wheels | 23:20 |
| *** bobh has joined #openstack-infra | 23:21 | |
| *** slaweq has quit IRC | 23:22 | |
| *** andreas_s has quit IRC | 23:22 | |
| ianw | jeblair: what seems odd though is, how do administrators have rights to abandon anything? it doesn't seem to be set there | 23:22 |
| *** andreas_s has joined #openstack-infra | 23:23 | |
| *** rtjure has quit IRC | 23:23 | |
| jeblair | ianw: there are a number of implicit admin rights. for some reason. i think that's one of them. (again, not sure with latest version, etc) | 23:23 |
| ianw | ok, so it would be give "abandon" rights to ref/heads/* to project bootstrappers? | 23:24 |
| *** sree has quit IRC | 23:25 | |
| *** gmann_afk is now known as gmann | 23:25 | |
| *** esberglu has quit IRC | 23:25 | |
| *** smatzek has joined #openstack-infra | 23:26 | |
| *** slaweq has joined #openstack-infra | 23:26 | |
| *** xarses has quit IRC | 23:26 | |
| *** salv-orlando has joined #openstack-infra | 23:28 | |
| ianw | jeblair: or, maybe it's better for release managers to gain that permission? | 23:28 |
| *** rtjure has joined #openstack-infra | 23:28 | |
| *** gyee has joined #openstack-infra | 23:28 | |
| fungi | ianw: yeah, i'm leaning toward release managers, because they have reason to abandon stuff since we're also giving them branch deletion privs once we can (as an analog for them having branch creation privs already) | 23:28 |
| jeblair | fungi: good point | 23:29 |
| ianw | so abandon rights to ref/heads/* for release managers? | 23:29 |
| fungi | whereas project-bootstrappers rarely, if ever, would have any automation-driven need to abandon changes | 23:29 |
| ianw | in all-projects | 23:29 |
| fungi | [access "refs/heads/*"] | 23:30 |
| fungi | abandon = group Release Managers | 23:30 |
| *** smatzek has quit IRC | 23:30 | |
| ianw | or maybe just refs/* ? | 23:30 |
| fungi | oh, though looking at other project-specific acls... | 23:30 |
| ianw | release managers already have "push signed tag" on refs/* | 23:30 |
| *** andreas_s has quit IRC | 23:31 | |
| fungi | yeah, nevermind what i was going to say. some projects have exclusivegrouppermissions for abandon on [access "refs/heads/stable/*"] but they also explicitly grant it for <project>-stable-maint groups which generally have the stable-maint-core group included in them | 23:32 |
| *** salv-orlando has quit IRC | 23:32 | |
| *** tosky has quit IRC | 23:33 | |
| fungi | so if someone is a member if release managers (which is included in stable-maint-core) then they aren't affected by that exclusive abandon for stable anyway | 23:33 |
| *** mlavalle has quit IRC | 23:33 | |
| *** rtjure has quit IRC | 23:33 | |
| fungi | worst case we have some projects which declare exclusive abandon on refs/heads/stable/* without including some group that ultimately includes release managers, and if that's the case we'll figure out a workaround (like adding the release managers as included for that group) | 23:34 |
| ianw | i've not touched all-projects before, do we just UI edit it? | 23:34 |
| fungi | ianw: yeah, that's what i do. and include a useful/descriptive commit message where it allows you to do so | 23:34 |
| jeblair | also update the docs | 23:36 |
| jeblair | https://docs.openstack.org/infra/system-config/gerrit.html#access-controls | 23:36 |
| *** andreas_s has joined #openstack-infra | 23:36 | |
| * tonyb is sorry for the extra work this caused | 23:37 | |
| *** weshay|ruck is now known as weshay|PTO | 23:38 | |
| ianw | fungi / jeblair: https://review.openstack.org/#/c/514862/ | 23:38 |
| *** rtjure has joined #openstack-infra | 23:39 | |
| *** smatzek has joined #openstack-infra | 23:41 | |
| fungi | right | 23:43 |
| *** rtjure has quit IRC | 23:44 | |
| fungi | grr. gertty gave me an "unable to sync change" when i tried to pull that one, followed by "git commits not present in local repository" on a subsequent try | 23:44 |
| fungi | oh! that's against all-projects | 23:46 |
| fungi | i haven't used the propose edits option in the acl editor, just usually treated the documentation change as the review process and then directly committed the all-projects acl update in the acl editor rather than proposing it | 23:47 |
| fungi | we don't really have permissions to vote on that or anything | 23:48 |
| fungi | nor ci to handle the merging | 23:48 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Add notes on Release Managers abandon permissions https://review.openstack.org/514863 | 23:49 |
| *** smatzek has quit IRC | 23:49 | |
| ianw | fungi: no, i just wanted to see the diff :) ^ is docs | 23:49 |
| *** bobh has quit IRC | 23:49 | |
| fungi | got it. we also used to be able to look at the commit history and diff in gitweb, back when we still had that | 23:50 |
| *** andreas_s has quit IRC | 23:50 | |
| *** rtjure has joined #openstack-infra | 23:50 | |
| *** edmondsw has joined #openstack-infra | 23:55 | |
| *** rtjure has quit IRC | 23:55 | |
| *** andreas_s has joined #openstack-infra | 23:55 | |
| ianw | fungi: apropos https://review.openstack.org/#/c/514591 ... are you aware of any debugging as to why the zuul check tests don't show up in changes like https://review.openstack.org/#/c/512450/ ? | 23:55 |
| ianw | i'm tentatively opening firebug now :/ | 23:55 |
| *** Swami has quit IRC | 23:56 | |
| fungi | ianw: i believe there's a change up for review to solve that | 23:57 |
| * fungi finds | 23:57 | |
| fungi | oh, you mentioned https://review.openstack.org/514591 | 23:57 |
| ianw | it seems to be that there's no pipeline | 23:57 |
| fungi | which you just approved | 23:57 |
| ianw | it just says "Build succeeded." | 23:58 |
| fungi | i think that's what it's trying to solve | 23:58 |
| ianw | and the regex is looking for Build succeeded (\w) | 23:58 |
| *** tbarron|bbl is now known as tbarron | 23:58 | |
| fungi | ahh! | 23:58 |
| fungi | you mean the comment from zuul doesn't mention the pipeline | 23:58 |
| ianw | yeah | 23:58 |
| fungi | great question, i do not know that it's been brought up yet | 23:58 |
| ianw | that change matches pipelines with a "-" in them | 23:58 |
| ianw | which seems another valid failure | 23:58 |
| fungi | yup | 23:59 |
| *** camunoz has quit IRC | 23:59 | |
| *** edmondsw has quit IRC | 23:59 | |
| *** andreas_s has quit IRC | 23:59 | |
| fungi | this explains my confusion over why the lack of match on - would have just suddenly caused this to stop working under v3 | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!