| *** rossella_s has quit IRC | 00:00 | |
| *** rossella_s has joined #openstack-infra | 00:00 | |
| ianw | hmm, it just continued on | 00:02 |
|---|---|---|
| *** piet has joined #openstack-infra | 00:02 | |
| ianw | so it must have been stuck for about 10 minutes? | 00:02 |
| *** sflanigan has joined #openstack-infra | 00:05 | |
| *** sflanigan has joined #openstack-infra | 00:05 | |
| *** inc0_ has quit IRC | 00:06 | |
| clarkb | maybe its not being forceful enough and is trying to shift time slowly to converge? | 00:10 |
| *** Apoorva has quit IRC | 00:11 | |
| lifeless | ntp won't converge under some scircumstances | 00:12 |
| lifeless | e.g. too erratic a hw clock | 00:12 |
| *** sputnik13 has quit IRC | 00:14 | |
| openstackgerrit | Matthew Thode proposed openstack-infra/project-config: Add Gentoo support to nodepool https://review.openstack.org/310865 | 00:15 |
| prometheanfire | bkero: https://review.openstack.org/310865 made it all the way though the image build, baring puppet fails | 00:16 |
| *** yamahata has quit IRC | 00:17 | |
| openstackgerrit | Matthew Thode proposed openstack-infra/project-config: Add Gentoo support to nodepool https://review.openstack.org/310865 | 00:17 |
| bkero | prometheanfire: I'm not actually sure everything run-tox.sh dib tests | 00:18 |
| bkero | looks like lint | 00:18 |
| *** jcoufal has quit IRC | 00:19 | |
| *** jcoufal has joined #openstack-infra | 00:20 | |
| prometheanfire | sec | 00:20 |
| *** sdake has quit IRC | 00:23 | |
| *** cloudtrainme has quit IRC | 00:24 | |
| *** hwcomcn has joined #openstack-infra | 00:24 | |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/puppet-hound: Fix example config.json puppet path https://review.openstack.org/339265 | 00:25 |
| *** tqtran has quit IRC | 00:25 | |
| *** yamamoto_ has joined #openstack-infra | 00:25 | |
| *** xarses has joined #openstack-infra | 00:26 | |
| *** cloudtrainme has joined #openstack-infra | 00:26 | |
| openstackgerrit | Matthew Thode proposed openstack/diskimage-builder: Optionally remove portage files https://review.openstack.org/348536 | 00:27 |
| openstackgerrit | Matthew Thode proposed openstack-infra/project-config: Add Gentoo support to nodepool https://review.openstack.org/310865 | 00:27 |
| prometheanfire | bkero: ^ that's a better way | 00:27 |
| prometheanfire | and not really depending on eachother | 00:28 |
| *** jcoufal has quit IRC | 00:28 | |
| *** ddieterly has joined #openstack-infra | 00:29 | |
| bkero | ok | 00:29 |
| openstackgerrit | K Jonathan Harker proposed openstack-infra/puppet-hound: Make the git source uri configurable https://review.openstack.org/348593 | 00:30 |
| *** yamamoto_ has quit IRC | 00:31 | |
| prometheanfire | the export didn't work anyway | 00:31 |
| prometheanfire | defaulting to not remove is better | 00:31 |
| *** cloudtrainme has quit IRC | 00:31 | |
| bkero | prometheanfire: I guess I'd need to test it by using dib, pulling down your patchset, and trying to build a nodepool image using your proposed elements | 00:32 |
| clarkb | bandersnatch is up to pydirix | 00:32 |
| ianw | clarkb / lifeless: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=808864 seems related "status line contains leap_none but took 40 minutes to reach that" | 00:32 |
| openstack | Debian bug 808864 in ntp "ntp-wait takes very long" [Minor,Open] | 00:32 |
| prometheanfire | bkero: I had to add 2 workarounds for not having puppet working, but ya, that's what I did | 00:35 |
| bkero | prometheanfire: that's a lot of iteration in just today :) | 00:35 |
| prometheanfire | each itteration was a build :P | 00:35 |
| *** kien-ha has joined #openstack-infra | 00:35 | |
| prometheanfire | just how I work | 00:35 |
| phschwartz | :( gertty just does not like me today. (I made the mistake of blowing away my database) | 00:37 |
| prometheanfire | phschwartz: lol | 00:37 |
| openstackgerrit | Joshua Harlow proposed openstack-infra/jenkins-job-builder: Use concurrent.futures (a built-in) to remove custom workers https://review.openstack.org/348595 | 00:39 |
| *** jcoufal has joined #openstack-infra | 00:39 | |
| *** woodster_ has quit IRC | 00:39 | |
| *** Swami has quit IRC | 00:41 | |
| *** andymaier has quit IRC | 00:44 | |
| *** kushal has quit IRC | 00:46 | |
| *** yamahata has joined #openstack-infra | 00:46 | |
| *** ddieterly is now known as ddieterly[away] | 00:47 | |
| *** piet has quit IRC | 00:48 | |
| *** thorst has joined #openstack-infra | 00:48 | |
| *** piet has joined #openstack-infra | 00:49 | |
| *** jamielennox is now known as jamielennox|away | 00:49 | |
| openstackgerrit | Philip Marc Schwartz proposed openstack-infra/jenkins-job-builder: Correct configuration file usage. https://review.openstack.org/347984 | 00:50 |
| prometheanfire | bkero: this is my workaround for puppet not running right http://bpaste.net/show/48ffa07c380e | 00:50 |
| prometheanfire | bkero: so you'll need that on top of my changes to build the image | 00:51 |
| *** ddieterly[away] has quit IRC | 00:51 | |
| bkero | Where is the jenkins user created in the other images? | 00:51 |
| prometheanfire | puppet | 00:52 |
| *** aviau has quit IRC | 00:52 | |
| *** aviau has joined #openstack-infra | 00:52 | |
| *** tonytan4ever has joined #openstack-infra | 00:53 | |
| *** thorst has quit IRC | 00:54 | |
| bkero | Ah, and you're not running the puppet yet | 00:54 |
| *** thorst has joined #openstack-infra | 00:54 | |
| *** hwcomcn has quit IRC | 00:55 | |
| prometheanfire | nope | 00:56 |
| *** gyee has quit IRC | 00:57 | |
| prometheanfire | the puppet stuff failing doesn't kill the build though | 00:57 |
| prometheanfire | DIB_INSTALLTYPE_pip_and_virtualenv=package DIB_INSTALLTYPE_simple_init=repo disk-image-create --no-tmpfs -o devstack-gate gentoo vm openstack-repos puppet nodepool-base cache-bindep infra-package-needs simple-init growroot | 00:58 |
| *** tonytan4ever has quit IRC | 00:58 | |
| prometheanfire | that's my command to run | 00:58 |
| *** thorst has quit IRC | 00:59 | |
| prometheanfire | bkero: https://gist.github.com/prometheanfire/285bc292724f8e6d86070df288d508a1 there you go, aslo including my envars | 00:59 |
| *** elo has quit IRC | 00:59 | |
| bkero | thanks | 00:59 |
| prometheanfire | image name could be changed, as this isn't really for devstack, but meh | 00:59 |
| ianw | clarkb: see if this gets rejected -> https://github.com/puppetlabs/puppetlabs-ntp/pull/338 ... first guess is it will | 00:59 |
| *** kushal has joined #openstack-infra | 01:01 | |
| *** roxanaghe has joined #openstack-infra | 01:03 | |
| *** jcoufal has quit IRC | 01:06 | |
| *** roxanaghe has quit IRC | 01:07 | |
| *** jcoufal has joined #openstack-infra | 01:08 | |
| *** rbuzatu has joined #openstack-infra | 01:08 | |
| *** mmandel has joined #openstack-infra | 01:09 | |
| *** rbuzatu has quit IRC | 01:09 | |
| *** yamahata has quit IRC | 01:10 | |
| *** itisha has quit IRC | 01:10 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update LogStash plugin to use convert xml https://review.openstack.org/348599 | 01:10 |
| openstackgerrit | Armando Migliaccio proposed openstack-infra/project-config: Append ubuntu-xenial to gate-neutron-python27 for Neutron Grafana https://review.openstack.org/348600 | 01:11 |
| *** elo has joined #openstack-infra | 01:11 | |
| *** Liuqing has joined #openstack-infra | 01:12 | |
| *** zhurong has joined #openstack-infra | 01:12 | |
| *** sdake has joined #openstack-infra | 01:13 | |
| *** sdake has quit IRC | 01:13 | |
| *** sdake has joined #openstack-infra | 01:13 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config: Add ntp-perl package for RedHat platforms https://review.openstack.org/348601 | 01:14 |
| *** osnexus has quit IRC | 01:14 | |
| *** sdake_ has joined #openstack-infra | 01:15 | |
| *** yanyanhu has joined #openstack-infra | 01:15 | |
| *** pt_15 has quit IRC | 01:16 | |
| *** sdake has quit IRC | 01:18 | |
| *** jcoufal has quit IRC | 01:20 | |
| *** jcoufal has joined #openstack-infra | 01:20 | |
| *** asettle has joined #openstack-infra | 01:22 | |
| *** yamamoto_ has joined #openstack-infra | 01:27 | |
| *** asettle has quit IRC | 01:27 | |
| *** hemanthm|afk is now known as hemanthm | 01:27 | |
| *** piet has quit IRC | 01:30 | |
| *** Daisy has joined #openstack-infra | 01:30 | |
| *** piet has joined #openstack-infra | 01:30 | |
| *** yamamoto_ has quit IRC | 01:32 | |
| *** Daisy has quit IRC | 01:33 | |
| *** tonytan4ever has joined #openstack-infra | 01:34 | |
| *** jamielennox|away is now known as jamielennox | 01:34 | |
| *** gordc has quit IRC | 01:34 | |
| *** rfolco has quit IRC | 01:38 | |
| *** winggundamth has quit IRC | 01:39 | |
| openstackgerrit | Qiming Teng proposed openstack-infra/project-config: Enable py35 jobs to vote for senlin(client) https://review.openstack.org/348606 | 01:39 |
| *** piet has quit IRC | 01:41 | |
| *** yamahata has joined #openstack-infra | 01:44 | |
| *** mmandel has quit IRC | 01:45 | |
| *** mriedem has quit IRC | 01:46 | |
| *** winggundamth has joined #openstack-infra | 01:46 | |
| *** tphummel has quit IRC | 01:48 | |
| *** rlandy has quit IRC | 01:50 | |
| *** jcoufal has quit IRC | 01:51 | |
| *** Daisy has joined #openstack-infra | 01:51 | |
| *** jcoufal has joined #openstack-infra | 01:52 | |
| *** thorst has joined #openstack-infra | 02:04 | |
| *** thorst has quit IRC | 02:05 | |
| *** thorst has joined #openstack-infra | 02:05 | |
| *** thorst has quit IRC | 02:10 | |
| *** jcoufal has quit IRC | 02:12 | |
| *** PollyZ has joined #openstack-infra | 02:12 | |
| prometheanfire | is gate unbroken? | 02:15 |
| prometheanfire | want to know before I submit rechecks | 02:15 |
| *** yamahata has quit IRC | 02:15 | |
| *** PollyZ has quit IRC | 02:18 | |
| *** sdake_ has quit IRC | 02:19 | |
| *** zhurong has quit IRC | 02:24 | |
| *** zhurong has joined #openstack-infra | 02:24 | |
| *** PollyZ has joined #openstack-infra | 02:26 | |
| *** yamamoto_ has joined #openstack-infra | 02:28 | |
| *** Daisy has quit IRC | 02:29 | |
| *** PollyZ has quit IRC | 02:31 | |
| *** Liuqing has quit IRC | 02:32 | |
| *** jamielennox is now known as jamielennox|away | 02:32 | |
| *** gordc has joined #openstack-infra | 02:33 | |
| *** Liuqing has joined #openstack-infra | 02:33 | |
| *** yamamoto_ has quit IRC | 02:34 | |
| prometheanfire | ok, well, submitting... | 02:36 |
| *** Liuqing has quit IRC | 02:37 | |
| *** Liuqing has joined #openstack-infra | 02:39 | |
| *** Daisy has joined #openstack-infra | 02:39 | |
| *** PollyZ has joined #openstack-infra | 02:40 | |
| *** tonytan4ever has quit IRC | 02:41 | |
| *** zz_dimtruck is now known as dimtruck | 02:43 | |
| *** PollyZ has quit IRC | 02:44 | |
| *** claudiub|2 has quit IRC | 02:46 | |
| *** gouthamr_ has quit IRC | 02:51 | |
| *** PollyZ has joined #openstack-infra | 02:53 | |
| *** elo has quit IRC | 02:56 | |
| *** yuanying_ has quit IRC | 02:57 | |
| *** PollyZ has quit IRC | 02:58 | |
| *** elo has joined #openstack-infra | 02:59 | |
| *** PollyZ has joined #openstack-infra | 03:07 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Add GitBucket plugin support https://review.openstack.org/348702 | 03:09 |
| *** PollyZ has quit IRC | 03:11 | |
| *** Daisy has quit IRC | 03:16 | |
| *** Daisy has joined #openstack-infra | 03:18 | |
| *** PollyZ has joined #openstack-infra | 03:20 | |
| *** PollyZ has quit IRC | 03:25 | |
| *** camunoz has joined #openstack-infra | 03:26 | |
| *** sxmxc has joined #openstack-infra | 03:29 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config: Add ntp-perl package for RedHat platforms https://review.openstack.org/348601 | 03:30 |
| *** yamamoto_ has joined #openstack-infra | 03:31 | |
| *** PollyZ has joined #openstack-infra | 03:34 | |
| *** yamamoto_ has quit IRC | 03:36 | |
| *** PollyZ has quit IRC | 03:39 | |
| *** Sukhdev has joined #openstack-infra | 03:40 | |
| *** tonytan4ever has joined #openstack-infra | 03:42 | |
| *** yamamoto_ has joined #openstack-infra | 03:44 | |
| *** gordc has quit IRC | 03:45 | |
| *** tonytan4ever has quit IRC | 03:47 | |
| *** PollyZ has joined #openstack-infra | 03:48 | |
| prometheanfire | guess not | 03:49 |
| *** yuanying has joined #openstack-infra | 03:51 | |
| *** PollyZ has quit IRC | 03:52 | |
| *** gordc has joined #openstack-infra | 03:53 | |
| *** coolsvap_ has joined #openstack-infra | 03:55 | |
| *** coolsvap_ is now known as coolsvap | 03:57 | |
| openstackgerrit | Steve Baker proposed openstack-infra/devstack-gate: Remove unused heat config https://review.openstack.org/348719 | 03:58 |
| *** sxmxc has quit IRC | 03:59 | |
| openstackgerrit | Merged openstack/diskimage-builder: Generalize logic for skipping final image generation https://review.openstack.org/334042 | 03:59 |
| *** PollyZ has joined #openstack-infra | 04:01 | |
| *** roxanaghe has joined #openstack-infra | 04:04 | |
| *** gordc has quit IRC | 04:04 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config: Add ntp-perl package for RedHat platforms https://review.openstack.org/348601 | 04:04 |
| *** PollyZ has quit IRC | 04:06 | |
| ianw | clarkb: yay!? kernel panic on osic ... | 04:06 |
| *** Daisy has quit IRC | 04:07 | |
| *** roxanaghe has quit IRC | 04:09 | |
| prometheanfire | ianw: consistent? we need to contact? | 04:11 |
| ianw | prometheanfire: sorry, just for f24 jobs. it would explain why the nodes disappear ... for whatever reason RAX dies with no info, but i at least get some help from osic | 04:12 |
| *** links has joined #openstack-infra | 04:13 | |
| cloudnull | ianw: if you need anything from within OSIC cluster let me know. | 04:13 |
| prometheanfire | cloudnull: hi :D | 04:14 |
| prometheanfire | cloudnull: was gonna bug you if something turned up | 04:14 |
| cloudnull | not that I can help you with a kernel panic, but if there's something cluster related I can help with, i'd be happy to lend a hand | 04:14 |
| cloudnull | o/ prometheanfire | 04:14 |
| *** PollyZ has joined #openstack-infra | 04:15 | |
| *** PollyZ has quit IRC | 04:20 | |
| *** Sukhdev has quit IRC | 04:24 | |
| *** Na3iL has quit IRC | 04:24 | |
| *** Sukhdev has joined #openstack-infra | 04:25 | |
| *** kushal has quit IRC | 04:25 | |
| *** PollyZ has joined #openstack-infra | 04:28 | |
| *** Sukhdev has quit IRC | 04:29 | |
| *** Daisy has joined #openstack-infra | 04:29 | |
| *** maishsk has joined #openstack-infra | 04:31 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Add display-name option to github plugin https://review.openstack.org/348726 | 04:33 |
| *** yamamoto_ has quit IRC | 04:34 | |
| *** PollyZ has quit IRC | 04:34 | |
| *** krtaylor has joined #openstack-infra | 04:34 | |
| *** psachin has joined #openstack-infra | 04:35 | |
| *** Na3iL has joined #openstack-infra | 04:36 | |
| *** maishsk has quit IRC | 04:38 | |
| *** rbuzatu has joined #openstack-infra | 04:40 | |
| *** rbuzatu has quit IRC | 04:45 | |
| *** tonytan4ever has joined #openstack-infra | 04:47 | |
| *** elo has quit IRC | 04:56 | |
| *** elo has joined #openstack-infra | 05:00 | |
| *** Daisy has quit IRC | 05:04 | |
| *** Daisy has joined #openstack-infra | 05:06 | |
| *** PollyZ has joined #openstack-infra | 05:09 | |
| *** rbuzatu has joined #openstack-infra | 05:09 | |
| *** PollyZ has quit IRC | 05:14 | |
| *** yamamoto_ has joined #openstack-infra | 05:15 | |
| *** sarob has quit IRC | 05:18 | |
| *** amitkqed has quit IRC | 05:19 | |
| *** yanyanhu has quit IRC | 05:19 | |
| *** amitkqed has joined #openstack-infra | 05:20 | |
| *** yanyanhu has joined #openstack-infra | 05:21 | |
| *** maishsk has joined #openstack-infra | 05:28 | |
| *** rbuzatu has quit IRC | 05:29 | |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config: neutron-fwaas: Fix a typo (ENABLE_SERVICES vs ENABLED_SERVICES) https://review.openstack.org/348734 | 05:30 |
| *** rbuzatu has joined #openstack-infra | 05:31 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update Cobertura docs to be 4 space indent syntax https://review.openstack.org/348736 | 05:35 |
| *** PollyZ has joined #openstack-infra | 05:36 | |
| *** maishsk has quit IRC | 05:36 | |
| *** PollyZ has quit IRC | 05:41 | |
| *** megm has quit IRC | 05:42 | |
| *** tphummel has joined #openstack-infra | 05:44 | |
| *** megm has joined #openstack-infra | 05:44 | |
| *** aeng has quit IRC | 05:46 | |
| *** amotoki has joined #openstack-infra | 05:46 | |
| *** esikachev has joined #openstack-infra | 05:48 | |
| *** pgadiya has joined #openstack-infra | 05:50 | |
| *** PollyZ has joined #openstack-infra | 05:50 | |
| *** PollyZ has quit IRC | 05:55 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update FindBugs docs to 4 space indent syntax https://review.openstack.org/348740 | 05:58 |
| *** harlowja_at_home has joined #openstack-infra | 05:59 | |
| *** esikachev has quit IRC | 05:59 | |
| *** kdas__ has joined #openstack-infra | 06:00 | |
| *** gouthamr has joined #openstack-infra | 06:01 | |
| *** hashar has joined #openstack-infra | 06:02 | |
| *** kdas__ has quit IRC | 06:02 | |
| *** PollyZ has joined #openstack-infra | 06:04 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update rebuild plugin to use convert xml https://review.openstack.org/346999 | 06:04 |
| *** Ravikiran_K has joined #openstack-infra | 06:06 | |
| *** david-lyle has joined #openstack-infra | 06:07 | |
| *** PollyZ has quit IRC | 06:09 | |
| *** gouthamr has quit IRC | 06:10 | |
| *** e0ne has joined #openstack-infra | 06:12 | |
| *** Daisy has quit IRC | 06:15 | |
| *** PollyZ has joined #openstack-infra | 06:17 | |
| *** nwkarsten has joined #openstack-infra | 06:17 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Fix service enablement for ceilometer https://review.openstack.org/348747 | 06:18 |
| AJaeger | jd__, please check ^ | 06:18 |
| flip214 | help, please: what does it mean if openstack logs are cut off in the middle? | 06:19 |
| flip214 | http://logs.openstack.org/67/347267/3/check/gate-cinder-python34-db/1fd931d/console.html | 06:19 |
| flip214 | should I just "recheck" them, or is there anything else that needs to be done? | 06:20 |
| *** harlowja_at_home has quit IRC | 06:20 | |
| *** nwkarsten has quit IRC | 06:20 | |
| *** nwkarsten has joined #openstack-infra | 06:21 | |
| *** PollyZ has quit IRC | 06:22 | |
| openstackgerrit | Ben Kero proposed openstack-infra/project-config: Move chef jobs on xenial https://review.openstack.org/347633 | 06:23 |
| bkero | AJaeger: ^ | 06:24 |
| AJaeger | morning, bkero ! | 06:26 |
| AJaeger | will review... | 06:26 |
| bkero | :) | 06:26 |
| * bkero waves good "morning" | 06:26 | |
| bkero | is 2326 here | 06:26 |
| *** nwkarsten has quit IRC | 06:26 | |
| bkero | and my phone makes funny noises | 06:26 |
| AJaeger | bkero: 8:26 here ;) | 06:26 |
| AJaeger | bkero: so, good night! | 06:27 |
| bkero | oh cest | 06:27 |
| *** Daisy has joined #openstack-infra | 06:27 | |
| *** kzaitsev_mb has joined #openstack-infra | 06:28 | |
| AJaeger | yes, CEST. | 06:28 |
| AJaeger | flip214: recheck seems the best way - if it happens again, somebody might need to check server logs... | 06:28 |
| *** Daisy has quit IRC | 06:29 | |
| *** kushal has joined #openstack-infra | 06:29 | |
| bkero | c'est la dort | 06:29 |
| flip214 | AJaeger: thanks. is "recheck Jenkins" the right way? I don't need it that often, and a google search only showed mailing list posts from 2012... | 06:29 |
| AJaeger | just "recheck" | 06:29 |
| flip214 | merci | 06:29 |
| *** tonytan4ever has quit IRC | 06:30 | |
| *** PollyZ has joined #openstack-infra | 06:31 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Fix service enablement for ceilometer https://review.openstack.org/348747 | 06:31 |
| *** camunoz has quit IRC | 06:34 | |
| *** csomerville has quit IRC | 06:35 | |
| *** cody-somerville has joined #openstack-infra | 06:36 | |
| *** PollyZ has quit IRC | 06:36 | |
| *** armax has quit IRC | 06:36 | |
| *** Hal has joined #openstack-infra | 06:37 | |
| *** tesseract- has joined #openstack-infra | 06:37 | |
| *** Hal is now known as Guest58208 | 06:37 | |
| *** andymaier has joined #openstack-infra | 06:41 | |
| *** gongysh has joined #openstack-infra | 06:42 | |
| *** sandanar has joined #openstack-infra | 06:44 | |
| *** PollyZ has joined #openstack-infra | 06:44 | |
| *** david-lyle has quit IRC | 06:45 | |
| *** pcaruana has joined #openstack-infra | 06:45 | |
| openstackgerrit | Merged openstack-infra/project-config: Add diskimage-builder nodepool test https://review.openstack.org/345816 | 06:48 |
| *** tphummel has quit IRC | 06:48 | |
| *** PollyZ has quit IRC | 06:50 | |
| *** kushal has quit IRC | 06:50 | |
| openstackgerrit | Merged openstack-infra/project-config: use git directly instead of through zuul-cloner https://review.openstack.org/348549 | 06:52 |
| *** matrohon has joined #openstack-infra | 06:52 | |
| *** itisha has joined #openstack-infra | 06:53 | |
| *** roxanaghe has joined #openstack-infra | 06:53 | |
| *** pt_15 has joined #openstack-infra | 06:54 | |
| *** pahuang has quit IRC | 06:55 | |
| *** Daisy has joined #openstack-infra | 06:56 | |
| *** rdx_matt has joined #openstack-infra | 06:57 | |
| *** roxanaghe has quit IRC | 06:58 | |
| flip214 | Is there a bot that can relay messages to people when they log in to IRC? | 06:59 |
| *** ccamacho has joined #openstack-infra | 06:59 | |
| yolanda | good morning | 07:00 |
| odyssey4me | o/ | 07:00 |
| bkero | flip214: it is called memoserv | 07:00 |
| bkero | flip214: /msg memoserv help | 07:00 |
| *** kien-ha has quit IRC | 07:01 | |
| flip214 | bkero: thank you! | 07:01 |
| *** Daisy has quit IRC | 07:01 | |
| bkero | You're welcome | 07:02 |
| *** zhenguo has joined #openstack-infra | 07:04 | |
| *** alee has quit IRC | 07:08 | |
| *** Daisy has joined #openstack-infra | 07:09 | |
| *** kzaitsev_mb has quit IRC | 07:09 | |
| *** nherciu has joined #openstack-infra | 07:10 | |
| *** kzaitsev_mb has joined #openstack-infra | 07:12 | |
| *** rdx_matt has quit IRC | 07:12 | |
| bkero | prometheanfire: running your dibelements on non-gentoo, I error out with: gpg: can't open 'diskimage_builder/../bin/../elements/gentoo/gentoo-releng.gpg': No such file or directory | 07:13 |
| openstackgerrit | Ben Kero proposed openstack-infra/project-config: Move chef jobs on xenial https://review.openstack.org/347633 | 07:15 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use constraints for venv https://review.openstack.org/348759 | 07:16 |
| *** yanyanhu has quit IRC | 07:17 | |
| openstackgerrit | Ben Kero proposed openstack-infra/project-config: Move chef jobs on xenial https://review.openstack.org/347633 | 07:17 |
| *** gildub has quit IRC | 07:19 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/tripleo-ci: Add --yes flags to heat stack-delete for liberty https://review.openstack.org/347202 | 07:19 |
| *** davidlenwell has quit IRC | 07:22 | |
| flip214 | can somebody give me a hint about what to do with "E1101, Instance of ... has no ... member"? the code is working, the member functions get imported from another library... | 07:22 |
| flip214 | http://logs.openstack.org/70/347270/3/check/gate-cinder-pylint/8b0f2cc/console.html | 07:23 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use constraints for translation sync https://review.openstack.org/348764 | 07:24 |
| AJaeger | fungi, translation jobs are the only jobs that use venv and run on a long-lived node. Could you carefully review 348764 and 348759, please? | 07:25 |
| flip214 | AJaeger: http://logs.openstack.org/74/347274/3/check/gate-cinder-python34-db/3d50814/console.html seems truncated, too | 07:25 |
| *** PollyZ has joined #openstack-infra | 07:25 | |
| flip214 | and http://logs.openstack.org/74/347274/3/check/gate-cinder-python27-db-ubuntu-xenial/d30ce20/console.html | 07:25 |
| * AJaeger marks 348764 as WIP for now | 07:27 | |
| AJaeger | flip214: that needs an infra-root to check. I don't know whether there's a known problem... | 07:27 |
| flip214 | thank you for trying, anyway! | 07:28 |
| *** matrohon has quit IRC | 07:29 | |
| *** PollyZ has quit IRC | 07:30 | |
| *** tonytan4ever has joined #openstack-infra | 07:31 | |
| *** ccamacho is now known as ccamacho|afk | 07:33 | |
| openstackgerrit | Merged openstack-infra/project-config: Add gate-tripleo-quickstart-linters non-voting job https://review.openstack.org/348565 | 07:34 |
| *** davidlenwell has joined #openstack-infra | 07:34 | |
| *** jlanoux has joined #openstack-infra | 07:36 | |
| *** tonytan4ever has quit IRC | 07:36 | |
| *** mikelk has joined #openstack-infra | 07:37 | |
| *** kzaitsev_mb has quit IRC | 07:37 | |
| *** vincentll has joined #openstack-infra | 07:39 | |
| *** PollyZ has joined #openstack-infra | 07:39 | |
| *** shardy has joined #openstack-infra | 07:39 | |
| *** kzaitsev_mb has joined #openstack-infra | 07:40 | |
| *** florianf has joined #openstack-infra | 07:40 | |
| openstackgerrit | Merged openstack-infra/project-config: Append ubuntu-xenial to gate-neutron-python27 for Neutron Grafana https://review.openstack.org/348600 | 07:41 |
| openstackgerrit | Merged openstack-infra/project-config: Enable py35 jobs to vote for senlin(client) https://review.openstack.org/348606 | 07:41 |
| *** afred312_ has joined #openstack-infra | 07:42 | |
| *** PollyZ has quit IRC | 07:43 | |
| *** dkehn_ has quit IRC | 07:44 | |
| *** afred312 has quit IRC | 07:45 | |
| *** dimtruck is now known as zz_dimtruck | 07:45 | |
| *** dkehn has quit IRC | 07:45 | |
| *** kzaitsev_mb has quit IRC | 07:45 | |
| openstackgerrit | Merged openstack-infra/project-config: Add new project: nova-mksproxy https://review.openstack.org/346740 | 07:48 |
| openstackgerrit | Merged openstack-infra/project-config: Upload tarballs for tripleo-ui https://review.openstack.org/343834 | 07:49 |
| openstackgerrit | Merged openstack-infra/project-config: Make gate-rally-dsvm-networking-ovn voting https://review.openstack.org/348402 | 07:50 |
| *** kaisers has quit IRC | 07:50 | |
| *** ccamacho|afk is now known as ccamacho | 07:51 | |
| *** kaisers has joined #openstack-infra | 07:51 | |
| openstackgerrit | Merged openstack-infra/project-config: Add gate-rally-dsvm-networking-ovn to grafana https://review.openstack.org/348413 | 07:51 |
| openstackgerrit | Merged openstack-infra/project-config: Move gate-project-config-layout to ubuntu-trusty https://review.openstack.org/348420 | 07:51 |
| openstackgerrit | Merged openstack-infra/project-config: Migrate infra jobs to ubuntu-xenial https://review.openstack.org/348424 | 07:51 |
| *** gongysh has quit IRC | 07:53 | |
| *** jpich has joined #openstack-infra | 07:55 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Use constraints for translation sync https://review.openstack.org/348764 | 07:56 |
| *** dkehn has joined #openstack-infra | 07:56 | |
| *** dkehn_ has joined #openstack-infra | 07:58 | |
| *** zzzeek has quit IRC | 08:00 | |
| *** rossella_s has quit IRC | 08:00 | |
| *** kdas__ has joined #openstack-infra | 08:00 | |
| *** rossella_s has joined #openstack-infra | 08:00 | |
| *** zzzeek has joined #openstack-infra | 08:00 | |
| *** e0ne has quit IRC | 08:00 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Remove broken service enablement for ceilometer https://review.openstack.org/348747 | 08:01 |
| *** rbuzatu has quit IRC | 08:03 | |
| *** alee has joined #openstack-infra | 08:04 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/348774 | 08:06 |
| *** PollyZ has joined #openstack-infra | 08:06 | |
| *** Daisy has quit IRC | 08:07 | |
| *** derekh has joined #openstack-infra | 08:09 | |
| *** PollyZ has quit IRC | 08:11 | |
| flip214 | I see quite a lot of failures to gate-tempest-dsvm-postgres-full-ubuntu-xenial recently... did someone already see that too? | 08:13 |
| *** electrofelix has joined #openstack-infra | 08:13 | |
| flip214 | spurious, because it works sometimes and sometimes not (for the same code, only a "recheck" away) https://review.openstack.org/#/c/347267/3 | 08:14 |
| flip214 | http://ci-watch.tintri.com/project?project=cinder&time=7+days confirms a (lone) high failure rate. | 08:14 |
| flip214 | ah sorry, misread there. | 08:15 |
| openstackgerrit | sulochan-acharya proposed openstack-infra/project-config: Enable zuul jobs for craton and cratonclient https://review.openstack.org/345453 | 08:16 |
| *** _degorenko|afk is now known as degorenko | 08:19 | |
| *** PollyZ has joined #openstack-infra | 08:20 | |
| *** alee has quit IRC | 08:21 | |
| openstackgerrit | Ben Kero proposed openstack-infra/project-config: Move tarball jobs over to xenial https://review.openstack.org/348777 | 08:24 |
| openstackgerrit | Mehdi Abaakouk (sileht) proposed openstack-infra/project-config: telemetry: Don't enable panko on old branch https://review.openstack.org/344836 | 08:24 |
| *** PollyZ has quit IRC | 08:25 | |
| *** kdas__ has quit IRC | 08:26 | |
| derekh | AJaeger: would you be able to take a look at the nodepool logs please, its starting VM's on rh1 but then deleting them a minute later, I'm not sure why | 08:27 |
| *** tkelsey has joined #openstack-infra | 08:30 | |
| * bkero curious what "node: release" is in jenkins/jobs/* | 08:30 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/system-config: Switch the floating ip source to neutron for rh1 https://review.openstack.org/348778 | 08:30 |
| *** yamamoto_ has quit IRC | 08:30 | |
| derekh | AJaeger: maybe that the problem ^^, not sure but rh2 is using neutron as the floating IP source and rh1 has been deployed the same way | 08:32 |
| AJaeger | derekh: I'm not an admin, can't look at the logs and help here. | 08:33 |
| *** bhavik has joined #openstack-infra | 08:33 | |
| *** vincentll has quit IRC | 08:33 | |
| *** PollyZ has joined #openstack-infra | 08:33 | |
| derekh | AJaeger: ahh ok, thanks anyways | 08:33 |
| AJaeger | bkero: "release" is a long lived node that we run special jobs on and that have credentials for uploading | 08:34 |
| *** rbuzatu has joined #openstack-infra | 08:36 | |
| *** yamahata has joined #openstack-infra | 08:37 | |
| *** PollyZ has quit IRC | 08:38 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Stop running dstat twice https://review.openstack.org/347917 | 08:40 |
| *** r-mibu has quit IRC | 08:41 | |
| *** roxanaghe has joined #openstack-infra | 08:41 | |
| openstackgerrit | Merged openstack-infra/project-config: Normalize projects.yaml https://review.openstack.org/348774 | 08:43 |
| openstackgerrit | Ben Kero proposed openstack-infra/project-config: Move specs jobs to xenial https://review.openstack.org/348787 | 08:43 |
| *** daemontool has joined #openstack-infra | 08:44 | |
| openstackgerrit | Ben Kero proposed openstack-infra/project-config: Move gate-{name}-chef-rake to ubuntu-xenial https://review.openstack.org/347633 | 08:44 |
| *** roxanaghe has quit IRC | 08:46 | |
| openstackgerrit | Ben Kero proposed openstack-infra/project-config: Move {name}-publish-specs job + others to ubuntu-xenial https://review.openstack.org/348787 | 08:46 |
| openstackgerrit | Ben Kero proposed openstack-infra/project-config: Move {name}-tarball jobs over to xenial https://review.openstack.org/348777 | 08:46 |
| bkero | AJaeger: Thanks for the info! | 08:47 |
| bkero | and sorry about the commit spam | 08:47 |
| *** PollyZ has joined #openstack-infra | 08:47 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Use delorean's exit code to decide on retry https://review.openstack.org/348790 | 08:47 |
| *** ramishra has quit IRC | 08:48 | |
| *** gongysh has joined #openstack-infra | 08:50 | |
| rcarrillocruz | fungi , clarkb , nibalizer : the Ansible designate modules to manage zones and recordsets have merged on ansible extras modules repo | 08:50 |
| rcarrillocruz | should there be an interest at some point to manage our DNS with Ansible | 08:50 |
| *** yanyanhu has joined #openstack-infra | 08:51 | |
| *** r-mibu has joined #openstack-infra | 08:51 | |
| *** PollyZ has quit IRC | 08:52 | |
| *** e0ne has joined #openstack-infra | 08:52 | |
| *** abregman has joined #openstack-infra | 08:52 | |
| *** BobBall_AWOL is now known as BobBall | 08:53 | |
| *** mhickey has joined #openstack-infra | 08:56 | |
| *** amotoki has quit IRC | 08:56 | |
| azvyagintsev | mmedvede ping | 08:59 |
| *** amotoki has joined #openstack-infra | 09:00 | |
| *** PollyZ has joined #openstack-infra | 09:00 | |
| azvyagintsev | mmedvede: I found, that i can simple use tox builder after devstack run...but, i catch issue in devstack installation: | 09:03 |
| azvyagintsev | http://paste.openstack.org/show/543855/ sinse i pass variables: http://paste.openstack.org/show/543856/ | 09:03 |
| azvyagintsev | for fetch murano-apps from review. And for some reason devstack.sh trye's to apply them to any repo-and for sure fail... | 09:03 |
| azvyagintsev | (my idea: fetch murno-apps git with review, install devstack, and tun some tox against it) | 09:03 |
| openstackgerrit | lokesh s proposed openstack-infra/project-config: Add py34 and py35 jobs for monasca-api and agent project https://review.openstack.org/348796 | 09:05 |
| *** PollyZ has quit IRC | 09:05 | |
| *** e0ne has quit IRC | 09:05 | |
| *** matrohon has joined #openstack-infra | 09:07 | |
| openstackgerrit | Merged openstack-infra/system-config: Add ntp-perl package for RedHat platforms https://review.openstack.org/348601 | 09:10 |
| Zara | hi! spotted this story about someone unable to submit contact info to gerrit; author is keen for help: https://storyboard.openstack.org/#!/story/2000687 ; sharing it here in case anyone can help and isn't subscribed to all the storyboard things like I am. :) | 09:13 |
| AJaeger | Zara, I'll add my usual text to it ;) | 09:14 |
| Zara | hehe, thank you. :) | 09:15 |
| *** yolanda has quit IRC | 09:19 | |
| *** yanyanhu has quit IRC | 09:20 | |
| *** yolanda has joined #openstack-infra | 09:20 | |
| Zara | (I'm tempted to make a script for it; seems to come up a lot, but it's probably surprisingly awkward to automate) | 09:20 |
| *** ihrachys has joined #openstack-infra | 09:21 | |
| *** ociuhandu has quit IRC | 09:25 | |
| *** PollyZ has joined #openstack-infra | 09:28 | |
| openstackgerrit | Luigi Toscano proposed openstack-infra/project-config: Keep localrc in CLI Sahara job https://review.openstack.org/348800 | 09:28 |
| *** chem|away is now known as chem | 09:31 | |
| *** PollyZ has quit IRC | 09:32 | |
| *** permalac has joined #openstack-infra | 09:33 | |
| *** john-davidge has joined #openstack-infra | 09:34 | |
| *** psachin has quit IRC | 09:35 | |
| *** john-dav_ has joined #openstack-infra | 09:35 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Remve old cache files on the mirror server https://review.openstack.org/348806 | 09:37 |
| *** john-davidge has quit IRC | 09:38 | |
| *** PollyZ has joined #openstack-infra | 09:41 | |
| AJaeger | infra-root, looking at the post queue with now 330+ changes: The missing jobs all need ubuntu-xenial nodes, the trusty ones have been run. Do we have some stale xenial nodes out there so that demand is miscalculated or do we need to raise min-ready for it? | 09:46 |
| *** PollyZ has quit IRC | 09:46 | |
| *** abregman has quit IRC | 09:50 | |
| *** tlbr has quit IRC | 09:52 | |
| *** tlbr has joined #openstack-infra | 09:53 | |
| *** tlbr has quit IRC | 09:53 | |
| *** tlbr has joined #openstack-infra | 09:54 | |
| *** tlbr has quit IRC | 09:54 | |
| *** PollyZ has joined #openstack-infra | 09:55 | |
| openstackgerrit | Kevin Benton proposed openstack-infra/elastic-recheck: Add query for Nova bug 1607714 https://review.openstack.org/348812 | 09:55 |
| openstack | bug 1607714 in OpenStack Compute (nova) "failure attaching interface" [Undecided,New] https://launchpad.net/bugs/1607714 | 09:55 |
| *** tlbr has joined #openstack-infra | 09:56 | |
| *** tlbr is now known as Guest45466 | 09:56 | |
| *** ociuhandu has joined #openstack-infra | 09:59 | |
| *** PollyZ has quit IRC | 10:00 | |
| *** Guest58208 has quit IRC | 10:00 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Move doc team jobs to Xenial https://review.openstack.org/348813 | 10:00 |
| *** Guest45466 has quit IRC | 10:00 | |
| *** ramishra has joined #openstack-infra | 10:00 | |
| *** e0ne has joined #openstack-infra | 10:01 | |
| *** zhurong has quit IRC | 10:03 | |
| *** Liuqing has quit IRC | 10:03 | |
| *** psachin has joined #openstack-infra | 10:08 | |
| *** Qiming has quit IRC | 10:08 | |
| *** yaume has joined #openstack-infra | 10:09 | |
| *** e0ne has quit IRC | 10:12 | |
| *** tlbr_ has joined #openstack-infra | 10:13 | |
| *** Qiming has joined #openstack-infra | 10:13 | |
| *** yamahata has quit IRC | 10:16 | |
| *** jordanP has joined #openstack-infra | 10:16 | |
| *** gildub has joined #openstack-infra | 10:16 | |
| *** tlbr_ has quit IRC | 10:17 | |
| *** tosky has joined #openstack-infra | 10:18 | |
| *** gongysh has quit IRC | 10:18 | |
| *** sdague has joined #openstack-infra | 10:19 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Refactor mirror logic https://review.openstack.org/326143 | 10:22 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Record testenv wait times in ovb jobs https://review.openstack.org/347037 | 10:23 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Complex priorities UI in stories https://review.openstack.org/312666 | 10:23 |
| *** Qiming has quit IRC | 10:25 | |
| *** sdake has joined #openstack-infra | 10:25 | |
| *** Qiming has joined #openstack-infra | 10:27 | |
| *** roxanaghe has joined #openstack-infra | 10:30 | |
| *** flepied1 has joined #openstack-infra | 10:30 | |
| *** flepied has quit IRC | 10:31 | |
| sdague | anyone know the state of the mirrors? | 10:32 |
| *** roxanaghe has quit IRC | 10:34 | |
| *** claudiub|2 has joined #openstack-infra | 10:35 | |
| *** vincentll has joined #openstack-infra | 10:36 | |
| *** john-dav_ is now known as john-davidge | 10:37 | |
| *** yaume has quit IRC | 10:39 | |
| *** yaume has joined #openstack-infra | 10:39 | |
| yolanda | AJaeger, i can take a look at nodepool | 10:41 |
| yolanda | no stale nodes, there at not ready nodes actually | 10:43 |
| AJaeger | yolanda: no ready nodes at all? So, just too high demand? | 10:43 |
| yolanda | AJaeger, looks like, all xenial nodes are on used or building | 10:44 |
| AJaeger | that's sad - thanks for checking. Seems there's nothing we can do ;( | 10:45 |
| *** Qiming has quit IRC | 10:46 | |
| yolanda | increase min-ready, as soon as more nodes are moved to xenial, would help | 10:46 |
| tosky | probably a left-over of the outage of few days ago? It looks like only the weekend could empty the long queue of post-merge check :) | 10:47 |
| tosky | in the meantime, if someone would like to merge this easy change in a Sahara job (which fixes a regression that I introduced, grrr), that would make me happy :) https://review.openstack.org/#/c/348800/ | 10:47 |
| AJaeger | tosky: we recovered from that one - but load is high and we lost a few nodes from our clouds, so have lower capacity than normal. | 10:48 |
| tosky | ah, I see - luckily, Friday | 10:49 |
| *** PollyZ has joined #openstack-infra | 10:49 | |
| AJaeger | tosky: have a lucky Friday - +2 ;) | 10:49 |
| tosky | thanks :) | 10:49 |
| openstackgerrit | garyk proposed openstack-infra/project-config: Networking-l2gw: enable python 3 voting https://review.openstack.org/348123 | 10:51 |
| *** Qiming has joined #openstack-infra | 10:52 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Move doc team jobs to Xenial https://review.openstack.org/348813 | 10:54 |
| *** PollyZ has quit IRC | 10:54 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient: Complex priorities UI in stories https://review.openstack.org/312666 | 10:56 |
| *** Qiming has quit IRC | 10:57 | |
| *** e0ne has joined #openstack-infra | 10:58 | |
| sdague | yolanda: any idea what the progress on the mirror rebuild is? | 10:58 |
| *** e0ne has quit IRC | 11:01 | |
| yolanda | sdague, i'm not aware of it sorry | 11:01 |
| sdague | ok | 11:01 |
| *** Qiming has joined #openstack-infra | 11:01 | |
| AJaeger | sdague: clarkb's last message in this channel was 11hours ago "bandersnatch is up to pydirix" | 11:01 |
| yolanda | i can check logs | 11:02 |
| sdague | yolanda: yeh, that would be cool. I think there also needs to be an AFS release to make it live | 11:02 |
| sdague | upper-constraints jobs still failing, and people still rechecking them | 11:02 |
| *** psachin has quit IRC | 11:04 | |
| AJaeger | sdague: speaking about constraints: https://review.openstack.org/348759 and https://review.openstack.org/348764 are the missing pieces to use constraints for venv. And I'm still testing releasenotes... | 11:04 |
| *** psachin has joined #openstack-infra | 11:05 | |
| yolanda | i don't see any bandersnatch process running on mirror update right now | 11:06 |
| AJaeger | yolanda: clarkb used a screen AFAIR. so, if there's none running, it might have finished ;) | 11:06 |
| flip214 | another question... https://review.openstack.org/#/c/347270/3 | 11:07 |
| flip214 | I asked for a recheck, but the new links to the results go to the old files | 11:07 |
| flip214 | eg. gate-cinder-pylint in jenkins 12:15 point to a console.log 2016-07-29 08:30 | 11:08 |
| flip214 | hmmm, seems that *no* checks have been re-run... all that I looked at have timestamps from 8 o' clock | 11:09 |
| *** azvyagintsev has quit IRC | 11:09 | |
| *** vsaienko has quit IRC | 11:09 | |
| yolanda | AJaeger, from that point, i'm not very experienced on how to check the afs status | 11:10 |
| *** lucasagomes is now known as lucas-hungry | 11:14 | |
| *** bhavik has quit IRC | 11:16 | |
| *** Na3iL has quit IRC | 11:17 | |
| *** rodrigods has quit IRC | 11:19 | |
| *** rodrigods has joined #openstack-infra | 11:19 | |
| *** asettle has joined #openstack-infra | 11:20 | |
| *** EricGonczer_ has joined #openstack-infra | 11:20 | |
| *** acruz has joined #openstack-infra | 11:21 | |
| *** azvyagintsev has joined #openstack-infra | 11:21 | |
| sulo | yolanda: if you have a min, would appreciate a quick looksee on this: https://review.openstack.org/#/c/345453/ :) | 11:22 |
| *** flepied1 has quit IRC | 11:23 | |
| *** vsaienko has joined #openstack-infra | 11:24 | |
| *** sdake has quit IRC | 11:27 | |
| *** PollyZ has joined #openstack-infra | 11:30 | |
| Zara | AJaeger: presenting... AJaegerbot! http://paste.openstack.org/show/543877/ :P | 11:30 |
| openstackgerrit | Miles Gould proposed openstack/diskimage-builder: Allow installing proliant tools from an OS package https://review.openstack.org/344214 | 11:30 |
| *** flepied has joined #openstack-infra | 11:30 | |
| *** flepied1 has joined #openstack-infra | 11:31 | |
| *** tonytan4ever has joined #openstack-infra | 11:32 | |
| *** abregman has joined #openstack-infra | 11:32 | |
| *** jkilpatr has joined #openstack-infra | 11:32 | |
| openstackgerrit | Merged openstack-infra/puppet-hound: Make the git source uri configurable https://review.openstack.org/348593 | 11:34 |
| *** PollyZ has quit IRC | 11:35 | |
| *** flepied has quit IRC | 11:35 | |
| Srinu | hi | 11:35 |
| *** Guest5363 has joined #openstack-infra | 11:36 | |
| *** Guest5363 has quit IRC | 11:36 | |
| openstackgerrit | Miles Gould proposed openstack/diskimage-builder: Allow installing proliant tools from an OS package https://review.openstack.org/344214 | 11:36 |
| openstackgerrit | Merged openstack-infra/puppet-germqtt: Ensure ssh keys are all setup https://review.openstack.org/347574 | 11:37 |
| Srinu | i am using two jobs in check pipeline. when one job finishing first on one slave and the finished slave is deleted by the nodepool. at the time of deletion jenkins (java process) in another is killed. how can i stop that. any plugin for that | 11:37 |
| Srinu | did any one face this issue | 11:37 |
| Srinu | please help me | 11:37 |
| *** tonytan4ever has quit IRC | 11:37 | |
| openstackgerrit | Merged openstack-infra/puppet-germqtt: Clean up stray whitespace https://review.openstack.org/347892 | 11:38 |
| *** itisha has quit IRC | 11:40 | |
| *** thorst has joined #openstack-infra | 11:40 | |
| openstackgerrit | Merged openstack-infra/puppet-germqtt: Add auth config to germqtt https://review.openstack.org/347977 | 11:40 |
| openstackgerrit | Merged openstack-infra/puppet-mosquitto: Remove unused config.pp manifest https://review.openstack.org/347922 | 11:40 |
| openstackgerrit | Merged openstack-infra/project-config: Enable zuul jobs for craton and cratonclient https://review.openstack.org/345453 | 11:41 |
| openstackgerrit | Miles Gould proposed openstack/diskimage-builder: Allow installing proliant tools from an OS package https://review.openstack.org/344214 | 11:41 |
| *** rhallisey has joined #openstack-infra | 11:42 | |
| *** PollyZ has joined #openstack-infra | 11:43 | |
| openstackgerrit | Merged openstack-infra/system-config: Add jessie-newton-backports repo https://review.openstack.org/347839 | 11:44 |
| *** acruz has quit IRC | 11:45 | |
| *** PollyZ has quit IRC | 11:48 | |
| *** e0ne has joined #openstack-infra | 11:51 | |
| openstackgerrit | Richard Theis proposed openstack-infra/project-config: Make gate-networking-ovn-dsvm-functional voting https://review.openstack.org/346978 | 11:51 |
| *** rbuzatu has quit IRC | 11:52 | |
| *** pradk has joined #openstack-infra | 11:55 | |
| *** ldnunes has joined #openstack-infra | 11:55 | |
| *** rfolco has joined #openstack-infra | 11:56 | |
| *** markusry has joined #openstack-infra | 11:59 | |
| *** rossella_s has quit IRC | 11:59 | |
| *** rossella_s has joined #openstack-infra | 12:00 | |
| *** rbuzatu has joined #openstack-infra | 12:00 | |
| *** weshay is now known as weshay_mtg | 12:01 | |
| *** Hal has joined #openstack-infra | 12:02 | |
| *** gordc has joined #openstack-infra | 12:02 | |
| *** Hal is now known as Guest38666 | 12:02 | |
| *** tlbr_ has joined #openstack-infra | 12:05 | |
| honza | how soon after a patch has been merged can i expect to see my new jobs running? | 12:05 |
| *** hwcomcn has joined #openstack-infra | 12:05 | |
| *** rbuzatu has quit IRC | 12:05 | |
| *** hwcomcn has quit IRC | 12:07 | |
| *** hwcomcn has joined #openstack-infra | 12:07 | |
| *** hwcomcn has quit IRC | 12:09 | |
| *** tlbr_ has quit IRC | 12:09 | |
| odyssey4me | honza it's eventually consistent over the course of an hour or so usually | 12:09 |
| honza | odyssey4me: cool, thanks | 12:10 |
| *** hwcomcn has joined #openstack-infra | 12:10 | |
| *** PollyZ has joined #openstack-infra | 12:11 | |
| *** aysyd has joined #openstack-infra | 12:11 | |
| *** hwcomcn has quit IRC | 12:11 | |
| *** hwcomcn has joined #openstack-infra | 12:12 | |
| *** hwcomcn has quit IRC | 12:13 | |
| flip214 | can somebody tell me why a "recheck" comment started jenkins, but no new logfiles arrived? | 12:13 |
| *** trown|outtypewww is now known as trown | 12:13 | |
| flip214 | https://review.openstack.org/#/c/347267/3 | 12:13 |
| *** hwcomcn has joined #openstack-infra | 12:14 | |
| flip214 | jenkins got "recheck" at 10:17, last run still points to http://logs.openstack.org/67/347267/3/check/gate-cinder-python27-db-ubuntu-xenial/3f28875/ being 08:44 | 12:14 |
| *** PollyZ has quit IRC | 12:16 | |
| *** itisha has joined #openstack-infra | 12:16 | |
| flip214 | ah, seems a TZ issue between review.os.org and logs.os.org | 12:16 |
| *** Liuqing has joined #openstack-infra | 12:20 | |
| *** lucas-hungry is now known as lucasagomes | 12:20 | |
| AJaeger | Zara: RTFL! | 12:20 |
| Zara | :D | 12:20 |
| AJaeger | Zara, I stole some lines from one of fungi's explanations. He deserves credit as first author ;) | 12:22 |
| *** PollyZ has joined #openstack-infra | 12:24 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Move doc team jobs to Xenial https://review.openstack.org/348813 | 12:24 |
| fungi | flip214: i think gerrit attempts to convert its ui to your local timezone? i tend not to notice since my "local" timezone is always set to utc | 12:25 |
| *** Na3iL has joined #openstack-infra | 12:25 | |
| Zara | AJaeger: hahaha, well now it can be called funjaegerbot! | 12:25 |
| AJaeger | Seems it's Friday ;) | 12:25 |
| fungi | flip214: whereas our logs are all always presented in utc | 12:25 |
| *** yaume has quit IRC | 12:25 | |
| flip214 | fungi: possibly, yeah. | 12:26 |
| flip214 | sadly the list of failing tests changes all the time, although I'm using "recheck" instead of "git commit" ;{ | 12:26 |
| *** mhu has quit IRC | 12:27 | |
| *** _ari_ has joined #openstack-infra | 12:27 | |
| *** _ari_ has quit IRC | 12:27 | |
| *** ccamacho is now known as ccamacho|lunch | 12:28 | |
| *** csomerville has joined #openstack-infra | 12:29 | |
| *** PollyZ has quit IRC | 12:29 | |
| *** haaj has joined #openstack-infra | 12:30 | |
| *** weshay_mtg is now known as weshay | 12:31 | |
| *** cody-somerville has quit IRC | 12:32 | |
| *** mhu has joined #openstack-infra | 12:34 | |
| fungi | yolanda: AJaeger: i logged into mirror-update.o.o and attached to clarkb's screen session there. it looks like run-bandersnatch crashed out with a nondescript exception/traceback at some point overnight | 12:35 |
| *** Guest41883 has joined #openstack-infra | 12:35 | |
| fungi | sdague: ^ | 12:35 |
| AJaeger | that's sad ;/ | 12:35 |
| fungi | i'll attempt to restart it and it should hopefully pick up where it left off | 12:36 |
| AJaeger | Let's hope ;) | 12:36 |
| AJaeger | thanks fungi! | 12:36 |
| sdague | fungi: yeh, lets hope, it was a full rebuild | 12:36 |
| *** psilvad has joined #openstack-infra | 12:36 | |
| *** xyang1 has joined #openstack-infra | 12:36 | |
| sdague | we're 30+ hours into not being able to update any upper constraints | 12:37 |
| sdague | which is definitely putting a number of release features at risk :( | 12:37 |
| *** PollyZ has joined #openstack-infra | 12:38 | |
| *** thiagop has joined #openstack-infra | 12:38 | |
| fungi | the traceback is http://paste.openstack.org/show/543895 | 12:38 |
| openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: Add 'ceph status' and tripleo-overcloud-passwords to host_info https://review.openstack.org/348862 | 12:38 |
| fungi | sdague: yep--good thing not everybody waited until the last minute! | 12:39 |
| sdague | fungi: some times development takes time | 12:39 |
| *** Guest41883 has quit IRC | 12:40 | |
| *** tpsilva has joined #openstack-infra | 12:42 | |
| *** PollyZ has quit IRC | 12:43 | |
| *** bhavik has joined #openstack-infra | 12:44 | |
| *** rlandy has joined #openstack-infra | 12:44 | |
| fungi | it looks like the run-bandersnatch wrapper was getting confused by something in the subprocess output, but not sure what. i've switched to running `bandersnatch mirror` under kstart instead and it seems to be progressing | 12:45 |
| fungi | looks like it's probably unwinding recent pypi serials so it's probably almost there | 12:46 |
| *** tlbr_ has joined #openstack-infra | 12:48 | |
| fungi | yeah, i the versions of packages i've spot-checked were uploads to pypi yesterday | 12:48 |
| Srinu | fungi: hi.i am using two jobs in check pipeline. when one job finishing first on one slave and the finished slave is deleted by the nodepool. at the time of deletion jenkins (java process) in another is killed. how can i stop that. any plugin for that | 12:49 |
| Srinu | fungi: did you face this issue anytime | 12:50 |
| Srinu | fungi: how to separate the two slave process on the same provider | 12:50 |
| fungi | Srinu: i have no idea. we don't use jenkins now, and i never saw behavior like that back when we did | 12:50 |
| Srinu | fungi: ok thank you | 12:50 |
| *** rbuzatu has joined #openstack-infra | 12:51 | |
| *** tlbr_ has quit IRC | 12:52 | |
| *** edmondsw has joined #openstack-infra | 12:54 | |
| *** nwkarsten has joined #openstack-infra | 12:54 | |
| *** matt-borland has joined #openstack-infra | 12:55 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: Move doc team jobs to Xenial https://review.openstack.org/348813 | 12:56 |
| *** jcoufal has joined #openstack-infra | 12:58 | |
| *** psachin has quit IRC | 12:59 | |
| *** pradk has quit IRC | 13:02 | |
| openstackgerrit | Vladyslav Drok proposed openstack-infra/project-config: Make ironicclient functional job voting https://review.openstack.org/345521 | 13:02 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Update FindBugs docs to 4 space indent syntax https://review.openstack.org/348740 | 13:03 |
| *** tonytan4ever has joined #openstack-infra | 13:04 | |
| *** links has quit IRC | 13:07 | |
| fungi | oh, actually these serials aren't in order. looks like it's still running through package resync it simply started with some names of packages which had been uploaded since after it crashed but which came earlier in alpha sort order than where it had left off | 13:07 |
| fungi | but it's back in the py.* package names now | 13:07 |
| *** tonytan4ever has quit IRC | 13:08 | |
| fungi | hrm, now it's already into names starting with r | 13:08 |
| *** ccamacho|lunch is now known as ccamacho | 13:08 | |
| fungi | now s. going way too fast to have died in the py.* names i guess | 13:09 |
| openstackgerrit | Craige McWhirter proposed openstack-infra/puppet-phabricator: Patches Required to Deliver Pholio https://review.openstack.org/342481 | 13:09 |
| fungi | so these may still all be more recent uploads after all | 13:09 |
| *** ianw has quit IRC | 13:10 | |
| *** uberjay has quit IRC | 13:10 | |
| *** ianw has joined #openstack-infra | 13:10 | |
| *** mdrabe has joined #openstack-infra | 13:11 | |
| *** uberjay has joined #openstack-infra | 13:12 | |
| openstackgerrit | afazekas proposed openstack/os-testr: Construct a list of test cases instead of passing a regexp https://review.openstack.org/348878 | 13:12 |
| openstackgerrit | Merged openstack-infra/jenkins-job-builder: Update Cobertura docs to be 4 space indent syntax https://review.openstack.org/348736 | 13:13 |
| *** jsavak has joined #openstack-infra | 13:14 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/project-config: Switch tripleo jobs back to rh1 https://review.openstack.org/348879 | 13:15 |
| openstackgerrit | Derek Higgins proposed openstack-infra/project-config: Restore the full compliment of tripleo ci jobs https://review.openstack.org/348880 | 13:15 |
| *** gouthamr has joined #openstack-infra | 13:18 | |
| *** PollyZ has joined #openstack-infra | 13:19 | |
| *** haaj has quit IRC | 13:19 | |
| derekh | would anybody be able to take a look at the nodepool logs please, its starting VM's on rh1 but then deleting them a minute later, I'm not sure why | 13:19 |
| derekh | but I suspect it may be because of this https://review.openstack.org/#/c/348778/ | 13:19 |
| *** tlbr has joined #openstack-infra | 13:19 | |
| *** rgerganov has joined #openstack-infra | 13:20 | |
| *** tlbr is now known as Guest28658 | 13:20 | |
| rgerganov | Hi, I recently requested a new project which was approved and created today: https://github.com/openstack/nova-mksproxy | 13:21 |
| rgerganov | I'd like to be added to nova-mksproxy-core and nova-mksproxy-release gerrit groups | 13:22 |
| *** kgiusti has joined #openstack-infra | 13:22 | |
| rgerganov | who should I talk to? | 13:22 |
| AJaeger | rgerganov: could you link to the change that created those, please? | 13:22 |
| rgerganov | AJaeger, https://review.openstack.org/#/c/346740/ | 13:22 |
| *** Vivek has quit IRC | 13:22 | |
| AJaeger | rgerganov: thanks - somebody will pick it up - and if not, please ask again or send an email to openstack-infra mailing list. You can start sending patches in already ;) | 13:23 |
| rgerganov | AJaeger, ok, thanks :) | 13:23 |
| *** PollyZ has quit IRC | 13:23 | |
| *** Guest28658 has quit IRC | 13:24 | |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Source rh1.env to decide how many testenv workers to run https://review.openstack.org/348883 | 13:25 |
| openstackgerrit | Derek Higgins proposed openstack-infra/tripleo-ci: Switch back to the net iso HA job https://review.openstack.org/348884 | 13:25 |
| *** tldr__ has joined #openstack-infra | 13:26 | |
| *** timl0101 has joined #openstack-infra | 13:27 | |
| *** kushal has joined #openstack-infra | 13:30 | |
| *** PollyZ has joined #openstack-infra | 13:32 | |
| *** andymaier has quit IRC | 13:32 | |
| *** sarob has joined #openstack-infra | 13:32 | |
| *** deadnull has joined #openstack-infra | 13:33 | |
| *** sarob has quit IRC | 13:33 | |
| *** nwkarsten has quit IRC | 13:33 | |
| *** nwkarsten has joined #openstack-infra | 13:34 | |
| *** mriedem has joined #openstack-infra | 13:34 | |
| *** ddieterly has joined #openstack-infra | 13:35 | |
| *** hongbin has left #openstack-infra | 13:36 | |
| fungi | sayeth bandersnatch... 2016-07-29 13:35:50,259 INFO: Generating global index page. | 13:36 |
| fungi | will probably still need one or two (very quick) cleanup passes once this completes | 13:36 |
| AJaeger | woot! | 13:36 |
| *** PollyZ has quit IRC | 13:37 | |
| fungi | first catch-up pass is already to the index generation stage | 13:38 |
| fungi | and that's done | 13:38 |
| *** nwkarsten has quit IRC | 13:38 | |
| fungi | and there's the second catch-up pass done. gonna run it once more, should hopefully transfer nothing this next time around | 13:39 |
| fungi | 2016-07-29 13:39:10,341 INFO: 0 packages to sync. | 13:39 |
| sileht | sdague, Hi can you take a look to https://review.openstack.org/#/c/335479/ ? | 13:39 |
| fungi | releasing the bandersnatch lock now and the cronjob should update afs | 13:39 |
| tldr__ | mordred, hi, could you please review https://review.openstack.org/#/c/346095/ ? We need +1 from PTL | 13:41 |
| *** xarses has quit IRC | 13:44 | |
| fungi | cron is running bandersnatch now | 13:46 |
| *** nwkarsten has joined #openstack-infra | 13:46 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: grafana: Follow rename of gate-tempest-dsvm-neutron-full https://review.openstack.org/348890 | 13:46 |
| *** bauzas is now known as bauwser | 13:49 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: grafana: Follow xenial renames https://review.openstack.org/348890 | 13:49 |
| *** dansmith is now known as superdan | 13:49 | |
| *** rbrndt has joined #openstack-infra | 13:50 | |
| fungi | okay, for some reason bandersnatch mirror is still exiting 1 instead of 0 | 13:50 |
| *** akshai has joined #openstack-infra | 13:50 | |
| fungi | which is causing the wrapper script to think it shouldn't vos release | 13:51 |
| rcarrillocruz | rgerganov: what's your gerrit email address? | 13:51 |
| rgerganov | rcarrillocruz, it's rgerganov@vmware.com | 13:51 |
| rcarrillocruz | hmm, ok, autocomplete on gerrit ui didn't pick it up at first | 13:51 |
| rcarrillocruz | rgerganov: done | 13:52 |
| rgerganov | rcarrillocruz, thanks! | 13:52 |
| rcarrillocruz | np | 13:52 |
| fungi | rcarrillocruz: i usually copy the e-mail address from the committer field in the gerrit change, since gerrit enforces that it's a valid e-mail address for the gerrit account which pushes it | 13:52 |
| rcarrillocruz | good call | 13:53 |
| rcarrillocruz | fungi: should I bump the xenial min-ready? | 13:53 |
| *** bhavik has quit IRC | 13:53 | |
| *** gildub has quit IRC | 13:53 | |
| fungi | rcarrillocruz: i probably missed earlier discussion around that. what's the current misbehavior? | 13:54 |
| rcarrillocruz | i'm seeing more changes to move from trusty to xenial, and post is choking a bit... | 13:54 |
| rcarrillocruz | it's the queue size really | 13:54 |
| rcarrillocruz | with more switches to xenial i guess we should lower trusty and bump xenial? | 13:54 |
| *** pgadiya has quit IRC | 13:54 | |
| fungi | right now i'm trying to figure out why `bandersnatch mirror` seems to complete successfully, but then at the end returns "<ConfigParser.ConfigParser instance at 0x7f4ca9c9fbd8>" and exits 1 | 13:55 |
| fungi | rcarrillocruz: do we have a bunch of trusty ready nodes not running jobs? | 13:55 |
| rcarrillocruz | let me see | 13:55 |
| *** jianghuaw has quit IRC | 13:55 | |
| fungi | normally min-ready only comes into play if we're not running jobs constantly | 13:56 |
| *** jianghuaw has joined #openstack-infra | 13:56 | |
| *** edmondsw has quit IRC | 13:56 | |
| *** Ravikiran_K has quit IRC | 13:56 | |
| fungi | when we're at capacity (like now) nodepool should be basing its proportion of node types built on the demand it sees for them in zuul | 13:56 |
| rcarrillocruz | yeah, not really, all used and or bulding | 13:56 |
| *** mmandel has joined #openstack-infra | 13:57 | |
| rcarrillocruz | sigh | 13:57 |
| rcarrillocruz | anyway, brb, have a call for the next hour | 13:57 |
| phschwartz | So a warning to all. If you blow away your gertty db and local clones, don't sync nova with anything else. Do it completely on its own. lol | 13:57 |
| *** amotoki has quit IRC | 13:58 | |
| fungi | right now there's an outstanding backlog of jobs in zuul and we're maxxed out on quota, so min-ready is probably having no perceivable impact | 13:58 |
| zaro | morning | 13:58 |
| *** thiagolib has quit IRC | 13:58 | |
| fungi | curious if anybody's looked deeper yet into the fact that zuul-cloner doesn't seem to be pulling updates from the specified origin url. i should probably focus on that once bandersnatch is figured out | 13:59 |
| fungi | sigh... https://bitbucket.org/pypa/bandersnatch/issues/67/always-exit-with-code-1 | 14:00 |
| *** pradk has joined #openstack-infra | 14:00 | |
| rcarrillocruz | fungi: for the future, how you usually check for quota being maxed out? you grep nodepool nodes to sum of max capacity or we have some fancy grafana grafh for that? | 14:00 |
| rcarrillocruz | graph too | 14:00 |
| fungi | rcarrillocruz: i was just eyeballing the test nodes graph on the zuul status page | 14:01 |
| *** claudiub|2 has quit IRC | 14:01 | |
| fungi | and also the job queue graph | 14:01 |
| rcarrillocruz | erm, k, zuul page gets frozen on me quite often on chromium | 14:02 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement a new job for undercloud testing only https://review.openstack.org/346147 | 14:02 |
| *** rgerganov has quit IRC | 14:02 | |
| *** hashar is now known as hasharAway | 14:03 | |
| rcarrillocruz | ah right | 14:03 |
| rcarrillocruz | http://status.openstack.org/zuul/ | 14:03 |
| rcarrillocruz | that's upstream for gozer's gozer.hpcloud.net | 14:04 |
| rcarrillocruz | gotcha | 14:04 |
| rcarrillocruz | also, infracloud has been cabled (from last night update) | 14:04 |
| rcarrillocruz | they need to reconfigure the ip range a bit tho for the public | 14:04 |
| rcarrillocruz | but we'll keep /19 | 14:04 |
| rcarrillocruz | i'm hoping ot get hold of the net engineer assigned tonight my time | 14:05 |
| fungi | heh. so their assertion that there wouldn't be any ip address changes... completely wrong? | 14:05 |
| fungi | i mean, we assumed they were probably being a bit optimistic about that, so fine | 14:05 |
| *** roxanaghe has joined #openstack-infra | 14:05 | |
| bswartz | it looks like paste.openstack.org is offline -- is there a known outage? | 14:06 |
| prometheanfire | bkero: that's odd, that's before it even gets to any changes | 14:06 |
| *** akshai has quit IRC | 14:06 | |
| bswartz | oh now it's working again! | 14:07 |
| bswartz | that was weird | 14:07 |
| *** akshai has joined #openstack-infra | 14:07 | |
| fungi | bswartz: it gets cranky if its mysql session to trove times out for inactivity | 14:07 |
| prometheanfire | bkero: I always have to run it from the root dir though (that which contains dib-utils diskimage-builder and project-config) | 14:07 |
| *** yaume has joined #openstack-infra | 14:07 | |
| vgridnev | is it possible to see bugsquashing stats for longer periods? I mean, http://status.openstack.org/bugday/ | 14:07 |
| vgridnev | Just wonder | 14:07 |
| fungi | bswartz: so if nobody's hit it in a while, it can take a few seconds to "wake up" | 14:07 |
| fungi | and stop returning 5xx errors | 14:08 |
| prometheanfire | bkero: it also passes gate tests (experimental iirc) | 14:08 |
| prometheanfire | in dib | 14:08 |
| *** cloudtrainme has joined #openstack-infra | 14:09 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck: Add query for Nova bug 1607714 https://review.openstack.org/348812 | 14:09 |
| openstack | bug 1607714 in OpenStack Compute (nova) "interface attach tests failing with tap device is busy errors in neutron xenial jobs since 7/28" [High,Confirmed] https://launchpad.net/bugs/1607714 | 14:09 |
| *** roxanaghe has quit IRC | 14:10 | |
| fungi | vgridnev: not sure, you'd probably need to skim (and maybe patch) http://git.openstack.org/cgit/openstack-infra/bugdaystats/tree/ to see | 14:10 |
| mriedem | did the gate-tempest-dsvm-neutron-full-ubuntu-xenial job start yesterday? | 14:11 |
| fungi | mriedem: that's possible. you might look at git log -p in openstack-infra/project-config and search for gate-tempest-dsvm-neutron-full-ubuntu-xenial or some substring | 14:11 |
| mriedem | https://review.openstack.org/#/c/348078/ | 14:12 |
| *** Vivek has joined #openstack-infra | 14:12 | |
| *** hwcomcn has quit IRC | 14:16 | |
| fungi | okay, so digging into the source code, bandersnatch's entrypoint executable does sys.exit(main()) | 14:17 |
| *** rbuzatu has quit IRC | 14:17 | |
| fungi | and the main() in bandersnatch.main ends with "return config" | 14:17 |
| fungi | that seems insane^H^H^H^H^H^Hstrange | 14:19 |
| corvus | fungi: o/ | 14:19 |
| fungi | that "return config" line was only added in april of this year according to blame | 14:20 |
| fungi | so i have a feeling when we switched to latest upstream bandersnatch a few days ago, we began consistently getting nonzero exit status | 14:21 |
| tldr__ | pabelanger, hi, is it ok that pkgdeb-build-pkg job (http://logs.openstack.org/35/348435/1/check/pkgdeb-build-pkg/e55e9dc/console.html) doesn't use by default custom repo created by reprepro ? | 14:21 |
| *** rbuzatu has joined #openstack-infra | 14:22 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Don't spam postci with deployment time loop https://review.openstack.org/346990 | 14:22 |
| tldr__ | pabelanger, i thought, that it should consume package produced by pkgdeb-build-openstack-pkg-tools job | 14:22 |
| fungi | corvus: i'm mostly to the end of the bandersnatch troubleshooting now. are you still ansible'ing today or is the conference over? | 14:22 |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Record testenv wait times in ovb jobs https://review.openstack.org/347037 | 14:22 |
| corvus | fungi: sorry i ended up being afk all of yesterday, but it's over now | 14:23 |
| openstackgerrit | Tony Breeds proposed openstack-infra/bindep: Add '_' to the list of valid characters acceptable in package name https://review.openstack.org/348901 | 14:23 |
| corvus | fungi: i agree it sure looks like main is returning config | 14:23 |
| fungi | corvus: oh, cool. hope it was a good conference! | 14:23 |
| *** thiagolib has joined #openstack-infra | 14:23 | |
| corvus | me too :) | 14:24 |
| fungi | corvus: there's another unrelated but disturbing issue we discovered yesterday that i don't think anybody's worked on yet | 14:24 |
| *** skelso has joined #openstack-infra | 14:24 | |
| fungi | corvus: it seems like zuul-cloner is either not performing (or silently failing) a git remote update now, and so ends up assuming the cached branch state on disk is current | 14:25 |
| corvus | oh neat. we carefully updated that part of z-c recently | 14:25 |
| AJaeger | fungi, oops ;( | 14:25 |
| fungi | corvus: i got far enough to confirm that on a several-days-old centos-7 image in rax-ord, jobs running zuul-cloner were checking out the master branch tip concurrent with the time at which the image was created rather than the current origin remote's master branch tip | 14:26 |
| corvus | fungi: do you have an example failure? | 14:26 |
| fungi | i'll find one from scrollback | 14:26 |
| mtreinish | yolanda, rcarrillocruz: thanks for reviewing all those firehose patches | 14:26 |
| rcarrillocruz | np | 14:26 |
| yolanda | mtreinish, glad to help | 14:26 |
| *** EricGonczer_ has quit IRC | 14:26 | |
| *** zz_dimtruck is now known as dimtruck | 14:26 | |
| mtreinish | if anyone else has some spare review cycles down to 5 open: https://review.openstack.org/#/q/status:open+branch:master+topic:firehose :) | 14:27 |
| AJaeger | fungi, I reviewed a couple of changes for my constraints work and those all looked fine. I'm curious whether I misread the logs or the situation has changed... | 14:27 |
| AJaeger | s/changes/logs/ | 14:27 |
| fungi | corvus: http://logs.openstack.org/06/339806/44/check/gate-puppet-openstack-integration-3-scenario002-tempest-centos-7/4bf1d45/console.html#_2016-07-28_12_49_16_315962 | 14:28 |
| fungi | corvus: discussion starts around http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-07-28.log.html#t2016-07-28T16:28:39 | 14:29 |
| * fungi finishes updating bandersnatch issue and testing local patch | 14:29 | |
| *** PollyZ has joined #openstack-infra | 14:29 | |
| *** xarses has joined #openstack-infra | 14:30 | |
| *** hieulq_ has joined #openstack-infra | 14:30 | |
| corvus | fungi: did you find this? https://bitbucket.org/pypa/bandersnatch/commits/3d1cead6aea04941add45a7de07cc933f8461268?at=default | 14:30 |
| fungi | corvus: yep, doesn't explain why that line was added, but i'm about to link it in my comment | 14:31 |
| corvus | ya | 14:31 |
| fungi | found from blame | 14:31 |
| * corvus was pleased hg had blame | 14:31 | |
| *** EricGonczer_ has joined #openstack-infra | 14:31 | |
| *** asettle has quit IRC | 14:32 | |
| corvus | fungi: if this is only on centos7 -- i wonder if we're about to find out why we used update rather than fetch in the past (ie, why we thought they were different) | 14:33 |
| *** piet_ has joined #openstack-infra | 14:33 | |
| *** tlbr_ has joined #openstack-infra | 14:33 | |
| fungi | corvus: i didn't get far enough to check whether it's elsewhere. the one example we started from was centos 7 | 14:34 |
| fungi | so quite possibly, yes | 14:34 |
| *** claudiub|2 has joined #openstack-infra | 14:34 | |
| *** edmondsw has joined #openstack-infra | 14:34 | |
| *** tlbr_ has quit IRC | 14:34 | |
| fungi | that did (briefly) cross my mind as well | 14:34 |
| *** tonytan4ever has joined #openstack-infra | 14:35 | |
| *** rbuzatu has quit IRC | 14:35 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Implement Xenial/CentOS jobs for OpenStack-Ansible https://review.openstack.org/329892 | 14:35 |
| Zara | hi, all! https://review.openstack.org/#/c/330922/ has been waiting a week with two +2s, but no +workflow! who wants to be the lucky person to merge it? it could be you! :) | 14:35 |
| zaro | fungi, corvus : i'm looking into creating a gerrit plugin that will do a reverse lookup for 'depends-on:'. was considering calling it zuul plugin but not sure what else zuul related would go in there atm. | 14:36 |
| *** dimtruck is now known as zz_dimtruck | 14:36 | |
| openstackgerrit | Thierry Carrez proposed openstack-infra/project-config: Set up stable/newton temporary ACLs https://review.openstack.org/346691 | 14:36 |
| *** vmbrasseur has joined #openstack-infra | 14:37 | |
| rcarrillocruz | corvus: heya, if you're back from doing Ansible things could you look https://review.openstack.org/#/c/346914/ for the zuul job auth? | 14:37 |
| rcarrillocruz | got it reviews by jhesketh | 14:37 |
| pabelanger | morning | 14:38 |
| vmbrasseur | It's sysadmin appreciation day in the US, so: Thank you, Infra! OpenStack literally could not happen without you. <3 | 14:38 |
| * vmbrasseur shuffles on back to ##openstack-social. | 14:38 | |
| *** vmbrasseur has left #openstack-infra | 14:38 | |
| *** psachin has joined #openstack-infra | 14:39 | |
| rcarrillocruz | heh ^ | 14:40 |
| *** zhurong has joined #openstack-infra | 14:40 | |
| *** pcrews has quit IRC | 14:40 | |
| AJaeger | yeah, nice drive through ;) | 14:41 |
| jesusaur | vmbrasseur speaks my mind | 14:41 |
| fungi | zaro: would that maintain a lookup table to improve efficiency over a query for message:"Depends-On: Ixxxxxx"? | 14:41 |
| *** pcrews has joined #openstack-infra | 14:41 | |
| *** aysyd has quit IRC | 14:41 | |
| *** aysyd has joined #openstack-infra | 14:42 | |
| *** hurgleburgler has joined #openstack-infra | 14:42 | |
| *** david-lyle has joined #openstack-infra | 14:42 | |
| *** rbuzatu has joined #openstack-infra | 14:42 | |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Allow Ansible role checks to collect logs https://review.openstack.org/348907 | 14:43 |
| *** zz_dimtruck is now known as dimtruck | 14:44 | |
| *** rbuzatu_ has joined #openstack-infra | 14:44 | |
| fungi | corvus: i've confirmed that after removing "return config" from the end of bandersnatch.main.main() our cron job is back to properly calling vos release. i've updated the bandersnatch bug and am preparing a patch (after i reacquaint myself with using hg with bitbucket) | 14:44 |
| zaro | fungi: i wasn't thinking of that. It would just simply be show that change Iyyyyy 'is a dependency of' Ixxxxx | 14:44 |
| *** zhurong_ has joined #openstack-infra | 14:44 | |
| *** jcoufal_ has joined #openstack-infra | 14:44 | |
| fungi | zaro: oh, for using in the webui? | 14:44 |
| corvus | fungi, zaro: if it's in the webui, then it would presumably also be in the api, which would mean that zuul would not need to perform the "message:" query, yeah? | 14:45 |
| corvus | rcarrillocruz: cool, will do, thanks! | 14:45 |
| zaro | fungi: yeah, that was driving me crazy with the puppet changes i needed to do for its-storyboard | 14:45 |
| *** zhurong has quit IRC | 14:46 | |
| fungi | corvus: presumably, yes | 14:46 |
| *** david-lyle_ has joined #openstack-infra | 14:46 | |
| *** jcoufal has quit IRC | 14:47 | |
| *** rbuzatu_ has quit IRC | 14:47 | |
| zaro | corvus: it's already in the api. i was thinking that webui would just do that same 'message:' query then just present the info. | 14:47 |
| *** david-lyle has quit IRC | 14:47 | |
| *** rbuzatu has quit IRC | 14:47 | |
| fungi | also might make it easier for other tools like gertty to support depends-on (perhaps wrapped in a plugin check?) | 14:47 |
| pabelanger | anybody mind +3 on https://review.openstack.org/#/c/347158/ adds tripleo-test-cloud-rh2 into ansible on puppetmaster.o.o | 14:47 |
| *** david-lyle__ has joined #openstack-infra | 14:48 | |
| fungi | oh, so not actually representing them as dependencies from gerrit's perspective, just displaying them in the webui so you get automated needed-by tracking | 14:48 |
| Zara | (the other its-storyboard patches can be found here: https://review.openstack.org/#/q/topic:its-storyboard+status:open . most have at least one +2 now; thanks again, everyone! :D) | 14:48 |
| zaro | fungi: yes, that's all i was thinking. | 14:49 |
| bhunter71 | Question: in my private openstackCI system - all my dsvm/devstack tests [trusty still] are currently failing with something like "fatal error: ffi.h: No such file or directory". Does this ring a bell? (I've been busy moving nodepool from snapshot to DIB, .. so it's *probably* something about my images.) | 14:49 |
| *** tlbr_ has joined #openstack-infra | 14:50 | |
| corvus | i think if you wanted to do needed-by you should also do depends-on; even though it's already in the commit, it's going to be weird if it doesn't also show up in the same way that needed-by does | 14:50 |
| fungi | oh, excellent. our bandersnatch cron wrapper only runs bandersnatch under a timeout, and not vos release! was worried for a moment that the latter might get prematurely terminated if this takes too long to transfer the complete update | 14:51 |
| fungi | someone was thinking ahead ;) | 14:51 |
| *** david-lyle_ has quit IRC | 14:52 | |
| fungi | Starting ForwardMulti from 536870932 to 536870932 on afs02.dfw.openstack.org (as of Wed Jul 27 09:35:49 2016). | 14:52 |
| corvus | fungi: did clarkb do the vos release after the full sync? | 14:52 |
| zaro | corvus: can you explain a little more of what you mean by 'do the same'? simply add that info to webui or something more? | 14:52 |
| *** Teresita-Warrior has joined #openstack-infra | 14:53 | |
| *** rbuzatu has joined #openstack-infra | 14:53 | |
| fungi | corvus: i completed the manual full sync plus a couple of catch-up passes until bandersnatch no longer had any new serials to process. then i released the flock and am watching the cronjob (now that i've also temporarily hand-patched bandersnatch to properly exit 0 on success) | 14:53 |
| *** tlbr_ has quit IRC | 14:53 | |
| *** rbuzatu has quit IRC | 14:53 | |
| *** siva_krish has joined #openstack-infra | 14:54 | |
| *** rbuzatu has joined #openstack-infra | 14:54 | |
| corvus | zaro: i mean that if you're going to add parsing of "Depends-On:" in commit messages to needed_by in the api/webui, then you probably also want them to show up as depends_on in the same place in the api/webui | 14:54 |
| corvus | fungi: ok, so yeah, that hopefully will take much less than 10 hours :) | 14:54 |
| *** rbuzatu_ has joined #openstack-infra | 14:54 | |
| *** burgerk has joined #openstack-infra | 14:54 | |
| corvus | fungi: (so we don't need to worry about running it on the server to avoid token expiration) | 14:55 |
| *** siva_krish1 has joined #openstack-infra | 14:55 | |
| *** jbernard has quit IRC | 14:55 | |
| odyssey4me | pabelanger ping me when you're ready to talk through https://review.openstack.org/348907 / https://review.openstack.org/329892 (or other options) | 14:56 |
| fungi | corvus: i hope it's not got 10 hours worth of data transfer pending anyway | 14:56 |
| pabelanger | tldr__: I'll have to check the job and if it is opting into the reprepro repo | 14:56 |
| fungi | corvus: since our mirror has basically already been stale for 29.5 hours | 14:57 |
| *** siva_krish has quit IRC | 14:58 | |
| *** rbuzatu has quit IRC | 14:58 | |
| zaro | corvus: 'depends-on' info is already in the commit message. wouldn't showing it in webui be just duplication? | 14:59 |
| tldr__ | pabelanger, great, thank you | 14:59 |
| corvus | zaro: yes, but if you're suggesting displaying needed-by somewhere _other_ than the commit message, then i think you should show depends_on there as well, right? otherwise it will be a confusing one-way relationship | 14:59 |
| zaro | corvus: also you mention api, are you suggesting a specific search term? like 'depends-on: Ixxxx' ? | 15:00 |
| corvus | zaro: no, i'm just referring to the fact that to use something in the webui, it has to be in the api | 15:00 |
| zaro | corvus: ok, i'll look into adding both on webui. | 15:01 |
| *** rbuzatu_ has quit IRC | 15:03 | |
| zaro | corvus: not sure i should call it zuul plugin if that's all it does. unless you have other suggestions? | 15:04 |
| *** mhickey has quit IRC | 15:05 | |
| *** vincentll has quit IRC | 15:05 | |
| tldr__ | pabelanger, will it take much time to check it ? | 15:05 |
| pabelanger | AJaeger: post queue is just starved for nodes, thats why it keeps growing. Hopefully when bluebox and infracloud come back online we'll have the nodes to keep up with it. | 15:06 |
| pabelanger | tldr__: looks like we need to land https://review.openstack.org/#/c/341856/ then update your job to symlink it | 15:07 |
| pabelanger | fungi: do you mind a +3 on the debian-openstack repo configure_mirror.sh | 15:07 |
| pabelanger | ^ | 15:07 |
| tldr__ | pabelanger, ok, thanks, will do that after merge | 15:07 |
| fungi | pabelanger: did the change that makes that repo merge? | 15:08 |
| fungi | and does that repo work? | 15:08 |
| openstackgerrit | Alexander Evseev proposed openstack-infra/project-config: Fix trusted host for NODEPOOL_PYPI_MIRROR https://review.openstack.org/336023 | 15:08 |
| pabelanger | fungi: ya, let me actually remove the backports for now in configure_mirror.sh, since we need to address that still | 15:08 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add debian-openstack source.list to debian https://review.openstack.org/341856 | 15:10 |
| *** jistr is now known as jistr|call | 15:10 | |
| *** rbuzatu has joined #openstack-infra | 15:11 | |
| *** jcoufal_ has quit IRC | 15:11 | |
| *** jcoufal has joined #openstack-infra | 15:12 | |
| *** armax has joined #openstack-infra | 15:12 | |
| *** e0ne has quit IRC | 15:14 | |
| *** jbernard has joined #openstack-infra | 15:17 | |
| EmilienM | fungi, pabelanger: did we find out what was wrong with zuul cloner? | 15:17 |
| *** jistr|call is now known as jistr | 15:17 | |
| fungi | EmilienM: i have not yet, though corvus is also aware now and may be investigating too | 15:17 |
| corvus | i am | 15:18 |
| *** psilvad has quit IRC | 15:18 | |
| fungi | EmilienM: current open question is whether it's only happening with the git version on centos 7 or also other platforms | 15:18 |
| *** PollyZ has quit IRC | 15:18 | |
| EmilienM | interesting | 15:18 |
| *** skelso has quit IRC | 15:21 | |
| *** PollyZ has joined #openstack-infra | 15:21 | |
| *** ayoung has joined #openstack-infra | 15:22 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Switch to gate-{name}-linters-{node} https://review.openstack.org/347939 | 15:23 |
| *** david-lyle has joined #openstack-infra | 15:23 | |
| *** david-lyle__ has quit IRC | 15:23 | |
| *** bhavik has joined #openstack-infra | 15:24 | |
| pabelanger | clarkb: AJaeger: ^ please double check the regex. | 15:25 |
| *** karthikp_ has joined #openstack-infra | 15:25 | |
| *** weshay is now known as weshay_bbiab | 15:25 | |
| *** esberglu has joined #openstack-infra | 15:26 | |
| clarkb | pabelanger: regexes look good | 15:26 |
| *** inc0 has joined #openstack-infra | 15:27 | |
| *** PollyZ has quit IRC | 15:27 | |
| *** psilvad has joined #openstack-infra | 15:27 | |
| *** karthikp_ has quit IRC | 15:27 | |
| *** mdrabe has quit IRC | 15:28 | |
| *** zhurong_ has quit IRC | 15:28 | |
| clarkb | pabelanger: looks like we can revert the ovh-gra1 max-servers limiting now too | 15:28 |
| *** Guest79767 has joined #openstack-infra | 15:29 | |
| fungi | okay, bandersnatch pull request submitted. i need to step away for a few but the vos release is still underway (tail /var/log/bandersnatch/mirror.log on mirror-update.o.o to check its status) | 15:29 |
| *** PollyZ has joined #openstack-infra | 15:30 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/tripleo: optimize multinode and undercloud layout https://review.openstack.org/348927 | 15:30 |
| mordred | morning all | 15:31 |
| clarkb | fungi: is the error you mentioned that my run ran into related to the return code of 1 or soemthing else? | 15:31 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul: Add docstrings to the model https://review.openstack.org/348928 | 15:31 |
| fungi | clarkb: i'm not completely sure since our wrapper script seems to eat all output if it runs into an error | 15:32 |
| fungi | that should probably be solved separately | 15:32 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config: zuul/tripleo: optimize multinode and undercloud layout https://review.openstack.org/348927 | 15:32 |
| *** Liuqing has quit IRC | 15:33 | |
| *** rwsu has quit IRC | 15:33 | |
| fungi | i'm _guessing_ that either the nonzero exit code and/or the additional "<ConfigParser.ConfigParser instance at 0x7fd5fdd74710>" line in the output were confusing our output parser | 15:33 |
| *** Guest79767 has quit IRC | 15:34 | |
| *** yamamoto has joined #openstack-infra | 15:34 | |
| *** mdrabe has joined #openstack-infra | 15:35 | |
| *** jlanoux has quit IRC | 15:36 | |
| mordred | clarkb: I wasn't fully able to follow yesterday, but it looked like we might still be having issues with async things in new ansible that need to be investigated? | 15:37 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config: Revert "Reduce ovh-gra1 to 40 nodes" https://review.openstack.org/348934 | 15:38 |
| *** nhandler has joined #openstack-infra | 15:38 | |
| clarkb | pabelanger: ^ should get us back to using all of ovh | 15:38 |
| clarkb | mordred: uh maybe? I was pretty swamped in elasticsearch and bandersnatch and xenial things | 15:38 |
| clarkb | elasticsearch remains happy looking this morning, the doc counts are climbing as expected | 15:39 |
| *** sdake has joined #openstack-infra | 15:39 | |
| mordred | clarkb: ok. I'm travlling today but shoudl be onlineish, then I'm actually online for the next 6 weeks, so poke me if it breaks again | 15:40 |
| *** tlbr_ has joined #openstack-infra | 15:40 | |
| *** rbuzatu has quit IRC | 15:40 | |
| *** PollyZ has quit IRC | 15:41 | |
| *** rbuzatu has joined #openstack-infra | 15:41 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Created DSVM Job for NPM Projects https://review.openstack.org/348056 | 15:42 |
| clarkb | mordred: I know pabelanger mentioned that ansible had oomed again yesterday so I don't think the patch to use ssh to localhost there changed that drastically | 15:43 |
| *** PollyZ has joined #openstack-infra | 15:43 | |
| mordred | clarkb: sweet | 15:43 |
| clarkb | mordred: probably worthwhile getting your facts handling one in next to see if that helps | 15:43 |
| mordred | ++ | 15:43 |
| mordred | clarkb: I'll put that on the list for Monday | 15:43 |
| * krotscheck does his little devstack dance | 15:43 | |
| clarkb | mordred: I am pretty sure I reviewed those but just didn't approive any because wanted people to do them one by one and watch the system to see if the changes affected memory use | 15:44 |
| clarkb | mordred: that way we'd know what actually helps and doesn't | 15:44 |
| *** tlbr_ has quit IRC | 15:45 | |
| mordred | clarkb: totally agree | 15:45 |
| pabelanger | clarkb: nice, what was the issue? | 15:46 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Created DSVM Job for NPM Projects https://review.openstack.org/348056 | 15:46 |
| clarkb | pabelanger: ovh didn't say in their email just thatthey had corrected it | 15:46 |
| clarkb | I guess I hsould double check on our side and ask nova for our quota usage | 15:46 |
| AJaeger | clarkb, pabelanger, fungi : Can we speed up merge of the ovh-gra1 change, please? https://review.openstack.org/#/c/348934 With the high load, we should consider doing a short-cut if possible... | 15:47 |
| clarkb | AJaeger: yup let me just double check that we have correct quota use reported by nova first | 15:47 |
| pabelanger | clarkb: did they reply to you directly? | 15:47 |
| AJaeger | clarkb: sure, thanks! | 15:48 |
| clarkb | pabelanger: no the original people you CC'd and yourself were all copied | 15:49 |
| pabelanger | clarkb: okay, thanks. Checking spam folder now | 15:49 |
| *** deadnull is now known as _deadnull | 15:49 | |
| *** karthikp_ has joined #openstack-infra | 15:50 | |
| clarkb | hrm openstack quota show not really easiest thing int he world to use | 15:50 |
| *** hieulq_ has quit IRC | 15:51 | |
| clarkb | wants me to be admin? | 15:51 |
| clarkb | I just want to see my own quota... | 15:51 |
| openstackgerrit | Merged openstack-infra/project-config: neutron-fwaas: Fix a typo (ENABLE_SERVICES vs ENABLED_SERVICES) https://review.openstack.org/348734 | 15:51 |
| EmilienM | can someone review https://review.openstack.org/#/c/348927/ please? it will save nodepool resources :) thanks | 15:51 |
| clarkb | oh right I have to use 'limits' | 15:51 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Created DSVM Job for NPM Projects https://review.openstack.org/348056 | 15:51 |
| *** roxanaghe has joined #openstack-infra | 15:52 | |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Created DSVM Job for NPM Projects https://review.openstack.org/348056 | 15:52 |
| pabelanger | AJaeger: clarkb: going to enqueue 348934 into gate so we don't have to wait 4 hours | 15:52 |
| krotscheck | Ok, I think I'm finally ready | 15:52 |
| clarkb | pabelanger: AJaeger totalinstances used is 40 and max instances is 200 so looks good from here | 15:52 |
| pabelanger | clarkb: great | 15:53 |
| pabelanger | and 348934,1 now it gate | 15:54 |
| clarkb | the incantation is `openstack limits show --absolute` | 15:55 |
| clarkb | that quota command is just a honey pot | 15:55 |
| ihrachys | pabelanger: howdy | 15:56 |
| *** skelso has joined #openstack-infra | 15:56 | |
| *** roxanaghe has quit IRC | 15:56 | |
| ihrachys | pabelanger: http://logs.openstack.org/68/347068/1/gate/gate-tempest-dsvm-neutron-full-ubuntu-trusty/44688d3/logs/devstacklog.txt.gz#_2016-07-29_04_12_51_362 | 15:56 |
| ihrachys | that's stable mitaka neutron | 15:56 |
| ihrachys | job was renamed lately | 15:56 |
| ihrachys | I see you migrated jobs to enial | 15:56 |
| ihrachys | *xenial | 15:56 |
| ihrachys | can it be related? | 15:56 |
| clarkb | ihrachys: that job ran on trusty so I really doubt it | 15:57 |
| *** tphummel has joined #openstack-infra | 15:57 | |
| *** sambetts has joined #openstack-infra | 15:57 | |
| clarkb | ihrachys: we need to go look at the setup workspace logs to see why the requirements repo was not cloned | 15:58 |
| ihrachys | clarkb: before, I haven't seen that -trusty suffix | 15:58 |
| fungi | in theory the only thing that changed with mitaka runs is that the job name grew a -trusty suffix | 15:58 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: gate-project-config-layout should use ubuntu-trusty https://review.openstack.org/348945 | 15:58 |
| clarkb | ihrachys: http://logs.openstack.org/68/347068/1/gate/gate-tempest-dsvm-neutron-full-ubuntu-trusty/44688d3/logs/devstack-gate-setup-workspace-new.txt.gz#_2016-07-29_04_12_35_815 beacuse it bailed out early | 15:58 |
| pabelanger | clarkb: AJaeger: Fixes a issue with gate-project-config-layout ^, should be ubuntu-trusty online | 15:58 |
| clarkb | so not related to xenial at all | 15:58 |
| fungi | ihrachys: that's so that we can limit that job to not run against master changes (where we run an equivalent job with a -xenial suffix instead) | 15:58 |
| ihrachys | oh | 15:58 |
| ihrachys | recheck then? | 15:58 |
| *** karthikp_ has quit IRC | 15:58 | |
| *** jcoufal has quit IRC | 15:59 | |
| *** sdague has quit IRC | 15:59 | |
| tonyb | ihrachys: I don't think so, I'm looking at devstack-gate | 15:59 |
| *** jcoufal has joined #openstack-infra | 15:59 | |
| *** rossella_s has quit IRC | 15:59 | |
| *** rossella_s has joined #openstack-infra | 16:00 | |
| pabelanger | fungi: thanks | 16:00 |
| ihrachys | tonyb: see replies there, seems like it's red herring | 16:00 |
| sambetts | Hi Infra, I have a question about how the gate processes the shell builders, in a shell builder I am sourcing a file, but its not behaving like I'm sourcing the file, exported variables aren't being passed through to it, and variables exported by it aren't accessable once the script leaves the sourced file, any ideas? this logic works perfectly fine locally with normal bash scripts | 16:00 |
| clarkb | ya I Think recheck is best bet, this job ran in rax ord so likely related to the intermittent network issues in rax | 16:00 |
| ihrachys | tonyb: ehm, sorry, was thinking we are on #stable :) | 16:01 |
| *** pcaruana has quit IRC | 16:01 | |
| ihrachys | clarkb: will recheck and see. thanks! | 16:01 |
| AJaeger | sambetts: do you have an example? | 16:01 |
| fungi | sambetts: each builder runs in a separate shell env, so sourcing variables in one builder won't make them available in another if that's what you're trying | 16:01 |
| tonyb | ihrachys: sorry I bounced over here because you said you'd poked infra. I figured it was best to kee the conversation in one place. | 16:01 |
| *** john-davidge has quit IRC | 16:01 | |
| sambetts | fungi: no, I have have one builder generating a file, and a second builder sourcing that file | 16:02 |
| *** karthikp_ has joined #openstack-infra | 16:02 | |
| ihrachys | tonyb: np, I just mixed things. anyhow, it should be good with recheck | 16:02 |
| *** john-davidge has joined #openstack-infra | 16:02 | |
| sambetts | AJaeger, fungi: http://logs.openstack.org/59/348859/2/check/gate-tempest-dsvm-ironic-multitenant-network-nv/fccc59a/console.html this is the log of the failing job | 16:02 |
| fungi | sambetts: should work in theory at least | 16:02 |
| sambetts | fungi: https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/ironic.yaml#L195 | 16:03 |
| tonyb | ihrachys: on an unrelated note that review mixes liberty and mikata code so ... is that right? | 16:03 |
| *** cloudtrainme has quit IRC | 16:03 | |
| ihrachys | tonyb: which one? | 16:03 |
| tonyb | ihrachys: 347068 | 16:03 |
| fungi | sambetts: i should be able to check it out in a few minutes if nobody beats me to the answer | 16:03 |
| AJaeger | sambetts: I only see touch ironic-extra-vars;source ironic-extra-vars - but nothing written to it. | 16:04 |
| ihrachys | tonyb: elaborate, not following | 16:04 |
| sambetts | AJaeger: the file is populated by other builders https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/ironic.yaml#L90 | 16:04 |
| *** tphummel has quit IRC | 16:04 | |
| AJaeger | sambetts: but that should be seen in the log file | 16:04 |
| tonyb | ihrachys: Look at http://logs.openstack.org/68/347068/1/gate/gate-tempest-dsvm-neutron-full-ubuntu-trusty/44688d3/logs/reproduce.sh ZUUL_CHANGES has references to stable/liberty | 16:04 |
| *** rbuzatu has quit IRC | 16:05 | |
| tonyb | ihrachys: not sure why it doesn;t seem to have any Depends-On | 16:05 |
| sambetts | AJaeger: you can see it in the log file after sources ironic-extra-vars every line ++ something is from that file | 16:05 |
| AJaeger | sambetts: let's look at the job definition, your change was https://review.openstack.org/340567 - give me a few minutes | 16:05 |
| ihrachys | tonyb: that's probably how zuul handles gate merges because you need to validate with all patches that are on top of the tested patch in merge queue | 16:06 |
| ihrachys | tonyb: for check queue that would be weird, but for merge it's fine | 16:06 |
| sambetts | AJaeger, fungi: in the main script we do export PROJECTS="openstack/ironic" and then in the sourced file we do export PROJECTS="openstack/generic-network-switch $PROJECTS", but the $PROJECTS part isn't being filled in, even though it should be | 16:06 |
| tonyb | ihrachys: Oh it was in the gate pipeline, sorry | 16:06 |
| AJaeger | sambetts: so, it's writing here: http://logs.openstack.org/67/340567/2/check/gate-project-config-compare-xml/d37019a/console.html.gz#_2016-07-12_11_34_18_518907 | 16:07 |
| tonyb | ihrachys: and clark already explained the probelm so I'll stop being a distraction. | 16:07 |
| vdrok | the ironic-multitenant builder overwrites the main builder completely - http://logs.openstack.org/18/285218/10/check/gate-tempest-dsvm-ironic-multitenant-network-nv/67947f3/logs/reproduce.sh | 16:07 |
| AJaeger | sambetts: the ansible files contains the scripts - that misled me. See http://logs.openstack.org/59/348859/2/check/gate-tempest-dsvm-ironic-multitenant-network-nv/fccc59a/_zuul_ansible/scripts/3e32f4cb4ab145c3b84a259f6400fde9.sh for an example | 16:08 |
| *** karthikp_ has quit IRC | 16:08 | |
| *** pilgrimstack has quit IRC | 16:09 | |
| sambetts | AJaeger: I have an idea of what might be happening | 16:09 |
| *** mikelk has quit IRC | 16:09 | |
| sambetts | AJaeger: I think that when Ansible is running the script to the write the file its trying to fill in $PROJECTS at that point | 16:10 |
| sambetts | AJaeger: instead of leaving it alone to be processed when the script is actually run later | 16:10 |
| *** rbuzatu has joined #openstack-infra | 16:10 | |
| *** skelso has quit IRC | 16:11 | |
| AJaeger | sambetts: that might be the case - since http://logs.openstack.org/59/348859/2/check/gate-tempest-dsvm-ironic-multitenant-network-nv/fccc59a/console.html#_2016-07-29_13_10_48_722076 shows that the old value of PROJECTS is empty | 16:11 |
| AJaeger | or was substituted in as empty | 16:12 |
| clarkb | oh huh we were only using 79 of our 200 quota in gra1 | 16:12 |
| sambetts | AJaeger: yeah I think that its actually processing $PROJECTS too early | 16:12 |
| clarkb | anyone know if we can just go straight to 200? | 16:12 |
| clarkb | corvus: ^ I think you were heavily involved in sorting out usage there | 16:12 |
| corvus | clarkb: see https://review.openstack.org/347075 | 16:13 |
| *** jcoufal has quit IRC | 16:13 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add openstackci-tripleo-test-clouds to ansible-clouds.yaml https://review.openstack.org/347158 | 16:13 |
| sambetts | AJaeger: shame I can't view the file that is generated by those builders in the logs | 16:13 |
| *** yaume has quit IRC | 16:13 | |
| clarkb | corvus: ya I just reverted ^ because ovh reported they had corrected the quota. But that change went from 79 instances to 40 | 16:13 |
| openstackgerrit | Sam Betts proposed openstack-infra/project-config: Prevent early variable substitution in Ironic extra vars https://review.openstack.org/348949 | 16:14 |
| clarkb | our quota in the cloud is 200 instances | 16:14 |
| sambetts | AJaeger: an attempt at fixing it ^ | 16:14 |
| corvus | clarkb: ooh i see | 16:14 |
| *** jcoufal has joined #openstack-infra | 16:14 | |
| corvus | clarkb: i don't recall why 79 | 16:14 |
| AJaeger | sambetts: will check later - dinner time... | 16:15 |
| pabelanger | clarkb: ya, 200 nodes would be nice | 16:15 |
| *** jed56 has quit IRC | 16:15 | |
| corvus | clarkb: https://review.openstack.org/249514 is previous change | 16:15 |
| *** rbuzatu has quit IRC | 16:15 | |
| sambetts | AJaeger: confirmed locally with scripting that its the early subsitution thats at least part of the problem | 16:16 |
| pabelanger | corvus: clarkb: when you have a moment, if you don't mind reviewing: https://review.openstack.org/#/c/347158/ Working on bringing tripleo-test-cloud-rh1 AFS mirror online | 16:16 |
| clarkb | maybe worth a follow up to Damien and ask if we can use the full 200? | 16:17 |
| *** rbuzatu has joined #openstack-infra | 16:17 | |
| corvus | clarkb: probably so | 16:17 |
| clarkb | pabelanger: I am going to channel mordred, Why are we hardcoding keystone v2 there? | 16:18 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Disable tripleo-test-cloud-rh1 until we have AFS mirrors https://review.openstack.org/348954 | 16:19 |
| pabelanger | clarkb: not sure honestly, maybe we can remove it | 16:19 |
| clarkb | I +2'd the change because it should work but I know mordred likes to not specify the version so that it autonegotiates v3 | 16:20 |
| openstackgerrit | Giulio Fidente proposed openstack-infra/tripleo-ci: Install Ceph Hammer and/or Jewel from CentOS Storage SIG repos https://review.openstack.org/340504 | 16:20 |
| pabelanger | clarkb: Ya, I'd need some help with the syntax, but can change it | 16:20 |
| clarkb | corvus: pabelanger ok I will respond and ask about using the full quota | 16:20 |
| *** Liuqing has joined #openstack-infra | 16:21 | |
| openstackgerrit | Merged openstack-infra/project-config: Revert "Reduce ovh-gra1 to 40 nodes" https://review.openstack.org/348934 | 16:22 |
| openstackgerrit | Merged openstack-infra/project-config: gate-project-config-layout should use ubuntu-trusty https://review.openstack.org/348945 | 16:22 |
| sambetts | fungi: when you get a chance https://review.openstack.org/348949 | 16:23 |
| sambetts | fungi: thanks a bunch | 16:23 |
| *** florianf has quit IRC | 16:23 | |
| *** tphummel has joined #openstack-infra | 16:24 | |
| *** weshay_bbiab is now known as weshay | 16:24 | |
| *** trown is now known as trown|lunch | 16:25 | |
| *** jed56 has joined #openstack-infra | 16:26 | |
| *** jpich has quit IRC | 16:26 | |
| *** sambetts is now known as sambetts|afk | 16:27 | |
| clarkb | huh our bhs1 quota is 600 instances (though with cpus I Think we can only do 300) | 16:27 |
| *** cloudtrainme has joined #openstack-infra | 16:27 | |
| *** Guest30197 has joined #openstack-infra | 16:28 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci: Source rh1.env to decide how many testenv workers to run https://review.openstack.org/348883 | 16:29 |
| *** tesseract- has quit IRC | 16:30 | |
| prometheanfire | any chance diskimagebuilder people can review this https://review.openstack.org/#/c/348536/ | 16:30 |
| prometheanfire | and system config people this? https://review.openstack.org/#/c/348514/ | 16:30 |
| anteaya | clarkb: 300 is still up from 159 which it is set at | 16:30 |
| *** permalac has quit IRC | 16:31 | |
| clarkb | yup | 16:31 |
| *** rbuzatu has quit IRC | 16:32 | |
| prometheanfire | the system-config change is a oneliner | 16:32 |
| *** jamielennox|away is now known as jamielennox | 16:32 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Mirror centos storate mirror to AFS https://review.openstack.org/348959 | 16:33 |
| clarkb | email sent, we shall see what they say | 16:33 |
| pabelanger | dmsimard: EmilienM: ^ will mirror the packages for us | 16:33 |
| * anteaya hopes they say some version of up your settings | 16:33 | |
| *** Guest30197 has quit IRC | 16:33 | |
| *** roxanaghe has joined #openstack-infra | 16:34 | |
| *** tqtran has joined #openstack-infra | 16:34 | |
| *** tqtran_ has joined #openstack-infra | 16:35 | |
| EmilienM | pabelanger: nice :D | 16:35 |
| dmsimard | pabelanger: well that was easy | 16:36 |
| dmsimard | pabelanger: note that storage also includes gluster | 16:36 |
| fungi | oh jeez, the new pypi also makes the simple api not at all browser-friendly, if anybody was relying on that :/ | 16:36 |
| *** psachin has quit IRC | 16:36 | |
| fungi | view-source:http://mirror.iad.rax.openstack.org/pypi/simple/python-tackerclient/ | 16:36 |
| pabelanger | dmsimard: yup, we have some good mirror infra atm | 16:36 |
| fungi | http://mirror.iad.rax.openstack.org/pypi/simple/python-tackerclient/ | 16:36 |
| fungi | but it looks from the page source like it'll be fine for pip | 16:36 |
| clarkb | huh thats probably a simpleish bandersnatch patch? of course simple means using hg | 16:37 |
| *** rbuzatu has joined #openstack-infra | 16:37 | |
| pabelanger | dmsimard: that should be fine, we'll add the repo into the images, but disable the repo. Projects will need to opt in | 16:37 |
| fungi | clarkb: yeah, i suppose i can take a shot at that since i just did the same dance | 16:37 |
| clarkb | I am guessing there is an html template that needs updating | 16:38 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Mirror centos storage mirror to AFS https://review.openstack.org/348959 | 16:38 |
| *** tqtran has quit IRC | 16:38 | |
| openstackgerrit | Ben Kero proposed openstack-infra/project-config: Move {name}-publish-specs job + others to ubuntu-xenial https://review.openstack.org/348787 | 16:39 |
| *** tlbr_ has joined #openstack-infra | 16:40 | |
| openstackgerrit | Anita Kuno proposed openstack-infra/storyboard: Add example commands for the Teams api https://review.openstack.org/339123 | 16:41 |
| *** lucasagomes is now known as lucas-dinner | 16:41 | |
| *** degorenko is now known as _degorenko|afk | 16:41 | |
| *** rbuzatu has quit IRC | 16:42 | |
| dmsimard | pabelanger: FWIW the centos virt sig repo is included in "centos-release-openstack-mitaka" | 16:42 |
| dmsimard | pabelanger: because it contains qemu-kvm-ev which is better than qemu-kvm but I keep forgetting why exactly.. I think it has some RHEL backports with improved features or something | 16:43 |
| dmsimard | and qemu-kvm-ev obsoletes qemu-kvm so if you try to install qemu-kvm, it'll pick qemu-kvm-ev if it's available | 16:43 |
| dmsimard | I mention this because I see it's also excluded | 16:43 |
| tosky | pabelanger: about images, can I download the centos 7 and the ubuntu 16.04 images used on the gates on my jobs? And can I check them (aka: are they accessible outside infra) to check if there is some specific package included? | 16:43 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Revert "Add back rh1" https://review.openstack.org/348960 | 16:44 |
| mordred | clarkb: thanks for channeling me | 16:44 |
| mordred | pabelanger: where is the keystone v2 hardcoded? | 16:44 |
| *** tlbr_ has quit IRC | 16:44 | |
| *** _deadnull is now known as deadnull | 16:44 | |
| *** Liuqing has quit IRC | 16:44 | |
| dmsimard | mordred: channeling ? like summoning a deity or something ? :p | 16:44 |
| mordred | heh | 16:44 |
| pabelanger | anteaya: clarkb: AJaeger: mind help with rh1 revert?^ nodepool is failing to launch instances right now, and won't be for a while until I launch the AFS server | 16:44 |
| * anteaya looks | 16:45 | |
| * mordred throws lightning bolts at people ... hopes other people can see them too and it's not only in his mindgrapes | 16:45 | |
| pabelanger | mordred: clarkb was commenting on 347158 | 16:45 |
| pabelanger | mordred: I'm happy to change it, just need some guidance on syntax | 16:45 |
| *** e0ne has joined #openstack-infra | 16:45 | |
| *** e0ne has quit IRC | 16:46 | |
| pabelanger | clarkb: ovh-gra1 patch just landed on nodepool.o.o, lets see if it works | 16:46 |
| mordred | pabelanger: yes - looks like you can just drop the v2.0 | 16:46 |
| clarkb | pabelanger: you have isntances in rh1 so you need to set max servers to -1 or 0 first | 16:46 |
| mordred | pabelanger: I tested that by doing: curl https://ci-overcloud.rh1.tripleo.org:13000 | 16:46 |
| clarkb | otherwise nodepool will get cranky about those instances in rh1 | 16:46 |
| pabelanger | clarkb: no, we don't have anything online yet | 16:47 |
| corvus | fungi, pabelanger: i'm fairly certain we're dealing with a git behavior difference on centos vs xenial at least | 16:47 |
| fungi | #status log Our PyPI mirrors should be current again as of 16:10 UTC today | 16:47 |
| openstackstatus | fungi: finished logging | 16:47 |
| mordred | pabelanger: that shows me that versionless does return the list of available versions so discovery should work | 16:47 |
| clarkb | pabelanger: they are in the database and set to delete | 16:47 |
| *** Liuqing has joined #openstack-infra | 16:47 | |
| clarkb | pabelanger: those deletes will fail if we remove the config entirely | 16:47 |
| pabelanger | clarkb: ah, right | 16:47 |
| mordred | pabelanger: I have not attempted to run commands yet though - lemme grab the creds from hiera real quick and make sure | 16:47 |
| clarkb | pabelanger: if you set max servers to -1 it should be fine | 16:47 |
| pabelanger | clarkb: okay, I'll update the patch | 16:47 |
| clarkb | it will stop booting instance and building images and give you time to make afs mirror things happy | 16:47 |
| corvus | fungi, pabelanger: i have reproduced the problem on a xenal and centos host; centos7 checks out the cache tip, xenial correctly updates. switching z-c back to using .update() instead of .fetch() causes it to work both places. | 16:48 |
| fungi | corvus: good, at least that means the damage (if any) is probably not as widespread as it could have been | 16:48 |
| *** tonytan_brb has joined #openstack-infra | 16:48 | |
| corvus | git version 1.8.3.1 vs 2.7.4 | 16:49 |
| pabelanger | clarkb: anteaya: 348954 should be the correct patch now | 16:49 |
| * anteaya looks again | 16:49 | |
| pabelanger | corvus: thanks for looking into it. It would have taken me a while to figure that out | 16:49 |
| mordred | pabelanger: also, I only see openstack_nodepool_tripleo_username in hiera | 16:50 |
| fungi | corvus: wow, centos 7 sure picked a surprisingly old version of git given what should have been contemporary with it | 16:50 |
| pabelanger | mordred: ya, I haven't updated hiera yet | 16:50 |
| fungi | git 1.9 became available as far back as 2014-02-14 | 16:51 |
| clarkb | its 1.7.something right? | 16:51 |
| fungi | i wonder when centos 7 was stabilized | 16:51 |
| clarkb | or was that 6? | 16:51 |
| *** tonytan4ever has quit IRC | 16:51 | |
| fungi | centos 7 has a 1.8.x version as corvus mentioned above | 16:51 |
| pabelanger | fungi: IIRC, we are installing a patched version to from ianw because of an SSL preformance issues. I'd have to check install_puppet.sh again to confirm | 16:51 |
| mordred | pabelanger: ah - ok. then ignore my review of those lines | 16:52 |
| *** rhallisey has quit IRC | 16:52 | |
| corvus | "Git Source Code Mirror - This is a publish-only repository and all pull requests are ignored. Please follow Documentation/SubmittingPatches procedure for any of your improvements. " | 16:52 |
| mordred | pabelanger: in any case, I have verified that no v2.0 works | 16:52 |
| corvus | pleia2: we should probably do something like that with our project descriptions on github ^ | 16:52 |
| pabelanger | mordred: great, I'll update the patch now | 16:52 |
| fungi | corvus: pleia2: agreed, that's a great example | 16:52 |
| mordred | corvus: +100 | 16:53 |
| corvus | (from https://github.com/git/git ) | 16:53 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config: Add openstackci-tripleo-test-clouds to ansible-clouds.yaml https://review.openstack.org/347158 | 16:53 |
| *** sputnik13 has joined #openstack-infra | 16:53 | |
| fungi | clarkb: i'm done with that screen session of yours on mirror-update.o.o so feel free to kill it if you are too | 16:53 |
| mordred | corvus: I like that better than setting a description once and then ignoring it | 16:53 |
| *** Na3iL has quit IRC | 16:53 | |
| clarkb | fungi: I am, will clean it up | 16:54 |
| *** Liuqing has quit IRC | 16:54 | |
| clarkb | fungi: window zero seems to be locking on something? is that a tail? | 16:54 |
| anteaya | pleia2: is flying to india today so might be a while before she sees that suggestion | 16:55 |
| *** Liuqing has joined #openstack-infra | 16:55 | |
| fungi | clarkb: yeah, it's just a tail -f of the bandersnatch log | 16:55 |
| fungi | clarkb: sorry | 16:55 |
| *** yamahata has joined #openstack-infra | 16:55 | |
| clarkb | ok screen is gone | 16:56 |
| *** PollyZ has quit IRC | 16:56 | |
| fungi | sambetts|afk: clarkb: 348949 is interesting. how does quoting EOF fix that? are we nesting heredocs somewhere? | 16:57 |
| *** derekh has quit IRC | 16:57 | |
| fungi | the commit message is a little terse | 16:57 |
| clarkb | fungi: http://tldp.org/LDP/abs/html/here-docs.html see example 19-7 | 16:58 |
| *** piet_ has quit IRC | 16:58 | |
| clarkb | its bash magic | 16:58 |
| bkero | heredocs++ | 16:58 |
| *** roxanaghe has quit IRC | 16:58 | |
| *** cloudtrainme has quit IRC | 17:01 | |
| AJaeger | clarkb: when we rename jobs, we have to remember them in grafana as well - what do you think of https://review.openstack.org/348890 ? | 17:01 |
| *** matrohon has quit IRC | 17:02 | |
| AJaeger | wow, 297 jobs in check queue and 446 in post ;( Good that it's Friday and a weekend coming.... | 17:02 |
| pabelanger | AJaeger: we just bumped ovh-gra1 back to 79 so more nodes on the way. | 17:03 |
| *** claudiub|2 has quit IRC | 17:04 | |
| AJaeger | pabelanger: I know ;) Still a lot... | 17:05 |
| *** jordanP has quit IRC | 17:05 | |
| clarkb | AJaeger: lgtm thanks | 17:06 |
| *** skelso has joined #openstack-infra | 17:08 | |
| corvus | fungi: i'm continuing to investigate, btw. | 17:08 |
| fungi | corvus: yeah, i'm curious whether it's ultimately a gitpython "bug" or simply expected platform-specific behavior | 17:10 |
| fungi | (as in whether gitpython would accept a fix to make behavior of that call consistent) | 17:10 |
| *** bhavik has quit IRC | 17:11 | |
| *** bhavik has joined #openstack-infra | 17:11 | |
| mordred | fungi: "expected platform-specific behavior" always rankles me :) | 17:13 |
| *** tosky has quit IRC | 17:14 | |
| corvus | fungi, pabelanger: found it: https://github.com/git/git/blob/master/Documentation/RelNotes/1.9.0.txt#L18-L20 | 17:14 |
| *** skelso has quit IRC | 17:14 | |
| *** ccamacho has left #openstack-infra | 17:15 | |
| mordred | corvus: oh! that is different (and the new behavior is more inline with what my brain has always wanted that command to do) | 17:15 |
| corvus | fungi, pabelanger: that explains why i thought 'git fetch --tags' only fetched tags -- because that's what it used to do. but a couple weeks ago, we tested it and could not verify that behavior. i guess we did not test on centos7. | 17:15 |
| *** piet has joined #openstack-infra | 17:15 | |
| fungi | corvus: seems likely, yes | 17:15 |
| corvus | mordred: ya. i *thought* there was a reason we wrote it that way originally. | 17:15 |
| *** nherciu has quit IRC | 17:15 | |
| corvus | fungi, mordred: so a correction to this might be: if git <1.9 run 'git fetch && git fetch --tags' | 17:16 |
| corvus | fungi, mordred: else run 'git fetch --tags' | 17:16 |
| mordred | corvus: ++ | 17:16 |
| *** sdague has joined #openstack-infra | 17:17 | |
| *** cloudtrainme has joined #openstack-infra | 17:17 | |
| clarkb | thought the < 1.9 command should be pretty quick even on >=1.9 since it will realize it already has all the refs | 17:17 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for Nova bug 1607714 https://review.openstack.org/348812 | 17:18 |
| openstack | bug 1607714 in OpenStack Compute (nova) "test_reassign_port_between_servers failing with tap device is busy errors in neutron xenial jobs since 7/28" [High,Triaged] https://launchpad.net/bugs/1607714 - Assigned to Matt Riedemann (mriedem) | 17:18 |
| fungi | also the release history for rhel 7 indicates that 7.0 hit ga all the way back on 2014-06-09, so if they had frozen package versions for at least 4 months prior to that (perhaps around 7.0 beta which started on 2013-12-11) then git 1.8.x would have been the current series at that point | 17:18 |
| *** adrian_otto has joined #openstack-infra | 17:19 | |
| *** roxanaghe has joined #openstack-infra | 17:20 | |
| corvus | clarkb: with nova on centos7 the first fetch takes about 5.5s and the second 4.5 | 17:22 |
| *** _ari_ has joined #openstack-infra | 17:22 | |
| mmandel | If I'm getting the error "The /opt/stack/new/noVNC project was not found; if this is a gate job, add the project to the $PROJECTS variable in the job definition." Do I need to add noVNC to openstack-infra/project-config/jenkins/jobs/devstack-gate.yaml or openstack-infra/project-config/jenkins/jobs/projects.yaml? | 17:22 |
| corvus | mmandel: we don't run novnc in the gate at all | 17:23 |
| mmandel | corvus: right, I'm trying to enable it so my tempest test will get executed | 17:24 |
| mmandel | will that be a possibility? | 17:24 |
| corvus | mmandel: is there a way to install it from packages? | 17:24 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: grafana: Follow xenial renames https://review.openstack.org/348890 | 17:25 |
| AJaeger | clarkb: I missed some changes, so updated ^ | 17:25 |
| *** tlbr_ has joined #openstack-infra | 17:25 | |
| fungi | mmandel: corvus: if memory serves, we stopped relying on it because the only way to get it was to pull the source from github | 17:25 |
| fungi | and their upstream development was either dead or disinterested in releasing/packaging | 17:26 |
| corvus | fungi: yeah, that matches my recollection; i don't know if that has changed since then? | 17:26 |
| clarkb | AJaeger: commented | 17:26 |
| fungi | looks like they made a new release a month ago | 17:27 |
| clarkb | AJaeger: grenade is still weird, details in change | 17:27 |
| *** cloudtrainme has quit IRC | 17:27 | |
| AJaeger | thanks for review, clarkb | 17:27 |
| fungi | corvus: mmandel: http://packages.ubuntu.com/xenial/novnc | 17:27 |
| fungi | looks like it's a 0.4 snapshot from git circa late 2013? | 17:28 |
| *** skelso has joined #openstack-infra | 17:28 | |
| fungi | same version also available on trusty | 17:28 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config: grafana: Follow xenial renames https://review.openstack.org/348890 | 17:29 |
| fungi | precise also had a package of it (circa 2012) | 17:29 |
| AJaeger | clarkb: updated, thanks | 17:29 |
| openstackgerrit | Merged openstack-infra/project-config: Disable tripleo-test-cloud-rh1 until we have AFS mirrors https://review.openstack.org/348954 | 17:29 |
| *** ddieterly is now known as ddieterly[away] | 17:29 | |
| *** tlbr_ has quit IRC | 17:30 | |
| *** rwsu has joined #openstack-infra | 17:31 | |
| fungi | mmandel: do you happen to know the minimum novnc version needed by nova's proxy implementation? | 17:32 |
| fungi | would the 0.4 package on trusty/xenial work for your purposes? | 17:33 |
| SpamapS | harlowja: my vote for what to name a project full of errbot plugins: err2d2 | 17:33 |
| *** shardy has quit IRC | 17:33 | |
| harlowja | lol | 17:34 |
| SpamapS | runner up name: stackerr | 17:34 |
| harlowja | SpamapS nice | 17:34 |
| AJaeger | fungi, could you put 348764 and 348759 on your review queue, please? These enable constraints for venv - translation jobs seems to be the only ones that use a long lived node and need the special setup. | 17:34 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Allow Ansible role checks to collect logs https://review.openstack.org/348907 | 17:35 |
| *** jamielennox is now known as jamielennox|away | 17:35 | |
| dhellmann | fungi : tag-releases is reporting "secret key not available". I don't know if that means it knows which key to use and can't find it, or it doesn't know which key to use (I suspect the former based on the error message) http://logs.openstack.org/d3/d351fcfdbee8f9fab868c7da0a3a303b4269355a/release-post/tag-releases/a4c46b8/console.html#_2016-07-29_16_55_33_264266 | 17:35 |
| *** Liuqing has quit IRC | 17:36 | |
| *** skelso has quit IRC | 17:37 | |
| *** tkelsey has quit IRC | 17:37 | |
| AJaeger | odyssey4me: check the setup in jenkins/jobs/macros for the publisher on what it does | 17:37 |
| odyssey4me | AJaeger thanks | 17:37 |
| openstackgerrit | lokesh s proposed openstack-infra/project-config: Add py34 and py35 jobs for monasca-api and agent project https://review.openstack.org/348796 | 17:38 |
| AJaeger | odyssey4me: or do you generate different log files then those? | 17:38 |
| *** trown|lunch is now known as trown | 17:39 | |
| bkero | fungi: did you see http://www.tapwithus.com/ ? Weird chorded keyboard. | 17:39 |
| odyssey4me | AJaeger yeah, we do | 17:39 |
| mmandel | fungi: I'm checking whether the 0.4 package will suffice | 17:39 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul: Cloner: correct fetch on pre-1.9 git https://review.openstack.org/348978 | 17:40 |
| odyssey4me | AJaeger yeah, devstack-logs actually is more suitable | 17:40 |
| AJaeger | odyssey4me: then your change deserves a comment, since your job is not a devstack job... | 17:40 |
| corvus | fungi, mordred, pabelanger, EmilienM: https://review.openstack.org/348978 | 17:40 |
| odyssey4me | AJaeger we capture system info, ansible logs, etc | 17:40 |
| harlowja | think SpamapS i'm gonna go with err2d2 | 17:40 |
| odyssey4me | cool, I'll add a comment | 17:40 |
| harlowja | +1 to that, ha | 17:40 |
| AJaeger | odyssey4me: so, go back to your first version again. Sorry for wrong guidenance | 17:40 |
| EmilienM | corvus: new irc nick? | 17:40 |
| corvus | EmilienM: old one, actually :) | 17:41 |
| EmilienM | corvus: ok. thanks for the patch | 17:41 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack-infra/project-config: Allow Ansible role checks to collect logs https://review.openstack.org/348907 | 17:41 |
| odyssey4me | AJaeger thanks for the pointer and double-checking :) | 17:41 |
| odyssey4me | hopefully that's better in terms of explanation | 17:42 |
| AJaeger | odyssey4me: it is - +2. | 17:42 |
| *** nwkarsten has quit IRC | 17:42 | |
| odyssey4me | AJaeger just one question - if the logs folder is not there - what happens? | 17:42 |
| mriedem | fungi: sdague: clarkb: can we bump this to the top https://review.openstack.org/#/c/348955/ ? | 17:43 |
| mriedem | gate is way backed up because of that test failing | 17:43 |
| mriedem | basically neutron integrated gate is busted on master | 17:43 |
| AJaeger | odyssey4me: that shouldn't be a problem... | 17:44 |
| fungi | dhellmann: i'll jump on the server and see what's up. i tested signing data with gpg without specifying a private key and it worked, but didn't try git tag -s explicitly | 17:44 |
| clarkb | mriedem: ya we can promote it to top of gate | 17:44 |
| odyssey4me | AJaeger ok, I don't want to break other consumers of the job - thanks for the advice | 17:44 |
| dhellmann | fungi : ack, thanks. In the mean time I'm going to work on cleaning up some of the verbose output in the log (bash -x makes some of these echo statements redundant) | 17:44 |
| clarkb | mriedem: ok promote command is running should be at the top of the queue in a sec | 17:46 |
| mriedem | clarkb: thanks | 17:47 |
| fungi | dhellmann: yep, confirmed i can reproduce the gpg: skipped "OpenStack Release Bot <infra-root@openstack.org>": secret key not available | 17:47 |
| dhellmann | fungi: is that a .gitconfig issue? does the user have to have the key id set? | 17:47 |
| fungi | dhellmann: i'll see if i can figure out why gpg is able to use the key except when called by git | 17:48 |
| *** mrhillsman has joined #openstack-infra | 17:48 | |
| mrhillsman | nibalizer fungi | 17:48 |
| *** amitgandhinz has quit IRC | 17:48 | |
| *** rhallisey_ has joined #openstack-infra | 17:48 | |
| fungi | bkero: that's nifty. i wonder what chordset they standardized on. hard to tell from the videos | 17:49 |
| mrhillsman | nvm nibalizer | 17:49 |
| mrhillsman | fungi i pm'd you | 17:49 |
| mordred | fungi: I have gpgsign = True on my git config and I have not needed to tell git anything else about the key | 17:50 |
| *** e0ne has joined #openstack-infra | 17:50 | |
| bkero | fungi: Yeah, I haven't been able to tell at all. I've been looking around, but the details are scarce. | 17:50 |
| mordred | fungi: is user.email set for the user to infra-root@openstack.org ? | 17:50 |
| openstackgerrit | Merged openstack-infra/project-config: Move gate-{name}-chef-rake to ubuntu-xenial https://review.openstack.org/347633 | 17:50 |
| *** kien-ha has joined #openstack-infra | 17:51 | |
| *** amitgandhinz has joined #openstack-infra | 17:52 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: clean up release tool output https://review.openstack.org/348985 | 17:52 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config: add better error tracking to release_from_yaml.sh https://review.openstack.org/348986 | 17:52 |
| *** links has joined #openstack-infra | 17:54 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/jeepyb: Allow for configuring a global description https://review.openstack.org/348990 | 17:55 |
| openstackgerrit | Monty Taylor proposed openstack-infra/jeepyb: Add script to reset all descriptions to global default https://review.openstack.org/348991 | 17:55 |
| *** sandanar has quit IRC | 17:56 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config: Set a global description override https://review.openstack.org/348992 | 17:56 |
| mordred | corvus, pleia2: ^^ nerdsniped | 17:56 |
| AJaeger | mordred: woot! | 17:56 |
| mordred | AJaeger: sometimes I write code while on airplanes :) | 17:57 |
| *** cloudtrainme has joined #openstack-infra | 17:58 | |
| *** jsavak has quit IRC | 18:02 | |
| *** jsavak has joined #openstack-infra | 18:02 | |
| mmandel | fungi: Devstack and OSA install noVNC from source, but there's a note in devstack saying how that is bad. I can go ahead and try .4 and see if that works by hand, then we can decide how we'd want to move forward in the gate? | 18:02 |
| fungi | mordred: my guess is that there's something going on with how git tries to inspect the keyring prior to signing, and either it's unhappy about only the signing subkey being present _or _ there's an additional file we're missing that it cares about in ~/.gnupg | 18:02 |
| *** rbrndt has quit IRC | 18:03 | |
| fungi | mmandel: sure, sounds good | 18:03 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Created DSVM Job for NPM Projects https://review.openstack.org/348056 | 18:05 |
| krotscheck | Ahhh, the unbearable lightness of waiting for the gate. | 18:05 |
| mordred | krotscheck: you should write that novel | 18:06 |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Add initial err2d2 project config https://review.openstack.org/348998 | 18:06 |
| *** EricGonc_ has joined #openstack-infra | 18:07 | |
| *** cloudtrainme has quit IRC | 18:07 | |
| krotscheck | mordred: It's a story about the intellectual life of a hungarian engineer, her boyfriend, their cat, in the modern revival of Budapest. | 18:07 |
| *** piet has quit IRC | 18:09 | |
| *** EricGonczer_ has quit IRC | 18:09 | |
| *** EricGonczer_ has joined #openstack-infra | 18:12 | |
| *** PollyZ has joined #openstack-infra | 18:12 | |
| *** skelso has joined #openstack-infra | 18:12 | |
| ihrachys | neutron eagerly ask for a second +2 for grafana: https://review.openstack.org/#/c/346953/ | 18:12 |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Add initial err2d2 project config https://review.openstack.org/348998 | 18:14 |
| *** mtanino has joined #openstack-infra | 18:14 | |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Add initial err2d2 project config https://review.openstack.org/348998 | 18:14 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/project-config: Swap neutron/nova-net multinode full jobs in nova check queue https://review.openstack.org/349002 | 18:14 |
| *** inc0 has quit IRC | 18:15 | |
| *** inc0 has joined #openstack-infra | 18:15 | |
| *** EricGonc_ has quit IRC | 18:16 | |
| corvus | harlowja: what's your plan for dealing with errbot plugin configuration? | 18:17 |
| harlowja | soooooo corvus the things i've learned is that plugin configuration in errbot is sorta differnt than what u normally do : | 18:17 |
| corvus | harlowja: last i looked, it was all done via irc commands rather than config files, which would make it hard to review changes to bot configuration if we went with the native approach. | 18:17 |
| *** piet has joined #openstack-infra | 18:17 | |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config: networking-midonet: switch to python-db-jobs https://review.openstack.org/335551 | 18:17 |
| harlowja | corvus right, its all command based | 18:18 |
| corvus | harlowja: that is pretty much the only reason i have not already done this | 18:18 |
| corvus | harlowja, fungi: are we no longer interested in controlling this stuff using git reviewed config files? | 18:18 |
| harlowja | but i don't see why we couldn't make the defaults (like @ https://github.com/harlowja/gerritbot2/blob/master/plugins/gerritbot/gerritbot.py#L307 ) just default to loading from a file | 18:18 |
| harlowja | and then defaults come from a config-file | 18:19 |
| harlowja | and if u want to override crap, thats ok to | 18:19 |
| mordred | anybody remember which project the angry respose to the pull request closer was on? | 18:19 |
| corvus | harlowja: yeah, the best i could come up with was a set of plugins that just didn't behave like normal errbot plugins. | 18:19 |
| harlowja | corvus or i'll go fight more with the errbot people about this, ha | 18:19 |
| corvus | harlowja: and read their config from files instead | 18:19 |
| fungi | corvus: harlowja: if configuration can be done in configuration and we can configure it to ignore config attempts from irc users, then i'd be fine with that as a solution | 18:19 |
| *** cloudtrainme has joined #openstack-infra | 18:20 | |
| harlowja | fungi afaik ya, admins can only do this kind of config anyway | 18:20 |
| harlowja | and admins are special people | 18:20 |
| fungi | i certainly am | 18:20 |
| harlowja | there is a 'acl' like system | 18:20 |
| corvus | harlowja: yeah, but i think we don't want our admins doing *anything* with errbot over irc. | 18:20 |
| corvus | our collective wisdom is in git history | 18:20 |
| harlowja | corvus https://github.com/errbotio/errbot/issues/799 was one i created, ha | 18:20 |
| corvus | and also, we don't scale well enough to handle manual requests like this | 18:21 |
| harlowja | corvus well some things are useful to do with the bots over irc | 18:21 |
| mordred | yah. we NEVER ever want to set config via irc | 18:21 |
| mordred | harlowja: we never want to do those things | 18:21 |
| mordred | no matter how useful they might be | 18:21 |
| harlowja | fair nuff | 18:21 |
| mordred | we're grump greybeards who like config files | 18:22 |
| harlowja | things like !restart plugin though are useful | 18:22 |
| harlowja | but those are more admin functionality, not config | 18:22 |
| mordred | yah | 18:22 |
| *** inc0_ has joined #openstack-infra | 18:22 | |
| clarkb | mordred: did you find that pull request? I can probably dig it out of my email | 18:22 |
| harlowja | but i think we can just use http://errbot.io/en/latest/user_guide/provisioning.html corvus to use stored configs or whatever | 18:23 |
| *** thiagop has quit IRC | 18:23 | |
| harlowja | that seems to be a <supported-way> | 18:23 |
| mordred | clarkb: I didn't. I thought it was in nova, but I can't remember | 18:23 |
| mordred | clarkb: it's not super important though - if you don't know off the top of your head, it's not worht digging | 18:23 |
| fungi | dhellmann: mordred: i've pinpointed the issue. gnupg configures the key description as "Full Name (Comment) <e-mail@address>" and git looks for a key matching "$user.name <$user.email>" as specified in your .gitconfig | 18:24 |
| corvus | harlowja: oh interesting -- so store the config in git, and when it's update, 'errbot --storage-merge' it? | 18:24 |
| mordred | harlowja: ooh, that does look potentially useful | 18:24 |
| mordred | corvus: ++ | 18:24 |
| harlowja | corvus possibly | 18:24 |
| harlowja | https://gist.github.com/harlowja/0741c41094c3b7cee08ceaed41b3e607 (is an example of some of the admin plugins) | 18:24 |
| harlowja | ignore <OsloBotPlugin> in that, ha | 18:24 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Use delorean-current for puppet modules https://review.openstack.org/341429 | 18:24 |
| fungi | dhellmann: mordred: now it's just a matter of playing with alternative solutions to avoid us having to update the ~jenkins/.gitconfig parameters for signing01.ci.o.o every time we rotate the keys and update the comment field | 18:24 |
| corvus | harlowja: yeah, the storage-merge approach sounds promising | 18:25 |
| harlowja | ya, storage-merge, then reload plugin | 18:25 |
| harlowja | and then plugin reads from <some place in storage> | 18:25 |
| harlowja | and then is now newly configured | 18:25 |
| corvus | harlowja: can you reload the plugin from the command line? | 18:25 |
| mordred | fungi: or, maybe updating the gitconfig stuff iseasier? :) | 18:25 |
| dhellmann | fungi : I have "signingkey = C428437A" in my .gitconfig | 18:25 |
| harlowja | corvus unsure | 18:25 |
| harlowja | i know u can do it via IRC | 18:26 |
| *** amitgandhinz has quit IRC | 18:26 | |
| dhellmann | fungi : that's under the "[user]" section | 18:26 |
| harlowja | `!plugin reload - reload a plugin: reload the code of the plugin leaving the activation status ...` | 18:26 |
| harlowja | via that command, ha | 18:26 |
| dhellmann | fungi : it's likely I had the same issue, because my "name" does not match the one in my key either | 18:26 |
| fungi | dhellmann: right, that will change every cycle, so i'm seeing if there's an easy way to use a different way of specifying the key | 18:26 |
| *** inc0 has quit IRC | 18:26 | |
| dhellmann | ah | 18:26 |
| *** amitgandhinz has joined #openstack-infra | 18:26 | |
| fungi | dhellmann: without having to update puppet at the same time | 18:26 |
| fungi | i have some ideas, just need to try them out for a few minutes | 18:27 |
| dhellmann | could puppet update the file from a template when a new key is generated? | 18:27 |
| anteaya | harlowja: can you set permissions on that reload command? | 18:27 |
| dhellmann | or pushed out, I guess | 18:27 |
| clarkb | mordred: gitdm | 18:27 |
| anteaya | if meetbot restarts during a meeting we lose some of that meeting | 18:27 |
| fungi | dhellmann: it could, or we could feed it from hiera since we update that with the key material when we rotate it | 18:27 |
| *** electrofelix has quit IRC | 18:28 | |
| dhellmann | fungi : sure | 18:28 |
| fungi | dhellmann: looks like just specifying the same thing in user.email and user.signingkey works too | 18:28 |
| fungi | dhellmann: so i'll go with that since it's an easy template update | 18:29 |
| dhellmann | ++ | 18:29 |
| harlowja | anteaya i do believe so, i haven't explored the acl stuff to much | 18:29 |
| anteaya | harlowja: okay thanks, we would need permissions on various admin type commands | 18:29 |
| corvus | anteaya: as mordred said, we don't want to issue them at all | 18:29 |
| fungi | dhellmann: our puppet manifest managing the .gnupg/* files already completely replaces the entire keychain, so there's no risk that it'll try to use the wrong key matching that e-mail address as there's only one | 18:30 |
| anteaya | corvus: oh sorry, I missed him saying that | 18:30 |
| anteaya | so not to worry then harlowja | 18:30 |
| fungi | yeah, we would want to "admin" the bots by updating on-disk configuration (applied through configuration management) | 18:30 |
| dhellmann | fungi : makes sense | 18:31 |
| harlowja | anteaya i got another guy from GD who has explored all the acl stuff, let me get him in here | 18:31 |
| fungi | dhellmann: it's about a 5-minute change | 18:31 |
| fungi | writing it now | 18:31 |
| *** john-davidge has quit IRC | 18:32 | |
| dhellmann | fungi : at this point the wait in the check queue before I can land another release test is a couple of hours. Which reminds me, at one point corvus said something about eliminating the requirement that we have a jenkins+1 before allowing a patch to go into the gate for the openstack/releasees repo. Is that something I can adjust in project-config? | 18:32 |
| fungi | dhellmann: right now it's per-pipeline configuration, not per-project | 18:33 |
| harlowja | fungi sure, we may have to do some stuff in errbot to SIGHUP handle and stuff, but probably nothing crazy | 18:33 |
| *** jsavak has quit IRC | 18:33 | |
| mordred | dhellmann: zuul v3 should alow it to be per-project | 18:33 |
| openstackgerrit | Merged openstack-infra/system-config: Document signing key management https://review.openstack.org/339248 | 18:33 |
| fungi | harlowja: or a small cli utility to rpc with it if it has a separate admin socket, a la eggdrop | 18:33 |
| corvus | mordred: no it's still per-pipeline in v3 | 18:33 |
| *** tkelsey has joined #openstack-infra | 18:33 | |
| *** jsavak has joined #openstack-infra | 18:33 | |
| harlowja | fungi sure | 18:33 |
| corvus | mordred: (but pipelines are per-tenant) | 18:33 |
| mordred | corvus: ah - yeah. right. that was what I was thinking | 18:34 |
| dhellmann | corvus : I think this came up in the context of my patch to define a new release-check pipeline with a different priority, so you're saying instead of a new priority I could just change a different setting there to not require a jenkins+1? | 18:34 |
| corvus | perhaps this is something that could be made per-project; i don't think we've discussed it though. | 18:34 |
| *** thorst has quit IRC | 18:34 | |
| *** stevemcquaid has joined #openstack-infra | 18:35 | |
| stevemcquaid | hello | 18:35 |
| *** kushal has quit IRC | 18:35 | |
| harlowja | stevemcquaid welcome | 18:35 |
| stevemcquaid | how can I be of service | 18:35 |
| corvus | dhellmann: i wish that we could remove that requirement from the gate pipeline, but it's strongly desired by some folks in the openstack project. | 18:35 |
| dhellmann | corvus : oh, I would need a new gate pipeline, not a new check pipeline? | 18:36 |
| corvus | dhellmann: yeah, it's a requirement on the gate pipeline | 18:36 |
| dmsimard | harlowja: errbot++ | 18:37 |
| harlowja | ya, i'm thinking that some of the way it configures stuff we may have to work with the community a little | 18:37 |
| harlowja | stevemcquaid has done some bots and has gone the config route | 18:37 |
| *** ociuhandu has quit IRC | 18:37 | |
| *** thorst has joined #openstack-infra | 18:37 | |
| * mordred waves at stevemcquaid | 18:37 | |
| harlowja | so he knows some things about errbot (more than i do, ha) | 18:38 |
| corvus | dhellmann: if you create a high-priority check pipeline, i'd like to move infra to it :) | 18:38 |
| *** tkelsey has quit IRC | 18:38 | |
| dmsimard | harlowja: the errbot upstream (gbin) is a googler and pretty reactive/helpful | 18:38 |
| harlowja | dmsimard yup | 18:38 |
| dhellmann | corvus : https://review.openstack.org/#/c/324768/ ;-) | 18:38 |
| harlowja | dmsimard i'm unsure how attached (he?) is to the IRC configuration mechanism though | 18:38 |
| harlowja | we'll have to explore that a little, see where we can take it | 18:38 |
| stevemcquaid | yeah from what ive seen - basically the in-chat config evolved because gbin orginally built a python script that did everything. Over time it was broken out into full framework. | 18:38 |
| dmsimard | harlowja: not sure what you mean by that | 18:39 |
| dhellmann | corvus: I would like to figure out some way to make it so we don't have to wait 2+ hours to tag a release that does not involve us manually tagging the release. I am open to all suggestions. | 18:39 |
| stevemcquaid | In my deployments of it, I just did all configs via static files via ansible - pretty easy | 18:39 |
| EmilienM | fungi: I have an outstanding request, tripleo CI is currently broken because we need 348816 which is currently in check pipeline waiting for a node to run puppet lint tests... could we promote it to run soon please? or move it to the gate directly? | 18:39 |
| dmsimard | harlowja: is there anything wrong with the irc backend ? | 18:39 |
| *** deadnull has quit IRC | 18:39 | |
| mordred | dmsimard, stevemcquaid: basically we want to be able to lay down config files with our config management | 18:39 |
| corvus | dhellmann: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n44 is the config in question | 18:39 |
| harlowja | dmsimard no no, sorry, the configuration mechanism is sorta different for plugins | 18:39 |
| stevemcquaid | yup not a problem | 18:39 |
| mordred | woot | 18:39 |
| * mordred loves when things aren't a problem | 18:39 | |
| harlowja | nothing wrong with the IRC backend, the configuration mechanism is just a little different than what people think | 18:39 |
| *** dkehn has quit IRC | 18:39 | |
| stevemcquaid | and to assure nothing changes - you can have setup ACL so no one can modify it from in chat | 18:39 |
| *** karthikp_ has joined #openstack-infra | 18:39 | |
| mordred | \o/ | 18:40 |
| dmsimard | mordred: I already sort of do that with ansible https://github.com/rdo-infra/ansible-role-rdobot | 18:40 |
| dmsimard | mordred: but it's very limited mind you | 18:40 |
| *** dkehn_ has quit IRC | 18:40 | |
| dhellmann | corvus : what would the effect be if I created a separate gate pipeline? is that even the right way for me to be thinking of it? | 18:40 |
| stevemcquaid | example plugin/module to errbot that I built: https://github.com/stevemcquaid/err-openstack/blob/master/gdopenstack.py | 18:40 |
| harlowja | stevemcquaid how do u trigger when a config is changed, do u just then reload the plugin? | 18:40 |
| corvus | stevemcquaid, harlowja: an example configuration of a current bot we have: http://git.openstack.org/cgit/openstack-infra/project-config/tree/gerritbot/channels.yaml | 18:40 |
| harlowja | corvus yup, that huge list | 18:40 |
| stevemcquaid | and deployment scripts to config everything for you: https://github.com/stevemcquaid/errbot-deploy | 18:40 |
| corvus | dhellmann: i think it would be potentially problematic for openstack/releases not to share a gate queue with openstack itself | 18:41 |
| stevemcquaid | harlowja - just reload the bot via systemd | 18:41 |
| harlowja | that kills the whole bot though right? | 18:41 |
| corvus | dhellmann: (depends-on will not work as well, for one) | 18:41 |
| harlowja | vs just reloading a single plugin? | 18:41 |
| stevemcquaid | yeah | 18:42 |
| *** thorst has quit IRC | 18:42 | |
| stevemcquaid | but - depending upon how the plugin gets its configs you could always just call the config loader in the plugin | 18:42 |
| dhellmann | corvus : we already have validation logic in place that requires that the patch being tagged is actually merged into the repo (not just available on the local cache) | 18:42 |
| stevemcquaid | im assuming we are talking about bot configs - aka ACL change or cred update | 18:42 |
| corvus | dhellmann: so if the choice is between a new gate vs check pipeline, i would lean toward gate. | 18:42 |
| mordred | corvus: importing zuul.manager.dependent.DependentPipelineManager not so much with working - there is an import loop (this is me trying to reference it in Sphinx) | 18:43 |
| dmsimard | stevemcquaid, harlowja: there's some mechanisms with !commands that can be used to reload/restart things too but I've never really used them | 18:43 |
| stevemcquaid | but if you want just a plugin config to be modified, yeah have a restricted method that will update your class vars in memory from reading a config file and call it when you want | 18:43 |
| corvus | dhellmann: well, if there really is so little relation between openstack/releases and openstack, maybe it'd be okay. | 18:43 |
| mordred | corvus: http://paste.openstack.org/show/543946/ | 18:43 |
| *** jcoufal has quit IRC | 18:43 | |
| corvus | mordred: might be time to move that out of __init__? | 18:44 |
| *** jcoufal has joined #openstack-infra | 18:44 | |
| mordred | corvus: I tried that - but it actually seems to be tied to configloader | 18:44 |
| dhellmann | corvus : maybe? it sounds like there is a lot of potential for unintended consequences, and that makes me nervous that I don't fully understand the ramifications. | 18:44 |
| mordred | corvus: I'mma see if I can break it | 18:45 |
| *** jed56 has quit IRC | 18:45 | |
| harlowja | dmsimard ya, so those are via admin private messages, which i think isn't to much desired, but a ability to reload (without shutting off) via SIGHUP seems possible, or via a local process admin socket as fungi was mentioning could be doable | 18:45 |
| dmsimard | harlowja: /me nods | 18:46 |
| *** yamahata has quit IRC | 18:46 | |
| *** nwkarsten has joined #openstack-infra | 18:46 | |
| *** dkehn has joined #openstack-infra | 18:46 | |
| *** skelso has quit IRC | 18:47 | |
| harlowja | a mini admin socket thing should be pretty easy i would think, and since errbot already has a local <backend> (that is like a REPL) its probably just simialr to that | 18:47 |
| *** sdake has quit IRC | 18:47 | |
| mordred | corvus: oh. it was easy. Ithought it was going to be harder and worthy of discussion on approach | 18:48 |
| corvus | mordred: yay | 18:49 |
| * corvus lunches | 18:49 | |
| AJaeger | fungi, harlowja , corvus commented on https://review.openstack.org/#/c/348998 to discuss whether err2d2 should belong to infra... | 18:50 |
| *** ddieterly[away] is now known as ddieterly | 18:50 | |
| anteaya | I think it should belong to infra | 18:50 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/puppet-jenkins: Add user.signingkey to .gitconfig https://review.openstack.org/349022 | 18:50 |
| harlowja | i'm fine with that | 18:50 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config: Set the gitpgpkey for signing nodes https://review.openstack.org/349023 | 18:51 |
| fungi | dhellmann: mordred: ^ | 18:51 |
| anteaya | the review core team can be expanded, but I can't see a better home for it | 18:51 |
| harlowja | k | 18:51 |
| mordred | dhellmann: I has sphinx question ... if I want to reference a class but have the link text be something different fomr the class name ... | 18:51 |
| mordred | dhellmann: I thought it should look like this: :py:class:`Manager <zuul.manager.BasePipelineManager>` | 18:51 |
| mordred | dhellmann: but that does not wind up linking to the anchor for zuul.manager.BasePipelineManager | 18:52 |
| dhellmann | mordred : hmm, yes, that's what I would have said, too. what does the link point to? | 18:52 |
| mordred | dhellmann: nothing. it just does a bolded word. (the word is Manager, so that's right) | 18:52 |
| dhellmann | I know you can do :py:class;`~zuul.manager.BasePipelineManager` to strip the link title down to just the class name | 18:52 |
| *** dkehn_ has joined #openstack-infra | 18:52 | |
| *** skelso has joined #openstack-infra | 18:52 | |
| mordred | oh - wait | 18:52 |
| mordred | I think I see a typo | 18:52 |
| dhellmann | oh, you'll get a bolded word if there's no such class | 18:53 |
| EmilienM | can someone from infra core please help me to bring TripleO CI back, we would need 348816 merged asap, could we promote it in zuul please? | 18:53 |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Add initial err2d2 project config https://review.openstack.org/348998 | 18:53 |
| *** achanda has joined #openstack-infra | 18:55 | |
| harlowja | i'll move over my gerritbot2 stuff --> that once this happens | 18:55 |
| harlowja | then can start to figure out how to more directly map that ontop of the existing config and <...> | 18:55 |
| harlowja | it will be a good simpler plugin | 18:56 |
| fungi | EmilienM: is it just affecting tripleo or is it affecting other projects running those jobs? | 18:56 |
| *** thorst has joined #openstack-infra | 18:56 | |
| EmilienM | fungi: just tripleoe | 18:56 |
| *** mctaylor has joined #openstack-infra | 18:58 | |
| fungi | EmilienM: normally when projects need to fix themselves we just let them wait in line like everyone else. we try to avoid stepping in unless it's impacting other projects too (either causing lots of gate resets for other projects sharing the same queue or blocking changes for other projects because the rely on the same broken job) | 18:58 |
| EmilienM | fungi: having xenial nodes take more than 3 jours | 18:58 |
| EmilienM | fungi: ok I understand, so we'll wait | 18:58 |
| EmilienM | 3 hours* not jours | 18:58 |
| *** adrian_otto has quit IRC | 18:59 | |
| *** rbrndt has joined #openstack-infra | 18:59 | |
| fungi | weird, there's an "integrated" queue in the check pipeline according to zuul status. it looks like maybe someone pushed a long patch series for nova | 18:59 |
| *** tlbr has joined #openstack-infra | 19:00 | |
| fungi | oh, i guess that's normally there | 19:00 |
| *** tlbr is now known as Guest84124 | 19:00 | |
| *** rfolco has quit IRC | 19:00 | |
| fungi | more that i'm seeing periodic rendering/update problems that make it look like some independent queues have no active changes only dependent ones | 19:00 |
| *** thorst has quit IRC | 19:00 | |
| *** thorst has joined #openstack-infra | 19:01 | |
| fungi | yeah, that's a 32-change-long series | 19:01 |
| fungi | and it's not superdan for a change! | 19:02 |
| *** PollyZ has quit IRC | 19:02 | |
| fungi | okay, i need to head out and run some errands before rush hour. bbiaw | 19:03 |
| *** thorst_ has joined #openstack-infra | 19:03 | |
| *** Guest84124 has quit IRC | 19:04 | |
| *** ihrachys has quit IRC | 19:04 | |
| *** elo has quit IRC | 19:06 | |
| *** yamamoto has quit IRC | 19:06 | |
| *** thorst has quit IRC | 19:06 | |
| anteaya | ha ha ha | 19:07 |
| *** fifieldt has quit IRC | 19:07 | |
| *** elo has joined #openstack-infra | 19:07 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul: Add an internals doc to the documentation https://review.openstack.org/349035 | 19:09 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul: Rename BasePipelineManager to PipelineManager https://review.openstack.org/349036 | 19:09 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul: Move code out of zuul/manager/__init__.py https://review.openstack.org/349037 | 19:09 |
| mordred | corvus: ^^ not done by a longshot - but wanted to put up what I had to make sure structure/approach looked ok before I got too much further | 19:10 |
| *** roxanaghe has quit IRC | 19:10 | |
| *** bnemec is now known as beekneemech | 19:12 | |
| *** skelso has quit IRC | 19:12 | |
| *** bhavik has quit IRC | 19:14 | |
| *** cloudtrainme has quit IRC | 19:17 | |
| *** fifieldt has joined #openstack-infra | 19:19 | |
| openstackgerrit | Tony Breeds proposed openstack-infra/project-config: Cross test constraints and project unittests. https://review.openstack.org/308739 | 19:19 |
| *** dkehn_ has quit IRC | 19:19 | |
| *** links has quit IRC | 19:20 | |
| tonyb | clarkb: ^^ That's my first pass. No idea how close it is to correct. | 19:20 |
| *** dkehn has quit IRC | 19:20 | |
| tonyb | AJaeger: I'd appreciate your review also | 19:20 |
| *** piet has quit IRC | 19:21 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement a new job for undercloud testing only https://review.openstack.org/346147 | 19:21 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: Implement a new job for undercloud testing only https://review.openstack.org/346147 | 19:22 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 19:22 |
| *** dkehn has joined #openstack-infra | 19:22 | |
| *** raildo has quit IRC | 19:23 | |
| *** PollyZ has joined #openstack-infra | 19:24 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update Git plugin to use convert xml https://review.openstack.org/349045 | 19:25 |
| harlowja | fungi dmsimard stevemcquaid corvus https://github.com/errbotio/errbot/issues/824 (dumped some ideas on that) | 19:25 |
| harlowja | feel free to add more | 19:25 |
| harlowja | the way reloads and such happen, might need a little massaging, but that shouldn't hinder really writing the code for the plugins | 19:26 |
| harlowja | not anything crazy, but prob some work | 19:27 |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Update Git plugin to use convert xml https://review.openstack.org/349045 | 19:27 |
| *** rwsu has quit IRC | 19:27 | |
| *** dkehn has quit IRC | 19:28 | |
| *** dkehn_ has joined #openstack-infra | 19:32 | |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Add initial err2d2 project config https://review.openstack.org/348998 | 19:33 |
| harlowja | fungi https://review.openstack.org/#/c/349046/ | 19:33 |
| harlowja | i guess u'll need to +1 that | 19:33 |
| *** skelso has joined #openstack-infra | 19:33 | |
| *** dkehn has joined #openstack-infra | 19:34 | |
| * mordred watches harlowja do all th ethings | 19:34 | |
| harlowja | ya, the lovely steps of pain | 19:34 |
| *** sdake has joined #openstack-infra | 19:35 | |
| *** edmondsw has quit IRC | 19:35 | |
| openstackgerrit | Kien Ha proposed openstack-infra/jenkins-job-builder: Remove handle_entity_children for convert xml https://review.openstack.org/349045 | 19:35 |
| harlowja | btw, now that infra doesn't use jenkins, who are the main folks that still reviewing jjb stuff? | 19:39 |
| harlowja | i was gonna start switching more of the internal jenkisn stuff to jjb yaml (so that it can be peer reviewed and ...) and just want to make sure its not a blackhole, lol | 19:39 |
| harlowja | (or only a small blackhole) | 19:39 |
| *** skelso has quit IRC | 19:39 | |
| openstackgerrit | Patrick East proposed openstack-infra/devstack-gate: Copy local.conf to the archived logs directory https://review.openstack.org/349053 | 19:41 |
| *** tlbr has joined #openstack-infra | 19:42 | |
| *** rhallisey_ is now known as rhallisey | 19:42 | |
| *** tlbr is now known as Guest79100 | 19:42 | |
| *** yamahata has joined #openstack-infra | 19:43 | |
| dmsimard | harlowja: pretty sure jjb is all but abandoned | 19:46 |
| dmsimard | harlowja: there's people working on v2 pretty regularly | 19:46 |
| *** hasharAway is now known as hashar | 19:46 | |
| dmsimard | afaik | 19:46 |
| harlowja | jjbv2 | 19:47 |
| harlowja | ? | 19:47 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul: Add an internals doc to the documentation https://review.openstack.org/349035 | 19:48 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul: Rename BasePipelineManager to PipelineManager https://review.openstack.org/349036 | 19:48 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul: Move code out of zuul/manager/__init__.py https://review.openstack.org/349037 | 19:48 |
| harlowja | or is the v2 stuff u mean the ansible stuffs? | 19:48 |
| harlowja | https://review.openstack.org/#/q/project:openstack-infra/jenkins-job-builder seems pretty active? | 19:48 |
| dmsimard | harlowja: v2 of jjb, not related to ansible at all | 19:49 |
| harlowja | ah | 19:49 |
| harlowja | where's that at, ha | 19:49 |
| dmsimard | well you can tell from the reviews | 19:49 |
| dmsimard | there's a lot of activity and there is a v2 topic somewhere | 19:49 |
| harlowja | https://review.openstack.org/#/q/status:open+project:openstack-infra/jenkins-job-builder+branch:master+topic:jjb-2.0.0-api | 19:49 |
| harlowja | ah | 19:49 |
| dmsimard | What organization is behind cirros ? cirros-cloud.net just redirects to launchpad | 19:50 |
| harlowja | the scott moser organization | 19:50 |
| harlowja | lol | 19:50 |
| dmsimard | It's not uncommon to get HTTP 400's | 19:50 |
| harlowja | apparently there is a #cirros channel | 19:50 |
| harlowja | i haven't been in it :-P | 19:50 |
| *** Guest79100 has quit IRC | 19:51 | |
| *** krtaylor has quit IRC | 19:52 | |
| *** raildo has joined #openstack-infra | 19:52 | |
| *** gordc has quit IRC | 19:56 | |
| dmsimard | cirros is 13MB, damn, never realized | 19:57 |
| *** skelso has joined #openstack-infra | 19:58 | |
| dmsimard | I knew it was small but that small is somthing | 19:58 |
| *** asettle has joined #openstack-infra | 19:58 | |
| *** sdake has quit IRC | 19:59 | |
| *** kien-ha has quit IRC | 19:59 | |
| *** rossella_s has quit IRC | 19:59 | |
| *** _ari_ has quit IRC | 20:00 | |
| *** rossella_s has joined #openstack-infra | 20:00 | |
| *** ddieterly is now known as ddieterly[away] | 20:01 | |
| *** ddieterly[away] is now known as ddieterly | 20:01 | |
| *** ddieterly is now known as ddieterly[away] | 20:01 | |
| *** sarob has joined #openstack-infra | 20:02 | |
| *** asettle has quit IRC | 20:03 | |
| *** sarob has quit IRC | 20:03 | |
| *** sarob has joined #openstack-infra | 20:03 | |
| *** ddieterly[away] is now known as ddieterly | 20:04 | |
| *** e0ne has quit IRC | 20:04 | |
| *** roxanaghe has joined #openstack-infra | 20:05 | |
| *** ociuhandu has joined #openstack-infra | 20:07 | |
| *** yamamoto has joined #openstack-infra | 20:07 | |
| *** baoli has quit IRC | 20:11 | |
| EmilienM | fungi: thanks | 20:11 |
| *** adrian_otto has joined #openstack-infra | 20:11 | |
| *** yamamoto has quit IRC | 20:12 | |
| *** rwsu has joined #openstack-infra | 20:18 | |
| *** rbuzatu has joined #openstack-infra | 20:19 | |
| clarkb | bkero: for when you return to a computer https://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n1122 is the base rule to make -nv non voting and is one reason for the {suffix} stuff | 20:19 |
| clarkb | bkero: and just below that you can see where we have rules for the ubuntu-trusty/ubuntu-xenial switch | 20:19 |
| *** mriedem has quit IRC | 20:20 | |
| *** rbuzatu has quit IRC | 20:21 | |
| *** coolsvap has quit IRC | 20:21 | |
| openstackgerrit | Tony Breeds proposed openstack-infra/project-config: Cross test constraints and project unittests. https://review.openstack.org/308739 | 20:22 |
| *** kien-ha has joined #openstack-infra | 20:23 | |
| *** kzaitsev_mb has joined #openstack-infra | 20:27 | |
| *** tonytan_brb has quit IRC | 20:31 | |
| *** kzaitsev_mb has quit IRC | 20:33 | |
| *** achanda has quit IRC | 20:34 | |
| clarkb | tonyb: yup that change looks good | 20:34 |
| *** tkelsey has joined #openstack-infra | 20:34 | |
| prometheanfire | nice | 20:36 |
| *** burgerk has quit IRC | 20:37 | |
| *** tkelsey has quit IRC | 20:39 | |
| *** gouthamr has quit IRC | 20:41 | |
| *** kzaitsev_mb has joined #openstack-infra | 20:42 | |
| pabelanger | clarkb: looks like were getting No valid host was found back from ovh-gra1 in some cases now | 20:42 |
| *** akshai has quit IRC | 20:42 | |
| clarkb | pabelanger: I think that means the nova scheduler isn't finding places to put instances | 20:42 |
| clarkb | pabelanger: which is why I wanted to ask before we used our full quota. Hopefully they can clarify what the actual availability is there | 20:43 |
| pabelanger | clarkb: I would agree with that | 20:45 |
| sdague | clarkb: didn't ovh put the infra nodes in a dedicated aggregate? maybe it's sized wrong for the quota | 20:45 |
| pabelanger | but looks like we got up to abut 65 nodes | 20:45 |
| clarkb | sdague: ya we have dedicated hardware with our own flavor. Our quota is for 200 instances, we are set to use 79, we had been using 40 | 20:45 |
| pabelanger | actually, 75 | 20:45 |
| clarkb | I sent mail earlier today asking them to clarify if we can use up to that quota or not because I don't actually know what the underlying hardware will support | 20:46 |
| *** EricGonczer_ has quit IRC | 20:50 | |
| clarkb | (also asked about bhs1 so hopeflly we get both regions sorted out) | 20:51 |
| *** markusry has quit IRC | 20:52 | |
| pabelanger | clarkb: we also have a node or 2 that failed to ping git.openstack.org in ovh-bhs1. Going to see if I can troubleshoot that | 20:52 |
| clarkb | ok, I wonder if its related to the ord problems | 20:54 |
| *** sarob has quit IRC | 20:55 | |
| pabelanger | ya, maybe | 20:58 |
| *** sxmxc has joined #openstack-infra | 20:59 | |
| *** trown is now known as trown|outtypewww | 20:59 | |
| *** kzaitsev_mb has quit IRC | 20:59 | |
| *** hashar has quit IRC | 21:00 | |
| *** jsavak has quit IRC | 21:01 | |
| *** raildo has quit IRC | 21:01 | |
| *** jsavak has joined #openstack-infra | 21:01 | |
| *** ddieterly is now known as ddieterly[away] | 21:02 | |
| *** ddieterly[away] is now known as ddieterly | 21:03 | |
| *** jkilpatr has quit IRC | 21:03 | |
| *** rfolco has joined #openstack-infra | 21:04 | |
| *** rfolco has quit IRC | 21:05 | |
| *** annegentle has joined #openstack-infra | 21:05 | |
| *** inc0_ has quit IRC | 21:07 | |
| openstackgerrit | Merged openstack-infra/project-config: Prevent early variable substitution in Ironic extra vars https://review.openstack.org/348949 | 21:08 |
| *** yamamoto has joined #openstack-infra | 21:08 | |
| *** esikachev has joined #openstack-infra | 21:08 | |
| *** siva_krish1 has quit IRC | 21:10 | |
| *** esikachev has quit IRC | 21:12 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Switch to gate-{name}-linters-{node} https://review.openstack.org/347939 | 21:12 |
| *** yamamoto has quit IRC | 21:13 | |
| *** elo has quit IRC | 21:14 | |
| *** piet has joined #openstack-infra | 21:14 | |
| *** ldnunes has quit IRC | 21:14 | |
| *** esikachev has joined #openstack-infra | 21:15 | |
| bkero | clarkb: Oh nice! | 21:16 |
| *** kien-ha has quit IRC | 21:16 | |
| *** matt-borland has quit IRC | 21:17 | |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Don't try to get stack details if the stack doesn't exist https://review.openstack.org/319337 | 21:17 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: TEST: Delete the overcloud when finished https://review.openstack.org/297328 | 21:17 |
| openstackgerrit | Ben Nemec proposed openstack-infra/tripleo-ci: Test overcloud deletion in periodic job https://review.openstack.org/296765 | 21:17 |
| openstackgerrit | Merged openstack-infra/project-config: clean up release tool output https://review.openstack.org/348985 | 21:18 |
| *** EricGonczer_ has joined #openstack-infra | 21:19 | |
| *** elo has joined #openstack-infra | 21:22 | |
| *** esikachev has quit IRC | 21:23 | |
| *** armax has quit IRC | 21:26 | |
| *** skelso has quit IRC | 21:27 | |
| *** elo has quit IRC | 21:27 | |
| krotscheck | mordred: Off the top of your head, is there a devstack ENV parameter that points at /etc/openstack/clouds.yaml? | 21:28 |
| clarkb | krotscheck: looks like they use CLOUD_YAML internally but its not exported | 21:29 |
| clarkb | but its to a known location (/etc/openstack/clouds.yaml) so probably not a big deal | 21:29 |
| krotscheck | clarkb: Yeah, we're trying to figure out how to pass that path through to an npm script in a way that doesn't reinvent the wheel for special unicorn javascript. | 21:30 |
| krotscheck | If CLOUD_YAML's exported, we could just use that. | 21:31 |
| clarkb | I think you just know thats where the file is located | 21:31 |
| clarkb | it shouldn't go anywhere | 21:31 |
| *** tonytan4ever has joined #openstack-infra | 21:31 | |
| krotscheck | Yeah, meh. We'll just define it ourselves. | 21:32 |
| krotscheck | But in case they export it in the future we'll name it something else. | 21:32 |
| *** Guest97891 has joined #openstack-infra | 21:32 | |
| *** ddieterly is now known as ddieterly[away] | 21:32 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci: WIP - Implement undercloud upgrade job - Mitaka -> Newton https://review.openstack.org/346995 | 21:33 |
| openstackgerrit | Matt Borland proposed openstack-infra/project-config: WIP: Splitting current/deprecated integ. test https://review.openstack.org/349077 | 21:36 |
| *** tonytan4ever has quit IRC | 21:36 | |
| *** amitgandhinz has quit IRC | 21:37 | |
| *** annegentle has quit IRC | 21:38 | |
| *** aysyd has quit IRC | 21:38 | |
| *** Guest97891 has quit IRC | 21:39 | |
| *** rhallisey has quit IRC | 21:40 | |
| *** esberglu has quit IRC | 21:41 | |
| fungi | harlowja: if i could +100 i would, for that repo name alone | 21:41 |
| *** daemontool has quit IRC | 21:41 | |
| fungi | harlowja: oh, actually need to reverse the order for the dependencies there | 21:43 |
| *** rbrndt has quit IRC | 21:43 | |
| fungi | left a comment on 349046 | 21:43 |
| harlowja | kk | 21:43 |
| harlowja | damn, thought i got it right | 21:43 |
| harlowja | lol | 21:43 |
| *** aorourke has quit IRC | 21:43 | |
| fungi | harlowja: infra is happy to create repos for official teams as quickly as the review process allows, but we don't want teams to have repos in governance which do not exist yet | 21:44 |
| *** dims_ has quit IRC | 21:44 | |
| harlowja | gotcha | 21:44 |
| *** ayoung has quit IRC | 21:44 | |
| fungi | including our own team (well, especially our own team, because if we can't keep it straight then we can't expect anyone else to either) | 21:45 |
| harlowja | i just a small peon from a small ungoverned village in the north of openstack | 21:46 |
| harlowja | lol | 21:46 |
| *** dims has joined #openstack-infra | 21:46 | |
| harlowja | not used to these big city governance processes | 21:46 |
| *** tlbr_ has joined #openstack-infra | 21:46 | |
| *** tlbr_ has quit IRC | 21:47 | |
| bkero | clarkb: what do you think about layout.yaml line 1139? | 21:48 |
| bkero | re: puppet apply requiring trusty | 21:49 |
| openstackgerrit | Joshua Harlow proposed openstack-infra/project-config: Add initial err2d2 project config https://review.openstack.org/348998 | 21:49 |
| *** sxmxc has quit IRC | 21:50 | |
| *** jsavak has quit IRC | 21:50 | |
| *** weshay is now known as weshay_afk | 21:51 | |
| *** Teresita-Warrior has quit IRC | 21:51 | |
| *** PollyZ has quit IRC | 21:51 | |
| *** timl0101 has quit IRC | 21:52 | |
| clarkb | bkero: on phone currently so cantopen the file but I think thats likely because we need to keep testing on truaty for infra? | 21:54 |
| clarkb | bkero since we deploy on trusty (and still precise too) | 21:54 |
| bkero | clarkb: # Make sure we run ubuntu-trusty on master for puppet-apply. | 21:54 |
| bkero | - name: ^gate-infra-puppet-apply-ubuntu-trusty$ | 21:54 |
| *** elo has joined #openstack-infra | 21:56 | |
| *** tlbr_ has joined #openstack-infra | 22:00 | |
| clarkb | yup my guess was right | 22:00 |
| *** ajmiller has joined #openstack-infra | 22:01 | |
| bkero | Ah okay | 22:01 |
| bkero | clarkb: and how about {pipeline}-tempest-dsvm-neutron-src-{name}{job-suffix} using node:ubuntu-trusty? | 22:02 |
| bkero | (from tempest-jobs.yml) | 22:02 |
| *** rbrndt has joined #openstack-infra | 22:03 | |
| *** ajmiller_ has joined #openstack-infra | 22:03 | |
| *** tlbr_ has quit IRC | 22:04 | |
| *** pradk has quit IRC | 22:04 | |
| *** kzaitsev_mb has joined #openstack-infra | 22:04 | |
| mnaser | how is the -infra team managing kick off puppet agent on a subset of servers? | 22:04 |
| mnaser | (or even all) | 22:05 |
| mnaser | we are using a bunch of shell scripts right now, i'm thinking ansible would be cleaner, but we've shut off the agent service on all nodes to prevent anything going wrong just in case, so we can control deployemnt of updates | 22:05 |
| harlowja | dmsimard is there any spec (or something) on what the major diff will be with jjbv2? | 22:06 |
| harlowja | (if u know) | 22:06 |
| *** ajmiller has quit IRC | 22:07 | |
| *** rbuzatu has joined #openstack-infra | 22:07 | |
| *** superdan has quit IRC | 22:07 | |
| *** PollyZ has joined #openstack-infra | 22:08 | |
| clarkb | bkero I think that one just hasnt been uodated yet | 22:08 |
| clarkb | mnaser: we use ansible to kick puppet apply | 22:08 |
| bkero | clarkb: okay, I can do that. Was wondering if it was valid to update | 22:08 |
| *** kzaitsev_mb has quit IRC | 22:09 | |
| clarkb | mnaser: it copies hiera data over to the hosts and then tries to collect facts and run infor for puppetdb | 22:09 |
| *** mmandel has quit IRC | 22:09 | |
| clarkb | bkero yup most of the tempest and devstack gate stuff still needs updating | 22:09 |
| clarkb | bkero we have done the integrated gate stuff only so far which is the base set | 22:09 |
| harlowja | anyone here know where the patchbot code is at? | 22:09 |
| harlowja | that will be on a conversion path i think to errbot plugin as well | 22:10 |
| bkero | clarkb: got it | 22:10 |
| clarkb | harlowja: that bot is notmyname's and not one of ours | 22:10 |
| harlowja | (but it'd be nice to use some of the original code, ha) | 22:10 |
| *** yamamoto has joined #openstack-infra | 22:10 | |
| harlowja | right | 22:10 |
| harlowja | notmyname linky? | 22:10 |
| *** kzaitsev_mb has joined #openstack-infra | 22:10 | |
| bkero | clarkb: I'm guessing stable-compat-jobs should still be run on trusty because mitaka? | 22:10 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config: Add gate-tripleo-quickstart-docs-ubuntu-xenial job https://review.openstack.org/349087 | 22:10 |
| *** mmandel has joined #openstack-infra | 22:11 | |
| *** rbuzatu has quit IRC | 22:11 | |
| clarkb | bkero: correct | 22:12 |
| bkero | okay | 22:12 |
| mnaser | clarkb, interesting so do you have some sort of logic that picks out what hiera data to copy over (and is that code somewhere we can look at?) | 22:12 |
| clarkb | yup its all in the ansible puppet module, and rough logic is copy the global hiera, the hiera fioe for the ansible grouo and the hiera file for the host | 22:13 |
| *** achanda has joined #openstack-infra | 22:13 | |
| *** Guest13477 has joined #openstack-infra | 22:14 | |
| *** mdrabe has quit IRC | 22:14 | |
| *** mctaylor has quit IRC | 22:14 | |
| *** yamamoto has quit IRC | 22:15 | |
| *** PollyZ has quit IRC | 22:16 | |
| openstackgerrit | Merged openstack-infra/zuul: Cloner: correct fetch on pre-1.9 git https://review.openstack.org/348978 | 22:17 |
| mnaser | thanks clarkb | 22:18 |
| *** Guest13477 has quit IRC | 22:19 | |
| krotscheck | If all else fails, submit a patch for devstack | 22:21 |
| *** ddieterly[away] is now known as ddieterly | 22:23 | |
| *** tlbr_ has joined #openstack-infra | 22:24 | |
| *** karthikp_ has quit IRC | 22:25 | |
| openstackgerrit | Ben Kero proposed openstack-infra/project-config: move {pipeline}-tempest-dsvm-ntrn-src-{name}{suffix} job to xenial https://review.openstack.org/349100 | 22:26 |
| openstackgerrit | Michael Krotscheck proposed openstack-infra/project-config: Created DSVM Job for NPM Projects https://review.openstack.org/348056 | 22:27 |
| *** tlbr_ has quit IRC | 22:29 | |
| *** psilvad has quit IRC | 22:32 | |
| *** psilvad has joined #openstack-infra | 22:32 | |
| pabelanger | ansible 2.1.10 was release yesterday, nice | 22:33 |
| pabelanger | err | 22:33 |
| pabelanger | 2.1.1.0 | 22:33 |
| *** jcoufal_ has joined #openstack-infra | 22:33 | |
| prometheanfire | just packaged it today :D | 22:33 |
| *** amitgandhinz has joined #openstack-infra | 22:34 | |
| *** jcoufal has quit IRC | 22:34 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-zuul: Upgrade to ansible 2.1.1.0 https://review.openstack.org/349103 | 22:34 |
| *** tlbr_ has joined #openstack-infra | 22:35 | |
| notmyname | harlowja: hi | 22:36 |
| harlowja | hi yo | 22:36 |
| notmyname | harlowja: https://github.com/notmyname/Patches | 22:36 |
| harlowja | thx | 22:36 |
| mtreinish | pabelanger, clarkb: if you get a sec want to take a quick look at the last few open firehose patches: https://review.openstack.org/#/q/status:open+topic:firehose | 22:37 |
| *** tkelsey has joined #openstack-infra | 22:37 | |
| *** stevemcquaid has quit IRC | 22:37 | |
| pabelanger | sure, have a few mins | 22:38 |
| *** tpsilva has quit IRC | 22:38 | |
| mtreinish | pabelanger: cool, thanks | 22:38 |
| *** amitgandhinz has quit IRC | 22:38 | |
| *** tlbr_ has quit IRC | 22:40 | |
| pabelanger | Hmm, I see some lint issues but the gate is green | 22:41 |
| pabelanger | I'll have to figure out why later | 22:41 |
| *** tkelsey has quit IRC | 22:42 | |
| prometheanfire | notmyname: hi | 22:43 |
| *** jcoufal_ has quit IRC | 22:43 | |
| *** mctaylor has joined #openstack-infra | 22:43 | |
| *** mctaylor has quit IRC | 22:44 | |
| *** mctaylor has joined #openstack-infra | 22:45 | |
| *** abregman has quit IRC | 22:45 | |
| *** gouthamr has joined #openstack-infra | 22:45 | |
| mtreinish | pabelanger: hmm, it passed my local running of puppet-lint (well my editor didn't complain at least) | 22:47 |
| mtreinish | pabelanger: is there repo specific config that needs to be done? I might not have done that correctly if so (like specifying rules or something) | 22:47 |
| *** mmandel has quit IRC | 22:48 | |
| pabelanger | mtreinish: I think you have it setup properly. It's possible something changed on the puppet side. | 22:48 |
| *** Guest41489 has joined #openstack-infra | 22:49 | |
| pabelanger | but it's a Friday and not motivated to dig into it. Will check next week | 22:49 |
| *** tphummel has quit IRC | 22:52 | |
| mtreinish | heh, fair enough :) | 22:53 |
| *** rlandy has quit IRC | 22:53 | |
| mtreinish | pabelanger: I also just spotted another thing, can you add me to: https://review.openstack.org/#/admin/groups/1499,members | 22:54 |
| mtreinish | I need to push a new release after that auth change merges, but just realized I don't have the acl | 22:54 |
| pabelanger | mtreinish: sure, email? | 22:55 |
| mtreinish | mtreinish@kortar.org | 22:55 |
| pabelanger | mtreinish: okay, found it | 22:55 |
| pabelanger | added | 22:55 |
| mtreinish | thanks | 22:55 |
| mtreinish | pabelanger: http://logs.openstack.org/23/347923/3/gate/gate-openstackci-beaker-centos-7/b0cd645/console.html#_2016-07-29_22_48_35_341479 we don't have a gems mirror? | 22:56 |
| *** Guest59655 has joined #openstack-infra | 22:56 | |
| *** piet has quit IRC | 22:57 | |
| bkero | we don't afaik | 22:58 |
| pabelanger | mtreinish: not yet, we have patches up. need to land them. I should do that next week | 22:58 |
| *** piet has joined #openstack-infra | 22:58 | |
| *** rbrndt has quit IRC | 22:59 | |
| *** sdague has quit IRC | 22:59 | |
| *** rbrndt has joined #openstack-infra | 22:59 | |
| *** rbrndt has quit IRC | 23:00 | |
| *** Guest59655 has quit IRC | 23:02 | |
| *** Guest41489 has quit IRC | 23:02 | |
| *** thorst_ has quit IRC | 23:04 | |
| *** thorst has joined #openstack-infra | 23:05 | |
| *** jcoufal has joined #openstack-infra | 23:07 | |
| *** mctaylor has quit IRC | 23:08 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/germqtt: Add docs explaining mqtt topic layout https://review.openstack.org/347618 | 23:08 |
| *** xarses has quit IRC | 23:10 | |
| *** yamamoto has joined #openstack-infra | 23:12 | |
| *** thorst has quit IRC | 23:13 | |
| *** xyang1 has quit IRC | 23:16 | |
| *** achanda has quit IRC | 23:17 | |
| *** yamamoto has quit IRC | 23:17 | |
| *** woodster_ has joined #openstack-infra | 23:21 | |
| *** tlbr_ has joined #openstack-infra | 23:22 | |
| *** EricGonczer_ has quit IRC | 23:23 | |
| *** ddieterly has quit IRC | 23:24 | |
| openstackgerrit | Merged openstack-infra/project-config: zuul/tripleo: optimize multinode and undercloud layout https://review.openstack.org/348927 | 23:24 |
| *** tonytan4ever has joined #openstack-infra | 23:25 | |
| *** Guest39119 has joined #openstack-infra | 23:26 | |
| *** tlbr_ has quit IRC | 23:27 | |
| *** kzaitsev_mb has quit IRC | 23:28 | |
| *** tlbr has joined #openstack-infra | 23:34 | |
| *** rwsu has quit IRC | 23:34 | |
| *** tlbr is now known as Guest16207 | 23:34 | |
| *** daemontool has joined #openstack-infra | 23:35 | |
| *** amitgandhinz has joined #openstack-infra | 23:35 | |
| *** Guest16207 has quit IRC | 23:38 | |
| *** amitgandhinz has quit IRC | 23:39 | |
| *** roxanaghe has quit IRC | 23:45 | |
| *** piet has quit IRC | 23:46 | |
| *** jcoufal_ has joined #openstack-infra | 23:48 | |
| *** jcoufal has quit IRC | 23:49 | |
| *** Guest39119 is now known as dansmith | 23:52 | |
| *** xarses has joined #openstack-infra | 23:53 | |
| *** piet has joined #openstack-infra | 23:53 | |
| openstackgerrit | Aaron Rosen proposed openstack-infra/project-config: Add new project vmware-nsxlib https://review.openstack.org/349118 | 23:56 |
| *** ajmiller_ has quit IRC | 23:58 | |
| *** tlbr_ has joined #openstack-infra | 23:58 | |
| *** achanda has joined #openstack-infra | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!