| mnaser | also ianw yes, ansible 2.5 apparently uses an empty env when it execs stuff and we were seeing issues like 'ip: command not found' because centos stores that in /usr/sbin and thats not in the default path | 00:00 |
|---|---|---|
| mnaser | re: https://review.openstack.org/#/c/576274/ | 00:00 |
| ianw | mnaser: ok, that makes sense ... i was wondering if the centos image broke it somehow. the "ip" location difference is a pita | 00:00 |
| mnaser | ianw: the failing jobs was fedora and centos | 00:01 |
| mnaser | which i guess was the odd one out in its decision of storing `ip` | 00:01 |
| ianw | yeah, everywhere there's hacks for bin/sbin on those platforms | 00:01 |
| *** ramishra has quit IRC | 00:01 | |
| fungi | well, on other platforms it's somewhere like /usr/bin because it's a useful tool for regular users of the system to run | 00:02 |
| mnaser | btw, 576156,4 has been waiting for a fedora machine for a very long time | 00:02 |
| mnaser | i dooon't think we have a lot of fedora consumers that its all exhausted | 00:03 |
| fungi | but rh also used to put ifconfig in (/usr)/sbin to similarly baffling explanation | 00:03 |
| clarkb | mnaser: we've been running behind all day largely due to tripleo | 00:03 |
| mnaser | does tripleo use fedora? | 00:03 |
| clarkb | while its a much better situation with working dockerhub caches they have still had costly restarts | 00:03 |
| clarkb | mnaser: no, but they are consuming the quota | 00:03 |
| clarkb | we fulfill gate before anything else | 00:04 |
| mnaser | oh i see yeah | 00:04 |
| clarkb | so restarts there starve check | 00:04 |
| mnaser | that's a big queue | 00:04 |
| mnaser | gotcha | 00:04 |
| * mnaser should do some math soon and see how much we can raise our quota | 00:04 | |
| mnaser | i guess i can do that now | 00:04 |
| clarkb | we should be bringing on packethost real soon now | 00:04 |
| clarkb | just have to sort out mirror sizing | 00:04 |
| mnaser | whos running the packethost cloud | 00:05 |
| fungi | platform 9 | 00:05 |
| ianw | clarkb: looking now | 00:05 |
| *** lifeless_ has joined #openstack-infra | 00:05 | |
| *** lifeless has quit IRC | 00:05 | |
| *** heyongli has quit IRC | 00:07 | |
| *** heyongli has joined #openstack-infra | 00:07 | |
| ianw | clarkb: ok, i have a bigger flavor. but let me see if it works now | 00:08 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/project-config master: Bump capacity at VEXXHOST https://review.openstack.org/576284 | 00:09 |
| mnaser | rough math, i know there's space for more but don't want to be too impactful and something goes wrong and then even more gate resets | 00:09 |
| mnaser | dunno if you want to make that jump the queue to gate ^ | 00:10 |
| clarkb | happy to enqueue it if we can get the reviews (looking at it now) | 00:10 |
| *** dingyichen has joined #openstack-infra | 00:13 | |
| ianw | Last exception: Multiple possible networks found, use a Network ID to be more specific. | 00:16 |
| ianw | mordred: ^ hmm, this is interesting because the default network should be in the clouds.yaml ... | 00:16 |
| *** felipemonteiro has quit IRC | 00:17 | |
| *** heyongli has quit IRC | 00:17 | |
| *** heyongli has joined #openstack-infra | 00:17 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool master: zk: remove leaked FAILED or FULFILLED request. https://review.openstack.org/576287 | 00:18 |
| *** heyongli has quit IRC | 00:27 | |
| *** heyongli has joined #openstack-infra | 00:27 | |
| clarkb | fungi: how close to the expiry date do you typically renew the certs? | 00:28 |
| clarkb | I'm thinking I should get those done this week | 00:28 |
| mnaser | ianw: would you mind reviewing the capacity bump change so maybe clarkb can enqueue to gate and we can get some more (little) breathing space | 00:28 |
| fungi | clarkb: about that long, yeah | 00:30 |
| fungi | need to save yourself enough time to work out the hiera mapping and script up the hieraedit updates | 00:31 |
| *** hph^ has joined #openstack-infra | 00:31 | |
| *** hph^ has quit IRC | 00:31 | |
| *** hph^ has joined #openstack-infra | 00:31 | |
| *** hph^ has quit IRC | 00:31 | |
| *** hph^ has joined #openstack-infra | 00:31 | |
| *** hph^ has quit IRC | 00:31 | |
| *** hph^ has joined #openstack-infra | 00:32 | |
| *** hph^ has quit IRC | 00:32 | |
| fungi | if you reuse the csrs saved in the certs dir (or at least the keys) then you should only need to update the certs (and _maybe_ the chain cert sets) | 00:32 |
| *** hph^ has joined #openstack-infra | 00:32 | |
| *** hph^ has quit IRC | 00:32 | |
| *** hph^ has joined #openstack-infra | 00:32 | |
| *** hph^ has quit IRC | 00:32 | |
| clarkb | ok looks like zuul failed on crio with latest run too, but my logging updates are working so I am generally happy | 00:33 |
| *** pahuang has quit IRC | 00:34 | |
| *** camunoz has quit IRC | 00:34 | |
| *** bdodd has joined #openstack-infra | 00:35 | |
| mnaser | clarkb: so it's failing at the same stage as jenkins right now? | 00:35 |
| clarkb | mnaser: yup | 00:36 |
| mnaser | yay | 00:36 |
| clarkb | http://logs.openstack.org/74/74/c3a54a71613ec4736adcdb363be6c17b01ccd9f1/third-party-check/kata-runsh/4284beb/job-output.txt.gz#_2018-06-19_00_28_31_026856 is zuul http://jenkins.katacontainers.io/job/kata-containers-proxy-ubuntu-16-04-PR/13/console is jenkins | 00:36 |
| *** heyongli has quit IRC | 00:37 | |
| clarkb | I'm not collecting the right logs from journalctl for crio (but neither is jenkins where I got that info, next step is to figure that out probably) | 00:37 |
| *** heyongli has joined #openstack-infra | 00:38 | |
| ianw | fungi/clarkb: ok, check out mirror.us-west-1.packethost.openstack.org now ... seem ok? | 00:38 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool master: zk: skip node already being deleted in cleanup leaked instance task https://review.openstack.org/576288 | 00:38 |
| fungi | lookin' | 00:39 |
| clarkb | ianw: yup that lgtm | 00:39 |
| fungi | ianw: yeah that should be plenty fine for space | 00:39 |
| clarkb | waiting for firefox to confirm http side | 00:39 |
| clarkb | and yup it renders the index | 00:40 |
| ianw | ok, maybe merge https://review.openstack.org/#/c/574999/ to add to cacti. i'll merge the initial job thing and we can see if it works or not ... | 00:40 |
| fungi | sure | 00:40 |
| fungi | reviewing now | 00:40 |
| mnaser | clarkb: https://review.openstack.org/#/c/576284/ has two +2's if you want to enqueue :) | 00:41 |
| tristanC | infra-root: could you please check the hiera syntax to add the firehose to the zuul connections here: https://review.openstack.org/574492 | 00:42 |
| clarkb | mnaser: done | 00:42 |
| mnaser | sweet, thank you clarkb | 00:45 |
| mnaser | it is still really strange how that job still doesnt get fedora vms (even though other projects and jobs after it are getting vms assigned) | 00:46 |
| *** yamamoto has joined #openstack-infra | 00:47 | |
| clarkb | mnaser it may have failed in pre and is at back of list | 00:47 |
| *** heyongli has quit IRC | 00:48 | |
| *** heyongli has joined #openstack-infra | 00:48 | |
| ianw | tristanC: doesn't it need to be an erb file? maybe i'm missing something | 00:48 |
| openstackgerrit | Merged openstack-infra/project-config master: Increase packethost max-servers to 10 https://review.openstack.org/575863 | 00:50 |
| tristanC | ianw: perhaps, I forgot how puppet/hiera works... should this proposed directly in the zuul.conf.erb then? | 00:51 |
| *** yamamoto has quit IRC | 00:51 | |
| *** pahuang has joined #openstack-infra | 00:52 | |
| openstackgerrit | Merged openstack-infra/project-config master: Bump capacity at VEXXHOST https://review.openstack.org/576284 | 00:52 |
| *** sshnaidm_ has joined #openstack-infra | 00:53 | |
| *** felipemonteiro has joined #openstack-infra | 00:54 | |
| *** sshnaidm|afk has quit IRC | 00:55 | |
| mnaser | nodepool will pick up the change with the next puppet i take it? | 00:56 |
| *** sshnaidm__ has joined #openstack-infra | 00:56 | |
| *** sshnaidm_ has quit IRC | 00:57 | |
| *** heyongli has quit IRC | 00:58 | |
| *** heyongli has joined #openstack-infra | 00:58 | |
| clarkb | mnaser it should | 00:59 |
| fungi | should, yes | 00:59 |
| *** sshnaidm__ has quit IRC | 01:02 | |
| openstackgerrit | Merged openstack-infra/system-config master: Add packethost mirror to cacti https://review.openstack.org/574999 | 01:04 |
| * mnaser misses that really old puppet dashboard | 01:05 | |
| mnaser | i forgot what it was called | 01:05 |
| fungi | puppetboard? | 01:05 |
| mnaser | puppetboard | 01:05 |
| mnaser | yes | 01:05 |
| *** hongbin has joined #openstack-infra | 01:05 | |
| mnaser | sigh | 01:05 |
| mnaser | fedora failed with retry_limit | 01:05 |
| fungi | i'm betting it wouldn't be hard to write something which subscribes to the firehose and logs the ansible callback events from our puppet runs | 01:06 |
| mnaser | "No viable v4 or v6 route found to git.openstack.org. The build node is assumed to be invalid." | 01:06 |
| mnaser | do we have a broken provider | 01:06 |
| mnaser | this was inap | 01:06 |
| mnaser | http://logs.openstack.org/56/576156/4/check/openstack-infra-multinode-integration-fedora-27/c4bd47f/job-output.txt.gz | 01:06 |
| ianw | urgh, i feel like sometimes we see that with fedora | 01:07 |
| fungi | looks like we didn't collect a zuul-info from the primary | 01:08 |
| fungi | http://logs.openstack.org/56/576156/4/check/openstack-infra-multinode-integration-fedora-27/c4bd47f/zuul-info/zuul-info.secondary.txt | 01:08 |
| fungi | has the network details for the secondary though | 01:08 |
| *** heyongli has quit IRC | 01:08 | |
| *** heyongli has joined #openstack-infra | 01:08 | |
| mnaser | looks like clarkb was right on the theory of going back of the line with fedora | 01:09 |
| *** eernst has joined #openstack-infra | 01:09 | |
| *** zhenguo has joined #openstack-infra | 01:10 | |
| mnaser | 8 hits in the past 12 hours | 01:10 |
| mnaser | spreading across different providers too (none which are inap either) | 01:11 |
| *** eernst has quit IRC | 01:12 | |
| mnaser | both https://review.openstack.org/#/c/576274/ and https://review.openstack.org/#/c/576156/ failed .. except fedora-latest passed | 01:12 |
| mnaser | but fedora-latest == fedora-27? | 01:12 |
| mnaser | oh no, 28 is out | 01:12 |
| ianw | no 28 now | 01:12 |
| *** eernst has joined #openstack-infra | 01:12 | |
| mnaser | so do i want to throw a recheck or shall we investigate first | 01:12 |
| mnaser | i see a fedora job for ansible-hardening which seems to have gone through | 01:13 |
| pabelanger | fedora-latest is 28, in fact fedora-27 should be removed, I just never finished that migration | 01:13 |
| pabelanger | since we only want to support a single fedora image | 01:13 |
| ianw | i think you can recheck, i can look at removing f27 | 01:15 |
| mnaser | converting to fedora-latest will be nice to avoid the constant rotations | 01:15 |
| pabelanger | yah, we'll just update the nodeset under the jobs | 01:15 |
| mnaser | pabelanger: was the planned thing is to use fedora-latest moving forward? | 01:16 |
| pabelanger | mnaser: right | 01:16 |
| mnaser | some openstack-ansible jobs are using fedora-27 so ill change those to reflect accordingly | 01:16 |
| pabelanger | right, never finished migrating everything to fedora-latest | 01:16 |
| *** sshnaidm__ has joined #openstack-infra | 01:16 | |
| pabelanger | but once all jobs on it, we'll make an announcement on ML when fedora-29 is online, the update the fedora-latest nodeset stanza to fedora-29 | 01:17 |
| pabelanger | in openstack-zuul-jobs | 01:17 |
| *** owalsh_ is now known as owalsh | 01:17 | |
| *** heyongli has quit IRC | 01:18 | |
| *** heyongli has joined #openstack-infra | 01:19 | |
| ianw | ok, just merged to start sending jobs to packethost | 01:19 |
| ianw | that was quick, all 10 are building now | 01:19 |
| pabelanger | guess we haven't updated grafana.o.o yet? | 01:20 |
| ianw | pabelanger: i haven't cut it over yet ... keep getting distracted sorry | 01:21 |
| pabelanger | ianw: np, was looking for packethost dashboard, didn't see it | 01:21 |
| ianw | pabelanger: though if you have a sec, i have some updates in grafyaml that would be helpful | 01:21 |
| ianw | oh, you mean the graph for it, no i don't think so | 01:21 |
| ianw | i don't think it's going well | 01:25 |
| ianw | no it's this damn default network issue | 01:26 |
| ianw | Multiple possible networks found, use a Network ID to be more specific. | 01:26 |
| *** masber has joined #openstack-infra | 01:27 | |
| ianw | it *has* the default network in clouds.yaml | 01:27 |
| *** zhangfei has joined #openstack-infra | 01:28 | |
| *** heyongli has quit IRC | 01:29 | |
| clarkb | I'm guessing nodepool doesnt honor that? | 01:29 |
| *** heyongli has joined #openstack-infra | 01:29 | |
| clarkb | can we even supply it in nidepools cobfig anymore? | 01:29 |
| *** jamesmcarthur has joined #openstack-infra | 01:30 | |
| pabelanger | I want to say you still need to set it in nodepool.yaml | 01:30 |
| *** dbecker has quit IRC | 01:30 | |
| ianw | well it's in ~/.config/openstack for nodepool | 01:30 |
| *** jamesmcarthur has quit IRC | 01:30 | |
| ianw | none of the other hosts with odd networking seem to have anything in nodepool.yaml config | 01:31 |
| *** jamesmcarthur has joined #openstack-infra | 01:31 | |
| pabelanger | looking at openstack/provider.py I don't see how it uses it from clouds.yaml, unless create_server from shade does. | 01:32 |
| ianw | right, openstacksdk/shade should be picking it up, i'd think | 01:33 |
| *** gyee has quit IRC | 01:33 | |
| ianw | how else does it get the username/password tec | 01:33 |
| pabelanger | yah, looks like you might be right | 01:34 |
| ianw | hang on, have i used the wrong config flag ... | 01:34 |
| *** jamesmcarthur has quit IRC | 01:35 | |
| ianw | yep i think so, it's "default_interface" ... sigh | 01:37 |
| ianw | yeah, so 147.75.38.153 is the first packethost job | 01:38 |
| *** heyongli has quit IRC | 01:39 | |
| *** heyongli has joined #openstack-infra | 01:39 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: packethost: fix default network setting https://review.openstack.org/576296 | 01:42 |
| *** eernst has quit IRC | 01:42 | |
| *** lifeless_ has quit IRC | 01:43 | |
| clarkb | ianw: what about ansible clouds.yaml? | 01:43 |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: packethost: fix default network setting https://review.openstack.org/576296 | 01:44 |
| ianw | clarkb: ^ now with even more clouds.yaml updates :) | 01:44 |
| openstackgerrit | Merged openstack-infra/grafyaml master: Add alerts to graphs https://review.openstack.org/574527 | 01:44 |
| openstackgerrit | Merged openstack-infra/grafyaml master: change default python 3 env in tox to 3.5 https://review.openstack.org/575574 | 01:44 |
| *** lifeless has joined #openstack-infra | 01:45 | |
| *** armaan_ has joined #openstack-infra | 01:46 | |
| *** armaan has quit IRC | 01:46 | |
| mnaser | clarkb: so do you think next steps would be "fix kata-proxy to see it green" in zuul and jenkins for kata side? | 01:47 |
| *** yamamoto has joined #openstack-infra | 01:47 | |
| *** dbecker has joined #openstack-infra | 01:48 | |
| clarkb | mnaser maybe see if kata is interested in fixjng it otherwise we have parity and maybe we atart looking at other repos if they want | 01:48 |
| *** heyongli has quit IRC | 01:49 | |
| *** heyongli has joined #openstack-infra | 01:49 | |
| *** pahuang has quit IRC | 01:49 | |
| *** yamahata has quit IRC | 01:50 | |
| ianw | can someone double check the default interface fix @ https://review.openstack.org/#/c/576296/ ... i'd prefer it not to miss too many puppet runs as i've just had to override it manually in nl02 | 01:50 |
| clarkb | I +2'd it mnaser pabelanger fungi ^ still around? | 01:51 |
| * mnaser is not system-config-core | 01:51 | |
| mnaser | :P | 01:51 |
| * mnaser looks anyawys | 01:51 | |
| ianw | i'll take it if we can confirm i don't have *another* typo :) | 01:52 |
| clarkb | it is project co fig | 01:52 |
| mnaser | +1'd, looks reasonable to me | 01:53 |
| mnaser | clarkb: i dunno but i never had +2 to system-config, only project-config | 01:53 |
| mnaser | and the above is a system-config change | 01:53 |
| *** yamamoto has quit IRC | 01:53 | |
| *** eernst has joined #openstack-infra | 01:53 | |
| clarkb | oh right the nodepool.yaml is project condig | 01:55 |
| mnaser | https://github.com/openstack/openstacksdk/blob/5d156585834fdd16cdc05ea64abcd3e313e01af2/doc/source/user/config/network-config.rst but docs line up | 01:55 |
| ianw | ok, not sure how to easily correlate active nodes to what jobs they're running | 01:55 |
| clarkb | ianw grep the node id number in zuul01's zuul debug.log iirc | 01:56 |
| *** heyongli has quit IRC | 01:59 | |
| *** heyongli has joined #openstack-infra | 02:00 | |
| *** pahuang has joined #openstack-infra | 02:03 | |
| ianw | yay, http://logs.openstack.org/65/571465/6/check/openstack-tox-lower-constraints/9ff42a3/zuul-info/zuul-info.ubuntu-xenial.txt ... a working packethost job | 02:03 |
| clarkb | nice | 02:05 |
| ianw | i'll let the fix merge, add a dashboard, describe creds in password file then we can probably turn it up | 02:09 |
| clarkb | to 11 | 02:09 |
| ianw | another 90 nodes would be nice | 02:09 |
| *** heyongli has quit IRC | 02:10 | |
| *** heyongli has joined #openstack-infra | 02:10 | |
| *** eernst has quit IRC | 02:10 | |
| *** eernst has joined #openstack-infra | 02:11 | |
| *** ramishra has joined #openstack-infra | 02:12 | |
| *** zhangfei has quit IRC | 02:14 | |
| *** felipemonteiro has quit IRC | 02:17 | |
| *** heyongli has quit IRC | 02:20 | |
| *** heyongli has joined #openstack-infra | 02:20 | |
| *** hemna_ has quit IRC | 02:21 | |
| *** roman_g has quit IRC | 02:22 | |
| *** armaan_ has quit IRC | 02:24 | |
| *** armaan has joined #openstack-infra | 02:25 | |
| *** zhangfei has joined #openstack-infra | 02:26 | |
| *** sthussey has quit IRC | 02:28 | |
| openstackgerrit | Merged openstack-infra/system-config master: packethost: fix default network setting https://review.openstack.org/576296 | 02:28 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Add PATH to `ip` command execution https://review.openstack.org/576274 | 02:30 |
| *** heyongli has quit IRC | 02:30 | |
| *** heyongli has joined #openstack-infra | 02:30 | |
| mnaser | i've noticed this often happening | 02:35 |
| mnaser | the above merged, https://review.openstack.org/#/c/576156/ depends on it, nothing 5 minutes later | 02:35 |
| mnaser | re-workflowing usually makes it go through | 02:35 |
| clarkb | if they dont share a gate queue that is necessary | 02:36 |
| *** felipemonteiro has joined #openstack-infra | 02:36 | |
| mnaser | i see, wanna +W it please clarkb ? avoiding a needless recheck | 02:38 |
| *** psachin has joined #openstack-infra | 02:40 | |
| clarkb | done | 02:40 |
| *** heyongli has quit IRC | 02:40 | |
| *** heyongli has joined #openstack-infra | 02:41 | |
| mnaser | https://review.openstack.org/#/c/576308/ is another 'Member' to 'member' reference (openstacksdk is ok, but because it runs shade jobs and shade had the same issue, it failed) | 02:47 |
| *** yamamoto has joined #openstack-infra | 02:49 | |
| *** rlandy has quit IRC | 02:49 | |
| *** heyongli has quit IRC | 02:51 | |
| *** heyongli has joined #openstack-infra | 02:51 | |
| mnaser | fedora-27 is doing that thing again | 02:52 |
| * mnaser grumbles | 02:52 | |
| mnaser | i guess that's my queue to head home | 02:53 |
| *** armaan has quit IRC | 02:54 | |
| *** yamamoto has quit IRC | 02:55 | |
| *** david-lyle has joined #openstack-infra | 03:00 | |
| *** armaan has joined #openstack-infra | 03:00 | |
| *** heyongli has quit IRC | 03:01 | |
| *** heyongli has joined #openstack-infra | 03:01 | |
| *** mriedem has quit IRC | 03:02 | |
| *** boris_42_ has quit IRC | 03:04 | |
| mnaser | heads-up: something is up with the fedora-27 image .. or maybe a few clouds have an odd/broken image that was pushed up | 03:07 |
| *** david-lyle has quit IRC | 03:09 | |
| *** hongbin has quit IRC | 03:10 | |
| *** heyongli has quit IRC | 03:11 | |
| *** heyongli has joined #openstack-infra | 03:11 | |
| *** diablo_rojo has quit IRC | 03:18 | |
| *** jesslampe has quit IRC | 03:19 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Add packethost grafana graphs https://review.openstack.org/576323 | 03:20 |
| *** heyongli has quit IRC | 03:21 | |
| *** heyongli has joined #openstack-infra | 03:22 | |
| *** rh-jelabarre has quit IRC | 03:22 | |
| *** felipemonteiro has quit IRC | 03:23 | |
| *** felipemonteiro has joined #openstack-infra | 03:24 | |
| *** rh-jelabarre has joined #openstack-infra | 03:26 | |
| *** jesslampe has joined #openstack-infra | 03:31 | |
| *** heyongli has quit IRC | 03:32 | |
| *** heyongli has joined #openstack-infra | 03:32 | |
| *** felipemonteiro has quit IRC | 03:36 | |
| *** heyongli has quit IRC | 03:42 | |
| *** heyongli has joined #openstack-infra | 03:42 | |
| *** eernst has quit IRC | 03:47 | |
| *** lpetrut has joined #openstack-infra | 03:48 | |
| *** eernst has joined #openstack-infra | 03:48 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: nodepool grafana : remove job runtimes https://review.openstack.org/576339 | 03:48 |
| *** ramishra has quit IRC | 03:49 | |
| *** yamamoto has joined #openstack-infra | 03:51 | |
| *** ramishra has joined #openstack-infra | 03:51 | |
| *** heyongli has quit IRC | 03:52 | |
| *** heyongli has joined #openstack-infra | 03:52 | |
| *** yamamoto has quit IRC | 03:57 | |
| openstackgerrit | Tomi Juvonen proposed openstack-infra/project-config master: New Fenix project https://review.openstack.org/571382 | 03:59 |
| *** sree has joined #openstack-infra | 04:01 | |
| *** yamamoto has joined #openstack-infra | 04:01 | |
| *** ykarel|pto has joined #openstack-infra | 04:01 | |
| *** heyongli has quit IRC | 04:02 | |
| *** heyongli has joined #openstack-infra | 04:03 | |
| *** eernst has quit IRC | 04:03 | |
| *** eernst has joined #openstack-infra | 04:04 | |
| openstackgerrit | Nguyen Van Trung proposed openstack/diskimage-builder master: Add iscsi-boot element for CentOS images https://review.openstack.org/542708 | 04:04 |
| *** jesslampe has quit IRC | 04:04 | |
| *** hongbin has joined #openstack-infra | 04:05 | |
| *** udesale has joined #openstack-infra | 04:06 | |
| *** germs has quit IRC | 04:07 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Add packethost grafana graphs https://review.openstack.org/576323 | 04:08 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: nodepool grafana : remove job runtimes https://review.openstack.org/576339 | 04:08 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Bump packethost nodes to 95 https://review.openstack.org/576345 | 04:08 |
| *** jesslampe has joined #openstack-infra | 04:11 | |
| *** heyongli has quit IRC | 04:13 | |
| *** heyongli has joined #openstack-infra | 04:13 | |
| *** jesslampe has quit IRC | 04:14 | |
| *** hongbin has quit IRC | 04:14 | |
| *** sree has quit IRC | 04:14 | |
| *** sree has joined #openstack-infra | 04:15 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/openstack-zuul-jobs master: Add NODEPOOL_PERCONA_PROXY to mirror_info https://review.openstack.org/576156 | 04:19 |
| openstackgerrit | Ian Wienand proposed openstack-infra/openstack-zuul-jobs master: Make fedora jobs non-voting https://review.openstack.org/576348 | 04:19 |
| *** Bhujay has joined #openstack-infra | 04:20 | |
| *** heyongli has quit IRC | 04:23 | |
| *** heyongli has joined #openstack-infra | 04:23 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/openstack-zuul-jobs master: Make fedora jobs non-voting https://review.openstack.org/576348 | 04:24 |
| openstackgerrit | Ian Wienand proposed openstack-infra/openstack-zuul-jobs master: Add NODEPOOL_PERCONA_PROXY to mirror_info https://review.openstack.org/576156 | 04:24 |
| *** lpetrut has quit IRC | 04:25 | |
| *** ykarel|pto is now known as ykarel | 04:26 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/openstack-zuul-jobs master: Make fedora jobs non-voting https://review.openstack.org/576348 | 04:27 |
| openstackgerrit | Ian Wienand proposed openstack-infra/openstack-zuul-jobs master: Add NODEPOOL_PERCONA_PROXY to mirror_info https://review.openstack.org/576156 | 04:27 |
| openstackgerrit | Ian Wienand proposed openstack-infra/openstack-zuul-jobs master: Make fedora jobs non-voting https://review.openstack.org/576348 | 04:29 |
| openstackgerrit | Ian Wienand proposed openstack-infra/openstack-zuul-jobs master: Add NODEPOOL_PERCONA_PROXY to mirror_info https://review.openstack.org/576156 | 04:29 |
| *** jaosorior has joined #openstack-infra | 04:32 | |
| *** lpetrut has joined #openstack-infra | 04:33 | |
| *** heyongli has quit IRC | 04:33 | |
| *** heyongli has joined #openstack-infra | 04:33 | |
| ianw | bet not one of these fedora jobs fails now | 04:33 |
| *** rh-jelabarre has quit IRC | 04:37 | |
| *** heyongli has quit IRC | 04:43 | |
| *** heyongli has joined #openstack-infra | 04:44 | |
| *** lpetrut has quit IRC | 04:46 | |
| *** strigazi has quit IRC | 04:46 | |
| *** strigazi has joined #openstack-infra | 04:48 | |
| *** pgadiya has joined #openstack-infra | 04:53 | |
| *** pgadiya has quit IRC | 04:53 | |
| *** heyongli has quit IRC | 04:54 | |
| *** heyongli has joined #openstack-infra | 04:54 | |
| *** strigazi has quit IRC | 04:55 | |
| *** pcichy_ has joined #openstack-infra | 04:55 | |
| *** pcichy has quit IRC | 04:56 | |
| *** pcichy_ is now known as pcichy | 04:56 | |
| *** strigazi has joined #openstack-infra | 04:57 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: Log output and exception when traceroute fails https://review.openstack.org/576351 | 05:00 |
| ianw | mnaser: ^ something like that would help diagnose this too | 05:00 |
| *** markvoelker has quit IRC | 05:01 | |
| mnaser | ianw: nice.. that will provide some very good info | 05:03 |
| *** heyongli has quit IRC | 05:04 | |
| *** heyongli has joined #openstack-infra | 05:04 | |
| mnaser | ianw: maybe a depends on change to test that? Looks like it didn’t run anything that would test that | 05:04 |
| ianw | yeah, i think we've still got the blind spot with the zuul-jobs repo? | 05:06 |
| mnaser | ianw: yeah I think so. We’re not running any jobs that check on the roles | 05:07 |
| ianw | oh i mean because it deploys on the executor it isn't tested | 05:07 |
| *** Bhujay has quit IRC | 05:08 | |
| *** iranzo has joined #openstack-infra | 05:08 | |
| mnaser | Ah, not sure how to resolve that :( | 05:09 |
| ianw | last i heard there was talk of doing some sort of two-level nesting | 05:12 |
| *** heyongli has quit IRC | 05:14 | |
| *** heyongli has joined #openstack-infra | 05:14 | |
| *** lpetrut has joined #openstack-infra | 05:22 | |
| *** cshastri_ has joined #openstack-infra | 05:24 | |
| *** heyongli has quit IRC | 05:24 | |
| *** heyongli has joined #openstack-infra | 05:25 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul-jobs master: Log output and exception when traceroute fails https://review.openstack.org/576351 | 05:30 |
| openstackgerrit | Tomi Juvonen proposed openstack-infra/project-config master: New Fenix project https://review.openstack.org/571382 | 05:32 |
| *** links has joined #openstack-infra | 05:33 | |
| frickler | ianw: you seem to be duplicating https://review.openstack.org/561758 | 05:33 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Make fedora jobs non-voting https://review.openstack.org/576348 | 05:33 |
| *** lpetrut has quit IRC | 05:33 | |
| *** heyongli has quit IRC | 05:35 | |
| *** heyongli has joined #openstack-infra | 05:35 | |
| ianw | frickler: yes, they're similar as i didn't think to look for reviews before doing mine :) personally i think it's a bit more consistent to stick things into "ret" and pass **ret but ... whatever | 05:36 |
| frickler | ianw: I didn't mean to imply that your approach is worse, didn't look in detail yet. since the dependencies seem to be blocked anyway, I'm fine with letting you and clarkb settle this ;) | 05:40 |
| *** eernst has quit IRC | 05:41 | |
| ianw | two reviews enter, one will be abandoned. this week on review wars :) | 05:41 |
| *** eernst has joined #openstack-infra | 05:41 | |
| *** Bhujay has joined #openstack-infra | 05:42 | |
| *** cshastri_ has quit IRC | 05:44 | |
| *** heyongli has quit IRC | 05:45 | |
| *** heyongli has joined #openstack-infra | 05:45 | |
| *** janki has joined #openstack-infra | 05:47 | |
| *** pcichy has quit IRC | 05:49 | |
| *** lifeless has quit IRC | 05:53 | |
| *** lifeless has joined #openstack-infra | 05:53 | |
| *** heyongli has quit IRC | 05:55 | |
| *** heyongli has joined #openstack-infra | 05:55 | |
| *** udesale has quit IRC | 06:02 | |
| *** heyongli has quit IRC | 06:05 | |
| *** heyongli has joined #openstack-infra | 06:06 | |
| *** cshastri has joined #openstack-infra | 06:11 | |
| *** jcoufal has joined #openstack-infra | 06:12 | |
| *** heyongli has quit IRC | 06:16 | |
| *** heyongli has joined #openstack-infra | 06:16 | |
| *** gongysh has joined #openstack-infra | 06:17 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Capture logs and install docker in kata-runsh job https://review.openstack.org/576178 | 06:17 |
| *** yamahata has joined #openstack-infra | 06:17 | |
| *** armaan has quit IRC | 06:17 | |
| *** armaan has joined #openstack-infra | 06:18 | |
| *** udesale has joined #openstack-infra | 06:18 | |
| *** lpetrut has joined #openstack-infra | 06:19 | |
| openstackgerrit | Tomi Juvonen proposed openstack-infra/project-config master: New Fenix project https://review.openstack.org/571382 | 06:24 |
| *** heyongli has quit IRC | 06:26 | |
| *** heyongli has joined #openstack-infra | 06:26 | |
| *** ykarel_ has joined #openstack-infra | 06:33 | |
| *** ykarel has quit IRC | 06:36 | |
| *** heyongli has quit IRC | 06:36 | |
| *** ykarel_ is now known as ykarel | 06:36 | |
| *** heyongli has joined #openstack-infra | 06:36 | |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder master: Fix bootloader for efi on rhel systems https://review.openstack.org/576041 | 06:37 |
| *** felipemonteiro has joined #openstack-infra | 06:40 | |
| *** Bhujay has quit IRC | 06:42 | |
| *** heyongli has quit IRC | 06:46 | |
| *** heyongli has joined #openstack-infra | 06:47 | |
| *** lpetrut has quit IRC | 06:49 | |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder master: Convert labels to upper case https://review.openstack.org/576098 | 06:51 |
| *** felipemonteiro has quit IRC | 06:51 | |
| *** psachin has quit IRC | 06:54 | |
| *** heyongli has quit IRC | 06:57 | |
| *** heyongli has joined #openstack-infra | 06:57 | |
| *** udesale has quit IRC | 07:00 | |
| *** udesale has joined #openstack-infra | 07:00 | |
| *** markvoelker has joined #openstack-infra | 07:01 | |
| *** slaweq has joined #openstack-infra | 07:02 | |
| *** tesseract has joined #openstack-infra | 07:02 | |
| *** alexchadin has joined #openstack-infra | 07:04 | |
| *** rcernin has quit IRC | 07:05 | |
| *** amoralej|off is now known as amoralej | 07:06 | |
| *** heyongli has quit IRC | 07:07 | |
| *** heyongli has joined #openstack-infra | 07:07 | |
| *** kamren has joined #openstack-infra | 07:08 | |
| *** rossella_s has joined #openstack-infra | 07:09 | |
| *** cshastri has quit IRC | 07:12 | |
| *** hashar has joined #openstack-infra | 07:14 | |
| *** florianf has joined #openstack-infra | 07:16 | |
| *** florianf has quit IRC | 07:16 | |
| *** florianf has joined #openstack-infra | 07:16 | |
| *** alexchadin has quit IRC | 07:16 | |
| *** heyongli has quit IRC | 07:17 | |
| *** heyongli has joined #openstack-infra | 07:17 | |
| *** ccamacho has joined #openstack-infra | 07:19 | |
| *** kjackal has quit IRC | 07:21 | |
| *** dhajare has joined #openstack-infra | 07:25 | |
| *** sshnaidm__ is now known as sshnaidm | 07:25 | |
| openstackgerrit | Adit Sarfaty proposed openstack-infra/project-config master: Add octavia to vmware_nsx jobs https://review.openstack.org/576394 | 07:26 |
| *** zoli is now known as zoli|wfh | 07:27 | |
| *** zoli|wfh is now known as zoli | 07:27 | |
| *** heyongli has quit IRC | 07:27 | |
| *** heyongli has joined #openstack-infra | 07:28 | |
| *** namnh has joined #openstack-infra | 07:28 | |
| *** cshastri has joined #openstack-infra | 07:29 | |
| *** alexchadin has joined #openstack-infra | 07:30 | |
| *** udesale has quit IRC | 07:30 | |
| *** jpena|off is now known as jpena | 07:33 | |
| *** tosky has joined #openstack-infra | 07:34 | |
| *** logan- has quit IRC | 07:35 | |
| *** petrovich has quit IRC | 07:35 | |
| *** mpeterson has quit IRC | 07:36 | |
| *** markvoelker has quit IRC | 07:36 | |
| *** heyongli has quit IRC | 07:38 | |
| *** mpeterson has joined #openstack-infra | 07:38 | |
| *** heyongli has joined #openstack-infra | 07:38 | |
| *** logan- has joined #openstack-infra | 07:39 | |
| openstackgerrit | Ghanshyam Mann proposed openstack-infra/irc-meetings master: Release the Nova API weekly meeting slot. https://review.openstack.org/576398 | 07:39 |
| *** kjackal has joined #openstack-infra | 07:43 | |
| *** petrovich has joined #openstack-infra | 07:48 | |
| *** heyongli has quit IRC | 07:48 | |
| *** heyongli has joined #openstack-infra | 07:48 | |
| *** kjackal has quit IRC | 07:50 | |
| *** pcaruana has joined #openstack-infra | 07:50 | |
| *** heyongli has quit IRC | 07:58 | |
| *** AJaeger has quit IRC | 07:58 | |
| *** heyongli has joined #openstack-infra | 07:58 | |
| *** Bhujay has joined #openstack-infra | 08:00 | |
| *** jpich has joined #openstack-infra | 08:03 | |
| *** kjackal has joined #openstack-infra | 08:03 | |
| *** AJaeger has joined #openstack-infra | 08:04 | |
| *** udesale has joined #openstack-infra | 08:04 | |
| *** roman_g has joined #openstack-infra | 08:05 | |
| dmellado | ianw: there can be only one? If we get a Christopher Lambert review I'll blame you xD | 08:07 |
| *** heyongli has quit IRC | 08:08 | |
| *** heyongli has joined #openstack-infra | 08:09 | |
| *** kjackal_ has joined #openstack-infra | 08:10 | |
| *** dougsz has joined #openstack-infra | 08:10 | |
| *** armaan has quit IRC | 08:12 | |
| *** armaan has joined #openstack-infra | 08:13 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Change barbican meeting time and storyboard link https://review.openstack.org/576177 | 08:16 |
| *** nicolasbock has joined #openstack-infra | 08:17 | |
| *** heyongli has quit IRC | 08:19 | |
| *** heyongli has joined #openstack-infra | 08:19 | |
| *** kashyap has left #openstack-infra | 08:19 | |
| *** zoli is now known as zoli|afk | 08:19 | |
| *** nicolasbock has quit IRC | 08:24 | |
| *** eernst has quit IRC | 08:26 | |
| *** lifeless has quit IRC | 08:28 | |
| *** heyongli has quit IRC | 08:29 | |
| *** sree_ has joined #openstack-infra | 08:29 | |
| *** heyongli has joined #openstack-infra | 08:29 | |
| *** sree_ is now known as Guest9684 | 08:29 | |
| *** sree has quit IRC | 08:32 | |
| *** markvoelker has joined #openstack-infra | 08:33 | |
| *** luzC_ has quit IRC | 08:34 | |
| *** isq_ has joined #openstack-infra | 08:34 | |
| *** e0ne has joined #openstack-infra | 08:34 | |
| *** isq has quit IRC | 08:35 | |
| *** luzC has joined #openstack-infra | 08:36 | |
| *** heyongli has quit IRC | 08:39 | |
| *** heyongli has joined #openstack-infra | 08:40 | |
| *** armaan has quit IRC | 08:40 | |
| *** armaan has joined #openstack-infra | 08:41 | |
| *** priteau has joined #openstack-infra | 08:42 | |
| *** derekh has joined #openstack-infra | 08:44 | |
| *** heyongli has quit IRC | 08:49 | |
| *** jaosorior has quit IRC | 08:49 | |
| *** heyongli has joined #openstack-infra | 08:50 | |
| *** shardy has joined #openstack-infra | 08:56 | |
| *** pblaho has joined #openstack-infra | 08:59 | |
| *** heyongli has quit IRC | 09:00 | |
| *** heyongli has joined #openstack-infra | 09:00 | |
| *** dingyichen has quit IRC | 09:03 | |
| *** markvoelker has quit IRC | 09:06 | |
| frickler | mordred: corvus: this is (to me) a new type of bounce for root mail from you: "2001:4801:7827:101:be76:4eff:fe10:8bc5 is listed in the Spamhaus DNSBL." http://paste.openstack.org/show/723803/ | 09:08 |
| *** heyongli has quit IRC | 09:10 | |
| *** heyongli has joined #openstack-infra | 09:10 | |
| *** pbourke has quit IRC | 09:10 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Move QuotaInformation to driver utils https://review.openstack.org/572860 | 09:11 |
| openstackgerrit | Merged openstack-infra/nodepool master: fix tox python3 overrides https://review.openstack.org/574344 | 09:11 |
| *** dhajare has quit IRC | 09:11 | |
| *** pbourke has joined #openstack-infra | 09:12 | |
| openstackgerrit | yolanda.robla proposed openstack/diskimage-builder master: Convert labels to upper case https://review.openstack.org/576098 | 09:13 |
| *** dizquierdo has joined #openstack-infra | 09:14 | |
| *** jaosorior has joined #openstack-infra | 09:15 | |
| *** heyongli has quit IRC | 09:20 | |
| *** heyongli has joined #openstack-infra | 09:20 | |
| *** sambetts|afk is now known as sambetts | 09:22 | |
| openstackgerrit | Merged openstack-infra/zuul master: Rename action-general to actiongeneral https://review.openstack.org/575617 | 09:23 |
| openstackgerrit | Merged openstack-infra/zuul master: Add missing __init__.py files https://review.openstack.org/575591 | 09:23 |
| openstackgerrit | Merged openstack-infra/zuul master: Remove failed_when when creating /tmp/console-None.log https://review.openstack.org/574672 | 09:23 |
| openstackgerrit | Merged openstack-infra/zuul master: Treat ::1 as localhost in addition to 127.0.0.1 https://review.openstack.org/572847 | 09:23 |
| openstackgerrit | Tomi Juvonen proposed openstack-infra/project-config master: Zuul for New Fenix project https://review.openstack.org/571390 | 09:24 |
| openstackgerrit | Merged openstack-infra/zuul master: Handle multiple stats in assertReportedStats https://review.openstack.org/572230 | 09:25 |
| openstackgerrit | Merged openstack-infra/zuul master: zuul-migrate: run / post-run playbooks need .yaml extensions https://review.openstack.org/572160 | 09:25 |
| openstackgerrit | Olivier Bourdon proposed openstack/diskimage-builder master: Fix /etc/network/interfaces file contents https://review.openstack.org/562535 | 09:26 |
| *** sree has joined #openstack-infra | 09:26 | |
| *** sree has quit IRC | 09:27 | |
| *** Guest9684 has quit IRC | 09:29 | |
| *** alexchadin has quit IRC | 09:29 | |
| *** heyongli has quit IRC | 09:30 | |
| *** heyongli has joined #openstack-infra | 09:31 | |
| *** zoli|afk is now known as zoli | 09:35 | |
| *** zoli is now known as zoli|wfh | 09:35 | |
| *** zoli|wfh is now known as zoli | 09:35 | |
| *** heyongli has quit IRC | 09:41 | |
| *** heyongli has joined #openstack-infra | 09:41 | |
| *** kjackal__ has joined #openstack-infra | 09:45 | |
| *** kjackal___ has joined #openstack-infra | 09:45 | |
| *** kjackal____ has joined #openstack-infra | 09:46 | |
| *** kjackal_____ has joined #openstack-infra | 09:46 | |
| *** dtantsur|afk is now known as dtantsur | 09:46 | |
| *** kjackal has quit IRC | 09:46 | |
| *** kjackal_ has quit IRC | 09:46 | |
| *** kjackal__ has quit IRC | 09:50 | |
| *** kjackal___ has quit IRC | 09:50 | |
| *** heyongli has quit IRC | 09:51 | |
| *** heyongli has joined #openstack-infra | 09:51 | |
| *** gongysh has quit IRC | 09:55 | |
| openstackgerrit | chenxiangui proposed openstack/diskimage-builder master: Remove redundant word https://review.openstack.org/576466 | 10:00 |
| *** heyongli has quit IRC | 10:01 | |
| *** heyongli has joined #openstack-infra | 10:01 | |
| *** eernst has joined #openstack-infra | 10:03 | |
| *** markvoelker has joined #openstack-infra | 10:03 | |
| *** heyongli has quit IRC | 10:11 | |
| *** heyongli has joined #openstack-infra | 10:12 | |
| *** namnh has quit IRC | 10:14 | |
| openstackgerrit | Tomi Juvonen proposed openstack-infra/project-config master: New Fenix project https://review.openstack.org/571382 | 10:16 |
| *** vtapia has quit IRC | 10:17 | |
| openstackgerrit | Tomi Juvonen proposed openstack-infra/project-config master: Zuul for New Fenix project https://review.openstack.org/571390 | 10:18 |
| *** dtantsur is now known as dtantsur|brb | 10:20 | |
| *** heyongli has quit IRC | 10:22 | |
| *** heyongli has joined #openstack-infra | 10:22 | |
| *** udesale has quit IRC | 10:28 | |
| *** heyongli has quit IRC | 10:32 | |
| *** heyongli has joined #openstack-infra | 10:32 | |
| *** eernst has quit IRC | 10:34 | |
| *** annp has quit IRC | 10:36 | |
| *** vtapia has joined #openstack-infra | 10:36 | |
| *** markvoelker has quit IRC | 10:37 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: Add tenant yaml validation option to scheduler https://review.openstack.org/574265 | 10:37 |
| *** kjackal_____ has quit IRC | 10:38 | |
| *** kjackal____ has quit IRC | 10:38 | |
| *** armaan has quit IRC | 10:38 | |
| *** armaan has joined #openstack-infra | 10:39 | |
| *** armaan has quit IRC | 10:39 | |
| *** apetrich has quit IRC | 10:39 | |
| *** armaan has joined #openstack-infra | 10:40 | |
| *** apetrich has joined #openstack-infra | 10:42 | |
| *** electrofelix has joined #openstack-infra | 10:42 | |
| *** heyongli has quit IRC | 10:42 | |
| *** heyongli has joined #openstack-infra | 10:42 | |
| *** udesale has joined #openstack-infra | 10:45 | |
| *** udesale has quit IRC | 10:46 | |
| *** udesale has joined #openstack-infra | 10:46 | |
| *** gfidente has joined #openstack-infra | 10:47 | |
| *** gfidente has joined #openstack-infra | 10:47 | |
| *** yamamoto has quit IRC | 10:50 | |
| *** alexchadin has joined #openstack-infra | 10:50 | |
| *** Deknos has joined #openstack-infra | 10:50 | |
| *** kjackal has joined #openstack-infra | 10:51 | |
| *** kjackal_ has joined #openstack-infra | 10:52 | |
| *** heyongli has quit IRC | 10:52 | |
| *** heyongli has joined #openstack-infra | 10:53 | |
| *** roman_g has quit IRC | 10:56 | |
| *** ykarel has quit IRC | 10:56 | |
| *** roman_g has joined #openstack-infra | 10:56 | |
| *** ykarel has joined #openstack-infra | 10:56 | |
| *** yamamoto has joined #openstack-infra | 10:58 | |
| *** yamamoto has quit IRC | 10:58 | |
| *** vtapia has quit IRC | 11:00 | |
| *** hemna_ has joined #openstack-infra | 11:01 | |
| *** heyongli has quit IRC | 11:03 | |
| *** heyongli has joined #openstack-infra | 11:03 | |
| *** links has quit IRC | 11:06 | |
| *** hemna_ has quit IRC | 11:06 | |
| *** jpena is now known as jpena|lunch | 11:07 | |
| *** panda is now known as panda|lunch | 11:09 | |
| *** heyongli has quit IRC | 11:13 | |
| *** heyongli has joined #openstack-infra | 11:13 | |
| *** shardy is now known as shardy_afk | 11:14 | |
| *** lifeless has joined #openstack-infra | 11:19 | |
| *** alexchadin has quit IRC | 11:21 | |
| *** links has joined #openstack-infra | 11:22 | |
| *** heyongli has quit IRC | 11:23 | |
| *** heyongli has joined #openstack-infra | 11:23 | |
| *** alexchadin has joined #openstack-infra | 11:24 | |
| *** lifeless has quit IRC | 11:28 | |
| *** vtapia has joined #openstack-infra | 11:28 | |
| *** lifeless has joined #openstack-infra | 11:28 | |
| *** dhajare has joined #openstack-infra | 11:30 | |
| *** sshnaidm is now known as sshnaidm|lnch | 11:31 | |
| *** jtomasek has quit IRC | 11:32 | |
| *** heyongli has quit IRC | 11:33 | |
| *** markvoelker has joined #openstack-infra | 11:34 | |
| *** heyongli has joined #openstack-infra | 11:34 | |
| *** shardy_afk is now known as shardy | 11:35 | |
| *** armaan has quit IRC | 11:37 | |
| *** armaan has joined #openstack-infra | 11:38 | |
| *** links has quit IRC | 11:39 | |
| *** heyongli has quit IRC | 11:44 | |
| *** dtantsur|brb is now known as dtantsur | 11:44 | |
| *** heyongli has joined #openstack-infra | 11:44 | |
| *** ldnunes has joined #openstack-infra | 11:45 | |
| *** dhajare has quit IRC | 11:45 | |
| *** yamamoto has joined #openstack-infra | 11:48 | |
| *** pcichy has joined #openstack-infra | 11:48 | |
| *** edmondsw has joined #openstack-infra | 11:50 | |
| *** jamesmcarthur has joined #openstack-infra | 11:51 | |
| *** heyongli has quit IRC | 11:54 | |
| *** heyongli has joined #openstack-infra | 11:55 | |
| *** jamesmcarthur has quit IRC | 11:55 | |
| *** amoralej is now known as amoralej|lunch | 11:57 | |
| *** dhajare has joined #openstack-infra | 11:58 | |
| *** tpsilva has joined #openstack-infra | 12:00 | |
| *** jpena|lunch is now known as jpena | 12:00 | |
| *** markvoelker has quit IRC | 12:01 | |
| *** markvoelker has joined #openstack-infra | 12:02 | |
| *** heyongli has quit IRC | 12:04 | |
| *** heyongli has joined #openstack-infra | 12:05 | |
| *** anteaya has quit IRC | 12:05 | |
| *** links has joined #openstack-infra | 12:05 | |
| *** salv-orlando has joined #openstack-infra | 12:06 | |
| *** trown|outtypewww is now known as trown | 12:07 | |
| *** rh-jelabarre has joined #openstack-infra | 12:08 | |
| fungi | frickler: rackspace preemptively lists all their ip space in the spamhaus pbl and we have to manually ask spamhaus to whitelist servers we want sending e-mail in any quantity. the kdcs probably don't fall in the "worth it to de-list" category | 12:09 |
| *** salv-orlando has quit IRC | 12:10 | |
| *** panda|lunch is now known as panda | 12:11 | |
| *** sshnaidm|lnch is now known as sshnaidm | 12:11 | |
| *** rnoriega has quit IRC | 12:13 | |
| *** _ari_ has quit IRC | 12:13 | |
| *** _ari_ has joined #openstack-infra | 12:13 | |
| *** rnoriega has joined #openstack-infra | 12:14 | |
| *** heyongli has quit IRC | 12:14 | |
| *** rnoriega has quit IRC | 12:14 | |
| *** heyongli has joined #openstack-infra | 12:15 | |
| *** yamamoto has quit IRC | 12:15 | |
| *** rnoriega has joined #openstack-infra | 12:16 | |
| *** alexchadin has quit IRC | 12:16 | |
| *** rlandy has joined #openstack-infra | 12:17 | |
| *** d0ugal has quit IRC | 12:24 | |
| *** heyongli has quit IRC | 12:25 | |
| *** heyongli has joined #openstack-infra | 12:25 | |
| *** jistr is now known as jistr|mtg | 12:28 | |
| *** alexchadin has joined #openstack-infra | 12:29 | |
| *** sshnaidm is now known as sshnaidm|afk | 12:29 | |
| *** sshnaidm__ has joined #openstack-infra | 12:29 | |
| *** heyongli has quit IRC | 12:35 | |
| *** heyongli has joined #openstack-infra | 12:35 | |
| *** armaan has quit IRC | 12:43 | |
| *** armaan has joined #openstack-infra | 12:43 | |
| *** d0ugal has joined #openstack-infra | 12:43 | |
| *** heyongli has quit IRC | 12:45 | |
| *** heyongli has joined #openstack-infra | 12:45 | |
| *** yamamoto has joined #openstack-infra | 12:46 | |
| *** dizquierdo has quit IRC | 12:51 | |
| *** amoralej|lunch is now known as amoralej | 12:52 | |
| *** camunoz has joined #openstack-infra | 12:52 | |
| *** mriedem has joined #openstack-infra | 12:53 | |
| *** heyongli has quit IRC | 12:55 | |
| *** heyongli has joined #openstack-infra | 12:56 | |
| *** armaan has quit IRC | 12:59 | |
| *** jtomasek has joined #openstack-infra | 12:59 | |
| *** armaan has joined #openstack-infra | 12:59 | |
| *** Goneri has joined #openstack-infra | 13:00 | |
| *** heyongli has quit IRC | 13:06 | |
| *** myoung|off is now known as myoung | 13:06 | |
| *** heyongli has joined #openstack-infra | 13:06 | |
| *** sree has joined #openstack-infra | 13:06 | |
| *** salv-orlando has joined #openstack-infra | 13:07 | |
| *** kgiusti has joined #openstack-infra | 13:07 | |
| *** sshnaidm__ is now known as sshnaidm | 13:08 | |
| *** salv-orlando has quit IRC | 13:11 | |
| *** jesslampe has joined #openstack-infra | 13:15 | |
| *** heyongli has quit IRC | 13:16 | |
| *** heyongli has joined #openstack-infra | 13:16 | |
| *** sshnaidm is now known as sshnaidm|afk | 13:21 | |
| *** links has quit IRC | 13:22 | |
| *** bobh has joined #openstack-infra | 13:22 | |
| openstackgerrit | Mohammed Naser proposed openstack-infra/openstack-zuul-jobs master: Add NODEPOOL_PERCONA_PROXY to mirror_info https://review.openstack.org/576156 | 13:24 |
| mnaser | frickler: ^ addressed your comment | 13:25 |
| *** heyongli has quit IRC | 13:26 | |
| *** heyongli has joined #openstack-infra | 13:26 | |
| frickler | mnaser: thx, +2 on that. just being curious, can you show me where these are being consumed? codesearch doesn't seem to find anything for me. | 13:30 |
| mnaser | frickler: percona one is not yet™ | 13:30 |
| mnaser | cause i cant do depends-on that (well, it fails) | 13:30 |
| mnaser | lemme find | 13:31 |
| mnaser | frickler: thats an override https://review.openstack.org/#/c/576239/1 | 13:31 |
| *** yamamoto has quit IRC | 13:34 | |
| frickler | mnaser: hmm, from that I cannot see why you couldn't move the "http://" part into the variable, too, but I also don't understand how you are setting that variable, so maybe it has to be that way | 13:35 |
| mnaser | frickler: i honestly remember making it that way and changing it because of some reason/discussion | 13:36 |
| mnaser | i don't have that off hand, but with http:// it was the first initial 'way' | 13:36 |
| mnaser | something didn't work somewhere | 13:36 |
| *** heyongli has quit IRC | 13:36 | |
| *** heyongli has joined #openstack-infra | 13:37 | |
| frickler | mnaser: yeah, would have been nice to stay consistent in the mirror definitions, but well, as long as it is documented, I think it's o.k. this way | 13:39 |
| *** zoli is now known as zoli|afk | 13:40 | |
| *** jistr|mtg is now known as jistr | 13:42 | |
| *** ccamacho has quit IRC | 13:42 | |
| *** dhajare has quit IRC | 13:45 | |
| mnaser | frickler: i agree, i'll look into seeing long term if we can make the swap | 13:46 |
| *** heyongli has quit IRC | 13:47 | |
| *** heyongli has joined #openstack-infra | 13:47 | |
| *** esarault has joined #openstack-infra | 13:51 | |
| *** hongbin has joined #openstack-infra | 13:53 | |
| *** rh-jelabarre has quit IRC | 13:53 | |
| *** yamamoto has joined #openstack-infra | 13:55 | |
| *** yamamoto has quit IRC | 13:55 | |
| fungi | infra-root: has anyone figured out how to get a new api key for updating rackspace dns? | 13:56 |
| mordred | fungi: no - but I didn't know I was supposed to be figuring that out | 13:56 |
| *** heyongli has quit IRC | 13:57 | |
| *** bobh has quit IRC | 13:57 | |
| *** heyongli has joined #openstack-infra | 13:57 | |
| fungi | the one for our old account is no longer authorized (since we switched logins i expect?) and the "get api key" link in the cloud resources dashboard goes to their account settings management page which is basically disabled for our account | 13:57 |
| *** yamamoto has joined #openstack-infra | 13:57 | |
| mordred | fungi: huh | 13:57 |
| panda | where can I find the documentation for the latest Depends-On: implementation changes ? | 13:58 |
| *** alexchadin has quit IRC | 13:58 | |
| fungi | mordred: my guess is that there is a different login for our tenant which has permission to generate api keys, but our login doesn't | 13:58 |
| fungi | panda: what do you mean by "latest"? | 13:58 |
| *** ethfci has joined #openstack-infra | 13:58 | |
| *** ethfci has quit IRC | 13:59 | |
| AJaeger | panda: https://docs.openstack.org/infra/manual/developers.html#cross-repository-dependencies explains everything | 13:59 |
| panda | fungi: I'm looking at a discussion from 24th Jan with the new Depends-On syntax | 13:59 |
| fungi | panda: there is zuul documentation at https://zuul-ci.org/docs/zuul/user/gating.html#cross-project-dependencies which is kept up to date | 13:59 |
| *** janki has quit IRC | 13:59 | |
| fungi | panda: oh, you want announcements, not documentation? | 13:59 |
| *** alexchadin has joined #openstack-infra | 13:59 | |
| panda | fungi: no , no documentation is perfect | 13:59 |
| panda | AJaeger: fungi thanks, I'll look at both | 14:00 |
| fungi | yeah, the one AJaeger linked is the workflow documentation specific to our deployment of zuul, while the documentation i linked is the official zuul documentation covering that feature. both should be compatible | 14:00 |
| *** ccamacho has joined #openstack-infra | 14:00 | |
| *** ethfci has joined #openstack-infra | 14:01 | |
| AJaeger | panda: so, check either - and if you found incompatibilities, please tell us ;) | 14:01 |
| panda | sure :) | 14:01 |
| *** rh-jelabarre has joined #openstack-infra | 14:01 | |
| *** ethfci has quit IRC | 14:01 | |
| *** rpioso|afk is now known as rpioso | 14:03 | |
| fungi | panda: the announcement summarizing the change was http://lists.openstack.org/pipermail/openstack-dev/2018-January/126535.html by the way, in case you also need that | 14:03 |
| *** mihalis68 has joined #openstack-infra | 14:03 | |
| *** esarault_ has joined #openstack-infra | 14:03 | |
| *** owalsh is now known as owalsh_brb | 14:06 | |
| panda | fungi: thanks, I have it, probably not a good idea to read all the thread to understand what was implemented in the end. | 14:06 |
| *** esarault has quit IRC | 14:06 | |
| *** heyongli has quit IRC | 14:07 | |
| *** salv-orlando has joined #openstack-infra | 14:07 | |
| *** heyongli has joined #openstack-infra | 14:07 | |
| *** cshastri has quit IRC | 14:08 | |
| *** germs has joined #openstack-infra | 14:08 | |
| *** germs has quit IRC | 14:08 | |
| *** germs has joined #openstack-infra | 14:08 | |
| *** rfolco is now known as folco | 14:11 | |
| *** salv-orlando has quit IRC | 14:11 | |
| *** folco is now known as rfolco | 14:12 | |
| *** jamesmcarthur has joined #openstack-infra | 14:13 | |
| mnaser | review.openstack.org took too long to respond. | 14:13 |
| dansmith | did gerrit just go away? | 14:14 |
| dansmith | heh | 14:14 |
| mnaser | ha | 14:14 |
| mnaser | infra-root: heads up ^ | 14:14 |
| * dansmith quickly copies the epic comment he has typed out | 14:14 | |
| fungi | i can ssh into it | 14:15 |
| *** germs has quit IRC | 14:15 | |
| johnsom | Same here, super long response times from gerrit | 14:15 |
| fungi | checking its performance graphs now | 14:15 |
| *** felipemonteiro_ has joined #openstack-infra | 14:15 | |
| fungi | once i get authenticated anyway | 14:16 |
| *** yamamoto has quit IRC | 14:16 | |
| dansmith | s'back | 14:17 |
| dansmith | fungi: ^ | 14:17 |
| *** dpawlik has quit IRC | 14:17 | |
| *** felipemonteiro__ has joined #openstack-infra | 14:17 | |
| *** heyongli has quit IRC | 14:17 | |
| *** armaan has quit IRC | 14:18 | |
| *** armaan has joined #openstack-infra | 14:18 | |
| *** heyongli has joined #openstack-infra | 14:18 | |
| fungi | yeah, sorry, getting hit with unrelated sysadmin requests from different places | 14:20 |
| fungi | according to javamelody, doesn't seem to have been related to any spike in jvm garbage collection | 14:21 |
| *** felipemonteiro_ has quit IRC | 14:21 | |
| *** armaan has quit IRC | 14:22 | |
| *** germs has joined #openstack-infra | 14:22 | |
| *** armaan has joined #openstack-infra | 14:22 | |
| fungi | the various http service graphs seen to have some very brief gaps in data collection right around 14:15 utc | 14:23 |
| fungi | open files didn't really spike but reached its highest value yet (around 3.35k) at the same time | 14:24 |
| *** owalsh_brb is now known as owalsh | 14:26 | |
| fungi | cacti indicates a spike in established tcp connections at the same time, though that's likely a side effect of whatever was taking responses so long to complete | 14:26 |
| *** germs has quit IRC | 14:27 | |
| *** e0ne has quit IRC | 14:27 | |
| *** alexchadin has quit IRC | 14:27 | |
| *** e0ne has joined #openstack-infra | 14:27 | |
| *** heyongli has quit IRC | 14:28 | |
| *** heyongli has joined #openstack-infra | 14:28 | |
| fungi | there's also been an increase in writes to the cinder volume where gerrit keeps its git repositories, though not a huge one, starting around 12:00z and spiking a bit just after 14:00z | 14:29 |
| *** yamamoto has joined #openstack-infra | 14:30 | |
| *** germs has joined #openstack-infra | 14:31 | |
| *** germs has quit IRC | 14:31 | |
| *** germs has joined #openstack-infra | 14:31 | |
| *** bobh has joined #openstack-infra | 14:32 | |
| fungi | could be gerrit's git garbage collection underway. iotop shows some gerrit java processes doing a fair amount of writing | 14:32 |
| *** hemna_ has joined #openstack-infra | 14:33 | |
| *** jamesmca_ has joined #openstack-infra | 14:34 | |
| *** yamamoto has quit IRC | 14:35 | |
| clarkb | fungi you sbould be able to list gerrits internal processes to see those too | 14:36 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Improve test case test_unprotected_branches https://review.openstack.org/576536 | 14:37 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Move exclude unprotected branches check into tenant https://review.openstack.org/576537 | 14:37 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Don't reconfigure tenant on unprotected branch pushes https://review.openstack.org/576538 | 14:37 |
| *** hamzy_ is now known as hamzy | 14:37 | |
| *** heyongli has quit IRC | 14:38 | |
| *** heyongli has joined #openstack-infra | 14:38 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Blackhole messages to ambassadors-owner on lists https://review.openstack.org/576539 | 14:39 |
| fungi | #status log lists.openstack.org added to emergency disable list until https://review.openstack.org/576539 merges | 14:40 |
| openstackstatus | fungi: finished logging | 14:40 |
| *** udesale has quit IRC | 14:42 | |
| *** udesale has joined #openstack-infra | 14:42 | |
| *** felipemonteiro__ has quit IRC | 14:43 | |
| *** sthussey has joined #openstack-infra | 14:44 | |
| *** yamamoto has joined #openstack-infra | 14:46 | |
| *** heyongli has quit IRC | 14:48 | |
| *** heyongli has joined #openstack-infra | 14:48 | |
| *** felipemonteiro has joined #openstack-infra | 14:49 | |
| *** yamamoto has quit IRC | 14:50 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Add NODEPOOL_PERCONA_PROXY to mirror_info https://review.openstack.org/576156 | 14:52 |
| *** biggles has joined #openstack-infra | 14:52 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul master: Don't reconfigure tenant on unprotected branch pushes https://review.openstack.org/576538 | 14:53 |
| *** ccamacho has quit IRC | 14:53 | |
| clarkb | fungi: thoughts on https://review.openstack.org/#/c/576345/1 ? | 14:53 |
| *** slaweq has quit IRC | 14:53 | |
| *** biggles has quit IRC | 14:53 | |
| *** slaweq has joined #openstack-infra | 14:54 | |
| *** dtantsur is now known as dtantsur|afk | 14:54 | |
| *** nicolasbock has joined #openstack-infra | 14:56 | |
| *** esarault has joined #openstack-infra | 14:58 | |
| *** heyongli has quit IRC | 14:58 | |
| *** heyongli has joined #openstack-infra | 14:59 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul master: Add a CVE-2018-12557 release note https://review.openstack.org/576545 | 14:59 |
| fungi | looking | 14:59 |
| mordred | clarkb: is there a reason we'd want to not land that? | 14:59 |
| clarkb | mordred: if we weren't handling the 10 nodes well | 15:00 |
| clarkb | new cloud and all that. The mirror still looks good to me. But I'm just getting started this morning and unsure of how jobs are doing | 15:00 |
| *** yamamoto has joined #openstack-infra | 15:01 | |
| *** esarault_ has quit IRC | 15:01 | |
| clarkb | we probably want to go ahead with it and then watch it | 15:02 |
| mordred | ah - nod | 15:04 |
| mordred | and yeah | 15:04 |
| openstackgerrit | Merged openstack-infra/project-config master: Add packethost grafana graphs https://review.openstack.org/576323 | 15:04 |
| *** yamamoto has quit IRC | 15:05 | |
| *** zoli|afk is now known as zoli | 15:07 | |
| *** zoli is now known as zoli|wfh | 15:07 | |
| *** zoli|wfh is now known as zoli | 15:07 | |
| *** heyongli has quit IRC | 15:09 | |
| *** heyongli has joined #openstack-infra | 15:09 | |
| *** dpawlik has joined #openstack-infra | 15:11 | |
| *** dizquierdo has joined #openstack-infra | 15:15 | |
| *** zhangfei has quit IRC | 15:15 | |
| *** dpawlik has quit IRC | 15:16 | |
| *** yamamoto has joined #openstack-infra | 15:16 | |
| *** r-daneel has joined #openstack-infra | 15:18 | |
| *** heyongli has quit IRC | 15:19 | |
| *** heyongli has joined #openstack-infra | 15:19 | |
| *** rossella_s has quit IRC | 15:19 | |
| clarkb | I figured ^ was a good first step | 15:20 |
| *** yamamoto has quit IRC | 15:20 | |
| clarkb | mordred: any idea what the process is to update the zuul app perms for third party check so that comments with recheck in them work? | 15:21 |
| clarkb | mordred: kata was willing to update those last night but not sure if the process was sorted out (it was late and I went to bed) | 15:21 |
| *** diablo_rojo_phon has joined #openstack-infra | 15:21 | |
| clarkb | mordred: sebastien.boeuf would be interested in them or I can play telephone if you know what it is off the top of your head | 15:21 |
| mordred | clarkb: I _think_ they should have an email in their inbox somewhere requesting that they accept new permissions? | 15:22 |
| clarkb | hrm I wonder if those all go to jbryce | 15:23 |
| clarkb | my suggestion was to remove the app and add it again | 15:23 |
| clarkb | (I don't know if that happened) | 15:23 |
| fungi | one thing i noticed about the zuul gh app behavior... even after completion the details link still goes to the status dashboard. is that supposed to switch to become a link to the published logs instead? | 15:24 |
| clarkb | oh actually looks like it worked so maybe they did get it sorted | 15:24 |
| *** Bhujay has quit IRC | 15:24 | |
| *** jesslampe has quit IRC | 15:24 | |
| clarkb | fungi: I think that is a known issue, we leave the logs link in a proper comment | 15:24 |
| clarkb | (which is not how most github ci things appear to work) | 15:24 |
| fungi | yeah, where "proper" comment is not proper from the perspective of most people used to other gh ci apps | 15:25 |
| clarkb | tobiash and spamaps probably have more insight into that | 15:25 |
| *** janki has joined #openstack-infra | 15:25 | |
| clarkb | fungi: mordred: you good for me to approve the max-servers bump in packet host? I can keep an eye on things but helpful if others can too | 15:26 |
| fungi | seems like the typical gh-using project would probably be less keen on having their pr discussions littered with comments from robots | 15:26 |
| mordred | clarkb: ++ | 15:26 |
| fungi | clarkb: sure, i just now +2d | 15:26 |
| clarkb | fungi: ya these sorts of things are why I wanted a mostly working setup that kata can poke at | 15:26 |
| mordred | clarkb, fungi: one of the goals/thoughts is to have a 'change' page in the dashboard that we can link to - that can have status, streaming and logs depending on lifecycle | 15:27 |
| clarkb | ok approved | 15:27 |
| fungi | especially given we've hacked our gerrit webui to hide bot comments from view | 15:27 |
| *** heyongli has quit IRC | 15:29 | |
| *** pcaruana has quit IRC | 15:29 | |
| *** heyongli has joined #openstack-infra | 15:29 | |
| fungi | clarkb: should we consider packethost/platform9 for the donors logo wall even though we're getting ever so slightly shy of 100 nodes? also i wonder which of their logos it should be... | 15:29 |
| clarkb | fungi: john may have insight as to which logo. Maybwe we can do logo1+logo2 somehow without violating trademark | 15:30 |
| *** yamamoto has joined #openstack-infra | 15:31 | |
| clarkb | fungi: as for logo wall, I'm inclined to say its probably fine simply because the aggregate is what we ask for | 15:31 |
| clarkb | we just reduce typical usage to allow for booting a mirror and test instances as necessary | 15:31 |
| mnaser | could an infra-root check logs for iad rax mirror if hits for '/percona/apt/' starting coming in properly? (cache) | 15:31 |
| *** felipemonteiro has quit IRC | 15:31 | |
| clarkb | mnaser: ya, one sec | 15:32 |
| *** felipemonteiro has joined #openstack-infra | 15:32 | |
| clarkb | mnaser: [19/Jun/2018:15:31:07 +0000] "GET /percona/apt/ HTTP/1.1" 200 907 No Last-Modified; Etag; Expires; Cache-Control:max-age or Cache-Control:s-maxage headers | 15:33 |
| clarkb | which happend after [19/Jun/2018:15:30:56 +0000] "GET /percona/apt HTTP/1.1" 301 178 cache miss: attempting entity save "-" | 15:33 |
| clarkb | I think that may mean it couldn't save to the cache due to lack of the above properties. Try a get on one of the package files and that should be different though | 15:34 |
| *** esarault_ has joined #openstack-infra | 15:34 | |
| clarkb | (since it should have at least last modified if mod auto indexing) | 15:34 |
| *** yamamoto has quit IRC | 15:35 | |
| mnaser | clarkb: those two requests were mine | 15:35 |
| *** pbourke has quit IRC | 15:35 | |
| mnaser | a bit before that, a job should have completed | 15:35 |
| mnaser | so made some hits to 'Packages' and downloading apts | 15:36 |
| clarkb | mnaser: ya those show up, but all were cache misses as they were the first ones | 15:36 |
| clarkb | mnaser: no issues with lacking headers though | 15:36 |
| clarkb | mnaser: I think next time through it should be cache hits for those packages | 15:36 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/zuul master: Add a CVE-2018-12557 release note https://review.openstack.org/576545 | 15:36 |
| mnaser | okay cool, i just wanted to verify because i noticed "/percona/apt" did a301 to their actual domain | 15:36 |
| clarkb | [19/Jun/2018:15:25:00 +0000] "GET /percona/apt/pool/main/p/percona-toolkit/percona-toolkit_3.0.10-1.bionic_amd64.deb HTTP/1.1" 200 4242658 cache miss: attempting entity save "-" "Debian APT-HTTP/1.3 (1.6.1)" | 15:36 |
| mnaser | so i was worried that it was just 301-ing everything | 15:36 |
| *** esarault has quit IRC | 15:37 | |
| mnaser | but if that's the case then that's cool, we should be good | 15:37 |
| clarkb | is an example. Looks like 3 packages were grabbed | 15:37 |
| mnaser | yeah we just grab those 3 usually only | 15:37 |
| *** pbourke has joined #openstack-infra | 15:38 | |
| clarkb | http://grafana.openstack.org/dashboard/db/nodepool-packethost is live now | 15:38 |
| *** manjeets has joined #openstack-infra | 15:39 | |
| *** heyongli has quit IRC | 15:39 | |
| *** heyongli has joined #openstack-infra | 15:40 | |
| fungi | i guess bionic only has e.g. percona-toolkit 3.0.6 and they need 3.0.10 backported to it (only available in cosmic at the moment)? | 15:40 |
| *** dklyle has joined #openstack-infra | 15:41 | |
| *** sshnaidm|afk is now known as sshnaidm | 15:41 | |
| *** anteaya has joined #openstack-infra | 15:42 | |
| *** felipemonteiro has quit IRC | 15:42 | |
| *** ykarel is now known as ykarel|away | 15:43 | |
| *** felipemonteiro has joined #openstack-infra | 15:44 | |
| fungi | mordred: oh, btw launch-node.py is working now with our respective fixes in place (other than us no longer having credentials to perform the rackdns api calls) | 15:44 |
| *** e0ne has quit IRC | 15:44 | |
| *** e0ne has joined #openstack-infra | 15:44 | |
| clarkb | fungi: those api calls used a separate api key instead of the password like our other rc files did/do? | 15:44 |
| fungi | right | 15:45 |
| clarkb | huh | 15:45 |
| fungi | api key tied to the old "openstack" user | 15:45 |
| fungi | our new "openstackinfra" user doesn't seem to be allowed to issue a new api key either | 15:45 |
| clarkb | jimmymcarthur was who set up the new user, may have insight into what permissions were delegated | 15:45 |
| fungi | jamesmcarthur: ^ | 15:45 |
| clarkb | er that is why it didn't tab complete | 15:46 |
| *** yamamoto has joined #openstack-infra | 15:46 | |
| fungi | that's his suit-and-tie name | 15:46 |
| *** dklyle has quit IRC | 15:46 | |
| *** jesslampe has joined #openstack-infra | 15:48 | |
| *** heyongli has quit IRC | 15:50 | |
| *** heyongli has joined #openstack-infra | 15:50 | |
| *** yamamoto has quit IRC | 15:50 | |
| *** felipemonteiro has quit IRC | 15:51 | |
| *** hamzy has quit IRC | 15:51 | |
| *** felipemonteiro has joined #openstack-infra | 15:51 | |
| *** ykarel|away has quit IRC | 15:54 | |
| cmurphy | got some passing puppet 4 beaker tests ^.^ https://review.openstack.org/#/c/571987/ | 16:00 |
| *** heyongli has quit IRC | 16:00 | |
| *** heyongli has joined #openstack-infra | 16:00 | |
| clarkb | oh right I reviewed like half the open changes then stopped because they were all one class of update and didn't want to juggle more than one thing if they broke | 16:02 |
| clarkb | I'll try to rereview again today | 16:02 |
| cmurphy | ya no worries i was working on other things | 16:03 |
| cmurphy | found out that the ipaddress6 fact evaluates differently between puppet 3 and puppet 4 so that's fun | 16:03 |
| *** jcoufal has quit IRC | 16:04 | |
| fungi | bizarre is what i'd call it | 16:05 |
| cmurphy | yeah by fun i meant "fun" | 16:05 |
| *** myoung is now known as myoung|lunch | 16:05 | |
| *** dizquierdo has quit IRC | 16:05 | |
| *** rpioso is now known as rpioso|eat | 16:06 | |
| clarkb | oh packethost node bump deps on grafana cleanup hcange, that is why it isn't gating /me reviews that one too | 16:06 |
| clarkb | I have firefox render Gerrit at 133% on laptop and that pushes the dep list off to the right and I hvae to horizontal scroll to see them | 16:06 |
| *** jesslampe has quit IRC | 16:06 | |
| fungi | silly gerrits | 16:07 |
| *** sshnaidm is now known as sshnaidm|afk | 16:08 | |
| *** iranzo has quit IRC | 16:09 | |
| *** heyongli has quit IRC | 16:10 | |
| *** heyongli has joined #openstack-infra | 16:10 | |
| *** dklyle has joined #openstack-infra | 16:11 | |
| *** dhajare has joined #openstack-infra | 16:12 | |
| clarkb | fungi: also silly eyeballs. I really wish they would stabilize. I think currently I need new glasses every 6-9 months. So I suffer until the 12 month mark and immediately get a new pair | 16:13 |
| *** jesslampe has joined #openstack-infra | 16:13 | |
| fungi | cybernetic optical implants can't come soon enough | 16:14 |
| *** yamamoto has joined #openstack-infra | 16:15 | |
| anteaya | funny how the solution to the effect of technology is more technology | 16:15 |
| openstackgerrit | Merged openstack-infra/project-config master: nodepool grafana : remove job runtimes https://review.openstack.org/576339 | 16:15 |
| openstackgerrit | Merged openstack-infra/project-config master: Bump packethost nodes to 95 https://review.openstack.org/576345 | 16:15 |
| *** armaan has quit IRC | 16:18 | |
| clarkb | there we go | 16:19 |
| *** armaan has joined #openstack-infra | 16:19 | |
| *** knikolla has quit IRC | 16:19 | |
| *** heyongli has quit IRC | 16:20 | |
| *** yamamoto has quit IRC | 16:20 | |
| *** heyongli has joined #openstack-infra | 16:21 | |
| *** jpich has quit IRC | 16:21 | |
| SpamapS | fungi: clarkb I had meant to update the gh pipelines to have success status urls but got distracted | 16:22 |
| SpamapS | One odd thing is that you have to reconstruct the log url in the pipeline def | 16:23 |
| SpamapS | status-url: &log_link "https://zuul.cloud.int.godaddy.com:8080/logs/{tenant.name}/{pipeline.name}/{change.project.canonical_name}/{change.number}/{buildset.uuid}/" | 16:23 |
| SpamapS | That's how I do it in our internal system. | 16:23 |
| SpamapS | would be nice if the zuul_result vars were available in pipeline. | 16:24 |
| fungi | oh, yeah that would make it simpler | 16:25 |
| SpamapS | and that thought is what distracted me | 16:25 |
| * SpamapS has come full circle | 16:25 | |
| SpamapS | I presume those vars make it back to the scheduler since they can inform child jobs. | 16:26 |
| *** felipemonteiro has quit IRC | 16:26 | |
| *** panda is now known as panda|off | 16:26 | |
| clarkb | nodepool is using new packethost max-servers now | 16:27 |
| *** yamamoto has joined #openstack-infra | 16:30 | |
| *** heyongli has quit IRC | 16:31 | |
| *** felipemonteiro has joined #openstack-infra | 16:31 | |
| clarkb | the error rate is climbing, lets see if that becomes more consistent once images are in place | 16:31 |
| *** heyongli has joined #openstack-infra | 16:31 | |
| *** armaan has quit IRC | 16:32 | |
| *** armaan has joined #openstack-infra | 16:32 | |
| *** kjackal_ has quit IRC | 16:32 | |
| *** kjackal has quit IRC | 16:32 | |
| dhellmann | clarkb : I tried to include the steps about moving zuul configuration out of project-config into each project tree when I wrote up the python3 goal for stein. I would appreciate it if you would look over that stuff and make sure I have all of the details correct. https://review.openstack.org/#/c/575933/ | 16:32 |
| clarkb | dhellmann: noted | 16:33 |
| dhellmann | clarkb: I'm putting together a team to help create the patches, so if you have time in the next week or so that would be good. I would hate for us to write scripts that do the wrong thing. :-) | 16:33 |
| *** yamahata has quit IRC | 16:34 | |
| *** yamamoto has quit IRC | 16:35 | |
| mordred | dhellmann: it looks good on first pass - but I didn't dig deeply yet | 16:38 |
| clarkb | dhellmann: I left a couple of notes, but +1 I don't see any issues with what you have written | 16:39 |
| *** heyongli has quit IRC | 16:41 | |
| dhellmann | mordred , clarkb : thank you both | 16:41 |
| *** heyongli has joined #openstack-infra | 16:41 | |
| *** udesale has quit IRC | 16:41 | |
| *** jamesmcarthur has quit IRC | 16:43 | |
| *** gfidente has quit IRC | 16:43 | |
| *** yamamoto has joined #openstack-infra | 16:45 | |
| *** gyee has joined #openstack-infra | 16:46 | |
| *** sambetts is now known as sambetts|afk | 16:47 | |
| *** yamamoto has quit IRC | 16:50 | |
| *** heyongli has quit IRC | 16:51 | |
| tobiash | clarkb, fungi: we could also just set the status link but not sure right now if zuul also can change that link with the result | 16:51 |
| *** heyongli has joined #openstack-infra | 16:51 | |
| tobiash | But that might already work if you can derive the log location from the item properties | 16:52 |
| *** auristor has quit IRC | 16:54 | |
| *** felipemonteiro has quit IRC | 16:56 | |
| *** jamesmca_ has quit IRC | 16:57 | |
| *** jamesmcarthur has joined #openstack-infra | 16:57 | |
| *** yamahata has joined #openstack-infra | 16:57 | |
| *** auristor has joined #openstack-infra | 16:57 | |
| *** dpawlik has joined #openstack-infra | 16:59 | |
| *** kjackal_ has joined #openstack-infra | 16:59 | |
| *** kjackal has joined #openstack-infra | 16:59 | |
| *** esarault has joined #openstack-infra | 16:59 | |
| *** derekh has quit IRC | 17:00 | |
| *** dougsz has quit IRC | 17:00 | |
| *** Deknos has quit IRC | 17:00 | |
| *** yamamoto has joined #openstack-infra | 17:01 | |
| *** heyongli has quit IRC | 17:01 | |
| *** Goneri has quit IRC | 17:01 | |
| *** heyongli has joined #openstack-infra | 17:02 | |
| *** jamesmcarthur has quit IRC | 17:02 | |
| *** dpawlik has quit IRC | 17:03 | |
| *** esarault_ has quit IRC | 17:03 | |
| *** pcaruana has joined #openstack-infra | 17:05 | |
| *** yamamoto has quit IRC | 17:05 | |
| *** zoli is now known as zoli|gone | 17:06 | |
| *** zoli|gone is now known as zoli | 17:06 | |
| *** felipemonteiro has joined #openstack-infra | 17:07 | |
| *** tesseract has quit IRC | 17:08 | |
| *** felipemonteiro has quit IRC | 17:08 | |
| *** ramishra has quit IRC | 17:09 | |
| *** jamesmcarthur has joined #openstack-infra | 17:10 | |
| *** jamesmcarthur has quit IRC | 17:10 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/openstack-zuul-jobs master: remove announce and constraint jobs from pre-release pipeline https://review.openstack.org/576589 | 17:11 |
| *** jamesmcarthur has joined #openstack-infra | 17:11 | |
| *** heyongli has quit IRC | 17:12 | |
| *** jamesmca_ has joined #openstack-infra | 17:12 | |
| *** heyongli has joined #openstack-infra | 17:12 | |
| *** lifeless_ has joined #openstack-infra | 17:12 | |
| *** lifeless has quit IRC | 17:12 | |
| *** dhajare has quit IRC | 17:14 | |
| *** Goneri has joined #openstack-infra | 17:15 | |
| *** yamamoto has joined #openstack-infra | 17:15 | |
| *** jamesmcarthur has quit IRC | 17:16 | |
| tobiash | SpamapS: do you use just that or combine it with the comments? | 17:16 |
| SpamapS | tobiash: I do not comment on success | 17:16 |
| SpamapS | only on fail | 17:16 |
| SpamapS | users hate success comments | 17:16 |
| tobiash | that's a good compromise | 17:17 |
| *** e0ne has quit IRC | 17:19 | |
| *** yamamoto has quit IRC | 17:20 | |
| *** heyongli has quit IRC | 17:22 | |
| *** heyongli has joined #openstack-infra | 17:22 | |
| *** trown is now known as trown|lunch | 17:27 | |
| *** bobh has quit IRC | 17:27 | |
| *** rpioso|eat is now known as rpioso | 17:27 | |
| *** bobh has joined #openstack-infra | 17:27 | |
| *** e0ne has joined #openstack-infra | 17:29 | |
| *** esarault_ has joined #openstack-infra | 17:29 | |
| *** yamamoto has joined #openstack-infra | 17:31 | |
| *** heyongli has quit IRC | 17:32 | |
| *** heyongli has joined #openstack-infra | 17:32 | |
| *** myoung|lunch is now known as myoung | 17:32 | |
| *** esarault has quit IRC | 17:33 | |
| *** e0ne has quit IRC | 17:34 | |
| *** yamamoto has quit IRC | 17:35 | |
| *** jamesmca_ has quit IRC | 17:35 | |
| *** hamzy has joined #openstack-infra | 17:36 | |
| *** jamesmcarthur has joined #openstack-infra | 17:36 | |
| *** r-daneel has quit IRC | 17:37 | |
| *** Goneri has quit IRC | 17:38 | |
| *** jamesmcarthur has quit IRC | 17:41 | |
| clarkb | as a heads up I'm meeting with some kata folks via Zoom at 1:30pm PST today to talk Zuul with mnaser. I think that is 2030UTC if anyone else is interested | 17:42 |
| fungi | so half an hour after the infra meeting ends | 17:42 |
| clarkb | yup | 17:42 |
| *** heyongli has quit IRC | 17:42 | |
| fungi | tempting but i'll be wrists deep in crab, making crabcakes | 17:42 |
| *** heyongli has joined #openstack-infra | 17:43 | |
| clarkb | ooh | 17:44 |
| fungi | we get tubs of lump crab meat really cheap here (perhaps unsurprisingly) | 17:44 |
| fungi | reduces the effort to near zero | 17:45 |
| *** yamamoto has joined #openstack-infra | 17:45 | |
| fungi | clarkb: confirmed, our openstackci user seems to also have similar challenges with the "get an api key" link, so maybe it's that the rackspace dashboard has gotten even more opaque | 17:47 |
| clarkb | anyone have the winterstack naming etherpad link handy? I'm on laptop and want to avoid x forwarding my desktop firefox to get it | 17:49 |
| fungi | https://etherpad.openstack.org/p/infra-services-naming | 17:49 |
| clarkb | tyty | 17:49 |
| fungi | it has to clarkbs now | 17:49 |
| fungi | we, two | 17:49 |
| * fungi gets back to mistyping other things besides in irc | 17:50 | |
| *** yamamoto has quit IRC | 17:50 | |
| *** boris_42_ has joined #openstack-infra | 17:52 | |
| *** heyongli has quit IRC | 17:53 | |
| *** heyongli has joined #openstack-infra | 17:53 | |
| *** amoralej is now known as amoralej|off | 17:53 | |
| *** shardy has quit IRC | 17:54 | |
| *** hongbin has quit IRC | 17:56 | |
| * cmurphy will miss the meeting | 17:59 | |
| *** jamesmcarthur has joined #openstack-infra | 18:00 | |
| *** yamamoto has joined #openstack-infra | 18:00 | |
| clarkb | cmurphy: watching Russia and Egypt? | 18:01 |
| *** heyongli has quit IRC | 18:03 | |
| clarkb | fungi: I've put mailman3,puppet,bionic,debian up as a meeting topic, but thought I'd double check that is worth talking about at this point | 18:03 |
| *** heyongli has joined #openstack-infra | 18:03 | |
| fungi | clarkb: sure, i have some notes summarized at https://etherpad.openstack.org/p/mm3poc we can link too | 18:04 |
| clarkb | I think it may help inform the config mgmt update spec and so is worth talking about early | 18:04 |
| clarkb | even if we are just feeling our way through it to start | 18:04 |
| fungi | i'm just about to watch our puppetmaster attempt to use puppet 3.8.5 to configure an ubuntu bionic server | 18:05 |
| *** yamamoto has quit IRC | 18:05 | |
| fungi | [edge of seat] | 18:05 |
| *** Goneri has joined #openstack-infra | 18:08 | |
| *** dklyle has quit IRC | 18:10 | |
| *** Elachance has joined #openstack-infra | 18:12 | |
| *** heyongli has quit IRC | 18:13 | |
| *** heyongli has joined #openstack-infra | 18:13 | |
| mordred | fungi: good luck! | 18:15 |
| *** yamamoto has joined #openstack-infra | 18:15 | |
| *** yamamoto has quit IRC | 18:20 | |
| Elachance | @mordred -- you have a sec to look at the senlin changes to shade again? eandersson made some changes that he thinks should make it good to merge: https://review.openstack.org/#/c/542537/ | 18:21 |
| *** heyongli has quit IRC | 18:23 | |
| *** heyongli has joined #openstack-infra | 18:24 | |
| *** jamesmca_ has joined #openstack-infra | 18:26 | |
| *** electrofelix has quit IRC | 18:27 | |
| *** esarault has joined #openstack-infra | 18:28 | |
| *** bobh has quit IRC | 18:28 | |
| *** trown|lunch is now known as trown | 18:28 | |
| *** jamesmcarthur has quit IRC | 18:30 | |
| *** camunoz has quit IRC | 18:30 | |
| mordred | Elachance: I rebased it and pushed it back up to see how it does with testin gnow | 18:30 |
| *** caphrim007 has joined #openstack-infra | 18:30 | |
| *** yamamoto has joined #openstack-infra | 18:30 | |
| Elachance | mordered: thank you! | 18:31 |
| Elachance | *mordred, sorry! | 18:31 |
| *** esarault_ has quit IRC | 18:31 | |
| *** e0ne has joined #openstack-infra | 18:31 | |
| *** janki has quit IRC | 18:32 | |
| *** dklyle has joined #openstack-infra | 18:32 | |
| *** heyongli has quit IRC | 18:34 | |
| fungi | mmm, puppet 3 on bionic seems to explode thusly: http://paste.openstack.org/show/723851 | 18:34 |
| *** heyongli has joined #openstack-infra | 18:34 | |
| *** yamamoto has quit IRC | 18:35 | |
| clarkb | fungi: ruby version too new? | 18:35 |
| *** yamamoto has joined #openstack-infra | 18:35 | |
| *** yamamoto has quit IRC | 18:36 | |
| fungi | that's my guess | 18:37 |
| *** hamzy_ has joined #openstack-infra | 18:39 | |
| *** hamzy has quit IRC | 18:39 | |
| *** heyongli has quit IRC | 18:44 | |
| *** jamesmca_ has quit IRC | 18:44 | |
| *** heyongli has joined #openstack-infra | 18:44 | |
| *** iranzo has joined #openstack-infra | 18:45 | |
| *** camunoz has joined #openstack-infra | 18:45 | |
| fungi | poking around, we don't actually have any systems using puppet 4 do we? | 18:49 |
| clarkb | fungi: just test jobs at this point I think | 18:49 |
| mordred | fungi: not yet - but maybe it's worth a try for a new system? | 18:49 |
| fungi | the puppetlabs packages for trusty are 3 and for xenial/bionic are 5. the distro packages for xenial are 3 and bionic are 5 | 18:49 |
| fungi | not sure how we'd actually install puppet 4 anywhere? | 18:50 |
| fungi | oh! i take that back. the puppetlabs rpms for centos 7 are puppet 4, not that it helps me much | 18:50 |
| mordred | fungi: https://review.openstack.org/#/c/481943/ | 18:50 |
| clarkb | fungi: we get our puppet4 on xenial from puppetlabs | 18:50 |
| clarkb | iirc | 18:50 |
| mordred | oh - I don't think that helps | 18:51 |
| clarkb | fungi: system-config/install_puppet.sh should show you how that happens | 18:51 |
| mordred | fungi: https://review.openstack.org/#/c/572218/ ... | 18:51 |
| fungi | http://apt.puppetlabs.com/pool/xenial/puppet/p/puppet-agent/ seems to all be puppet 5 | 18:51 |
| mordred | and yes, it seems install_puppet.sh with PUPPET_VERSION=4 should do it | 18:51 |
| *** camunoz has quit IRC | 18:52 | |
| fungi | oh, http://apt.puppetlabs.com/pool/xenial/PC1/ is maybe puppet 4 | 18:52 |
| *** mihalis68 has quit IRC | 18:53 | |
| mordred | fungi: yah - I think you're right | 18:54 |
| *** heyongli has quit IRC | 18:54 | |
| *** heyongli has joined #openstack-infra | 18:54 | |
| fungi | okay, puppet --version now reports 4.10.12 on this bionic vm and doesn't just asplode | 18:56 |
| clarkb | Meeting time in a few minutes | 18:56 |
| fungi | waiting to see if the puppetmaster can manage it now | 18:56 |
| fungi | but my hopes have increased at least ever so slightly | 18:57 |
| *** esarault_ has joined #openstack-infra | 18:58 | |
| *** r-daneel has joined #openstack-infra | 19:00 | |
| *** bobh has joined #openstack-infra | 19:02 | |
| *** esarault has quit IRC | 19:03 | |
| *** camunoz has joined #openstack-infra | 19:04 | |
| *** heyongli has quit IRC | 19:04 | |
| *** r-daneel_ has joined #openstack-infra | 19:05 | |
| *** heyongli has joined #openstack-infra | 19:05 | |
| *** r-daneel has quit IRC | 19:05 | |
| *** r-daneel_ is now known as r-daneel | 19:05 | |
| openstackgerrit | Merged openstack-infra/system-config master: Add playbook to upgrade puppet https://review.openstack.org/572218 | 19:11 |
| *** esarault_ has quit IRC | 19:12 | |
| *** r-daneel_ has joined #openstack-infra | 19:13 | |
| *** r-daneel has quit IRC | 19:14 | |
| *** r-daneel_ is now known as r-daneel | 19:14 | |
| *** felipemonteiro has joined #openstack-infra | 19:14 | |
| *** heyongli has quit IRC | 19:15 | |
| *** heyongli has joined #openstack-infra | 19:15 | |
| *** dpawlik has joined #openstack-infra | 19:15 | |
| *** dpawlik has quit IRC | 19:16 | |
| *** e0ne has quit IRC | 19:16 | |
| *** dpawlik has joined #openstack-infra | 19:16 | |
| *** e0ne has joined #openstack-infra | 19:17 | |
| ianw | packethost-us-west-1-main]: Predicted remaining tenant quota: {'compute': {'cores': 536, 'instances': 67, 'ram': 8118272}} | 19:21 |
| ianw | so i think it's reporting ok, and nodepool knows it's ok | 19:22 |
| fungi | so the elastic quota calculation doesn't seem to think we're overrunning it | 19:22 |
| clarkb | thats good, probably just lack of demand then | 19:23 |
| *** dpawlik has quit IRC | 19:23 | |
| *** e0ne has quit IRC | 19:24 | |
| *** heyongli has quit IRC | 19:25 | |
| *** heyongli has joined #openstack-infra | 19:25 | |
| fungi | maybe we need to hurry up and fix tripleo so we can see what happens at max capacity again ;) | 19:25 |
| ianw | http://grafana.openstack.org/dashboard/db/zuul-status seems to be on a very big downward slope ... is it just expected EOD ? | 19:26 |
| fungi | tripleo hasn't been approving any changes today per a ml thread i saw earlier | 19:31 |
| fungi | and they were driving a lot (most?) of the test load earlier | 19:32 |
| *** heyongli has quit IRC | 19:35 | |
| *** heyongli has joined #openstack-infra | 19:36 | |
| *** yamamoto has joined #openstack-infra | 19:36 | |
| dtroyer | wiki question: does wiki.o.o have a restriction on uploading images? I've got a new-ish user, it says he need to log in to upload, he says he is logged in… | 19:39 |
| *** e0ne has joined #openstack-infra | 19:39 | |
| clarkb | dtroyer: I want to say you have to be a in a trusted group of users, this is fallout of all the spamming that was happening | 19:40 |
| clarkb | fungi: ^ is that right? | 19:40 |
| dtroyer | I wondered if that might be the case | 19:40 |
| dtroyer | it seems like a logical precaution to take | 19:40 |
| *** yamamoto has quit IRC | 19:42 | |
| *** isq_ has quit IRC | 19:43 | |
| *** slaweq has quit IRC | 19:43 | |
| *** slaweq has joined #openstack-infra | 19:44 | |
| *** iranzo has quit IRC | 19:44 | |
| fungi | yes, the account needs to have been vetted by one of the volunteer patrollers for the wiki | 19:45 |
| *** heyongli has quit IRC | 19:45 | |
| *** heyongli has joined #openstack-infra | 19:46 | |
| fungi | that user's edits get marked for patrolling and our patrollers periodically look at the flagged edits, make sure they're legitimate, and then usually add the corresponding users to a verified users group which is then autopatrolled from that point on. we also rely on membership in that verified users group to also allow things like moving/renaming/deleting pages and uploading images | 19:46 |
| *** anteaya has quit IRC | 19:49 | |
| *** isq has joined #openstack-infra | 19:49 | |
| dtroyer | fungi: thanks. This is for someone setting up the starlingx docs, can I wave a warm cinnamon roll in front of someone to help? | 19:50 |
| clarkb | dtroyer: if you get their wiki username one of us here can probably add them to the correct group | 19:50 |
| clarkb | (or they should feel free to join the channel and we can work direclty ) | 19:51 |
| dtroyer | my initial response includes that suggestion… | 19:51 |
| dtroyer | thanks | 19:51 |
| fungi | yeah, sorry, just buried under simultaneous conversations at the moment | 19:52 |
| dtroyer | no worries | 19:53 |
| *** heyongli has quit IRC | 19:56 | |
| *** heyongli has joined #openstack-infra | 19:56 | |
| fungi | dtroyer: i see a starlingx page edit from an austin.sun | 19:59 |
| *** dklyle has quit IRC | 20:00 | |
| fungi | oh, also from a dtroyer who hasn't been verified either | 20:00 |
| dtroyer | hmmm… not sure, it is David Kinder that is adding stuff | 20:00 |
| dtroyer | that dtroyer guy is suspect | 20:00 |
| fungi | i'll have to go back more than a few days in the recent changes list, looks like | 20:00 |
| dtroyer | I thought he was doing it today, I'm trying to find him on internal chat | 20:01 |
| fungi | aha, found | 20:02 |
| *** anteaya has joined #openstack-infra | 20:02 | |
| fungi | https://wiki.openstack.org/wiki/Special:Contributions/David.b.kinder | 20:02 |
| dtroyer | yup | 20:02 |
| fungi | also it looks like none of our patrollers are patrolling edits any longer. i see some spam edits from a week ago which haven't been rolled back yet | 20:03 |
| fungi | and tons of legit edits from non-verified users | 20:03 |
| fungi | he's a verified user now | 20:04 |
| fungi | as are you now | 20:05 |
| *** heyongli has quit IRC | 20:06 | |
| dtroyer | fungi: thanks! | 20:06 |
| *** heyongli has joined #openstack-infra | 20:06 | |
| fungi | no problem | 20:07 |
| *** Goneri has quit IRC | 20:08 | |
| *** kjackal_ has quit IRC | 20:09 | |
| *** kjackal has quit IRC | 20:09 | |
| *** priteau has quit IRC | 20:12 | |
| clarkb | ianw: fungi shade.exc.OpenStackCloudHTTPError: (403) Client Error for url: https://packet.platform9.net/nova/v2.1/ba163363643541d88a737f8af23cfa7f/servers Maximum number of ports exceeded I think that is what is limiting us in packethost cloud | 20:16 |
| *** heyongli has quit IRC | 20:16 | |
| *** heyongli has joined #openstack-infra | 20:16 | |
| fungi | oho | 20:16 |
| ianw | hmm ports | 20:17 |
| fungi | port quota or leakage | 20:17 |
| clarkb | logged by nodepool | 20:17 |
| *** felipemonteiro has quit IRC | 20:17 | |
| clarkb | limits show doesn't seem to have ports in it | 20:17 |
| clarkb | neutron quota-show says the docs | 20:19 |
| clarkb | oh thats newton docs | 20:19 |
| mnaser | clarkb: neutron quota-show --tenant-id <foo> | 20:20 |
| anteaya | clarkb, what time is the kata meeting scheduled? | 20:20 |
| clarkb | anteaya: in 10 minutes | 20:20 |
| clarkb | 2030UTC | 20:20 |
| anteaya | thanks | 20:20 |
| mnaser | clarkb: neutron quota-update --tenant-id <foo> --port -1 | 20:20 |
| clarkb | mnaser: is this only an admin thing? osc quota list --network says I am not authorized | 20:21 |
| mnaser | quota are admin thing yes | 20:21 |
| clarkb | this seems like something the user should be able to view themselves | 20:21 |
| mnaser | i thought you had admin access | 20:21 |
| mnaser | oh to view your current usage | 20:21 |
| clarkb | we do have admin, but I shouldn't need admin | 20:21 |
| ianw | | port | 50 | | 20:21 |
| mnaser | clarkb: you want osc quota show | 20:21 |
| anteaya | I can't seem to join, can one of you let me know once you have joined? | 20:21 |
| *** felipemonteiro has joined #openstack-infra | 20:22 | |
| clarkb | mnaser: that did it, thanks | 20:22 |
| clarkb | ianw: we probably want to bump that up to ~110 | 20:22 |
| mnaser | if you boot with two interfaces | 20:22 |
| clarkb | little overhead on top of max instance quota to accomodate nodepools periodic cleanup of leaked ports | 20:22 |
| clarkb | ianw osc quota set --ports 110 $project looks like | 20:23 |
| *** diablo_rojo has joined #openstack-infra | 20:24 | |
| ianw | | port | 100 | | 20:24 |
| ianw | alright, 110 ... let's see if it starts increasing | 20:24 |
| clarkb | ports can leak is the only reason I suggest it being higher, gives nodepool a chance to delete them before they become a problem | 20:25 |
| ianw | ok, | floatingip | 100 | too | 20:25 |
| *** felipemonteiro has quit IRC | 20:26 | |
| *** heyongli has quit IRC | 20:26 | |
| *** heyongli has joined #openstack-infra | 20:27 | |
| *** pcaruana has quit IRC | 20:29 | |
| *** felipemonteiro has joined #openstack-infra | 20:30 | |
| clarkb | mnaser: I am able to join now | 20:30 |
| ianw | clarkb: great find ... seems load is coming up in packethost now | 20:33 |
| *** kgiusti has left #openstack-infra | 20:33 | |
| *** hashar has quit IRC | 20:36 | |
| *** felipemonteiro has quit IRC | 20:36 | |
| *** heyongli has quit IRC | 20:37 | |
| *** heyongli has joined #openstack-infra | 20:37 | |
| *** yamamoto has joined #openstack-infra | 20:38 | |
| *** hemna_ has quit IRC | 20:39 | |
| *** jtomasek has quit IRC | 20:43 | |
| *** felipemonteiro has joined #openstack-infra | 20:45 | |
| *** e0ne has quit IRC | 20:46 | |
| *** heyongli has quit IRC | 20:47 | |
| *** yamamoto has quit IRC | 20:47 | |
| *** heyongli has joined #openstack-infra | 20:47 | |
| *** edmondsw has quit IRC | 20:47 | |
| *** armaan has quit IRC | 20:50 | |
| *** anteaya has quit IRC | 20:50 | |
| *** armaan has joined #openstack-infra | 20:51 | |
| *** camunoz has quit IRC | 20:51 | |
| *** anteaya has joined #openstack-infra | 20:51 | |
| *** hamzy_ has quit IRC | 20:54 | |
| *** heyongli has quit IRC | 20:57 | |
| *** heyongli has joined #openstack-infra | 20:57 | |
| cmurphy | so puppet 4 works on bionic huh? | 20:59 |
| *** trown is now known as trown|outtypewww | 21:01 | |
| *** felipemonteiro has quit IRC | 21:02 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Fixed wrong mapping on tracks endpoints https://review.openstack.org/576643 | 21:02 |
| clarkb | cmurphy: apparently | 21:03 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Fixed wrong mapping on tracks endpoints https://review.openstack.org/576643 | 21:03 |
| *** katyafervent_awa has quit IRC | 21:03 | |
| clarkb | kata meeting went well. They were looking for next steps. What we've roughly decided on is working with a person or two in kata to continue to update the jobs so that the correct logs are corrected and they pass. Then expand to centos and fedora (their other platforms) and then start running job(s) on one of the busier repos | 21:04 |
| clarkb | so far all of my work has been on one of the most idle repos so others in kata aren't really seeing the results | 21:04 |
| clarkb | then based on that we can decide if we move forward to long term zuulification and make their own tenant | 21:05 |
| clarkb | mnaser: anteaya ^ did I capture that ewll? | 21:05 |
| anteaya | that was what I got out of it | 21:05 |
| anteaya | that and there will be a learning curve as kata devs start consuming logs | 21:06 |
| *** r-daneel has quit IRC | 21:06 | |
| *** r-daneel has joined #openstack-infra | 21:06 | |
| *** katyafervent_awa has joined #openstack-infra | 21:06 | |
| *** heyongli has quit IRC | 21:07 | |
| *** heyongli has joined #openstack-infra | 21:08 | |
| fungi | cmurphy: puppet 4 _seems_ to work on bionic (using the puppetlabs pc1 packages marked for use on xenial) however we've still got some errors and deprecation warnings. i'll work up a paste | 21:08 |
| fungi | i'm guessing a lot of these are addressed by your changes though | 21:08 |
| fungi | cmurphy: http://paste.openstack.org/show/723861 | 21:10 |
| cmurphy | fungi: i think the main thing there is "Could not evaluate: Could not retrieve information from environment production source(s) puppet:///modules/openstack_project/sources.list.bionic.amd64" | 21:12 |
| cmurphy | though the 'error while resolving custom fact "python_version"' is also concerning | 21:12 |
| clarkb | ianw: I still see 50 ports on openstackzuul-packethost | 21:13 |
| clarkb | (also still seeing the port limit errors) | 21:13 |
| clarkb | did it revert maybe? | 21:13 |
| *** rh-jelabarre has quit IRC | 21:17 | |
| *** heyongli has quit IRC | 21:18 | |
| *** heyongli has joined #openstack-infra | 21:18 | |
| *** rfolco is now known as rfolco|off | 21:18 | |
| ianw | clarkb: hrmm, weird. yes back to 50, hrm | 21:19 |
| *** bobh has quit IRC | 21:20 | |
| ianw | clarkb: i've re-upped it and will keep an eye. i can't imagine the cloud-launcher is touching it; maybe something on the cloud side that resets things to defaults? | 21:23 |
| *** dpawlik has joined #openstack-infra | 21:24 | |
| clarkb | or platform9 enforcing quotas? | 21:24 |
| *** Swami has joined #openstack-infra | 21:24 | |
| clarkb | ianw: seems to be 50 again | 21:27 |
| clarkb | seems quick for it not to be cloud side | 21:27 |
| *** heyongli has quit IRC | 21:28 | |
| *** dpawlik has quit IRC | 21:28 | |
| *** heyongli has joined #openstack-infra | 21:28 | |
| *** anteaya has quit IRC | 21:29 | |
| ianw | i just reset it and got it running under watch | 21:32 |
| ianw | see if it drops at some regular period that suggests some sort of job ... | 21:32 |
| openstackgerrit | Kendall Nelson proposed openstack-infra/project-config master: Add Vitrage to StoryBoard https://review.openstack.org/576651 | 21:33 |
| dhellmann | corvus : iirc, a while back you said we no longer need to worry about the issue that caused memory problems when there were a lot of zuul configuration changes up for review at the same time. Is that correct? I'm thinking ahead to the migration of the project-config settings into project trees, and want to make sure that we batch those if we have to. | 21:34 |
| fungi | if someone ends up reaching out to them about this, it would probably also be a good opportunity to mention that we had to work around having the two accounts in the same tenant, and that we needed to create a custom flavor for the mirror because cinder seems to not be working | 21:34 |
| ianw | fungi: yeah i will send a mail with details, i need to update passwords.gpg too | 21:35 |
| *** nicolasbock has quit IRC | 21:35 | |
| clarkb | dhellmann: I believe that is the case http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=392&rra_id=all shows a dramatic change since config was fixed for memory use | 21:35 |
| ianw | you know, i wonder if *I* reset things. i used neutron to set the quota, and i think maybe it sets anything you're not setting back to default? | 21:35 |
| clarkb | dhellmann: corvus is traveling right now though and may not respond quickly | 21:35 |
| clarkb | ianw: that would be an interesting behavior | 21:36 |
| dhellmann | clarkb : ah, thanks, I remember seeing a note about his travel. and thanks for confirming the issue is gone | 21:36 |
| ianw | so when i upped the floating ip's, i reset the ports, and then when i upped the ports again, it lowered floating ips, ... and so on :) | 21:36 |
| clarkb | hahaha | 21:36 |
| *** heyongli has quit IRC | 21:38 | |
| clarkb | I feel like that is a bug | 21:38 |
| clarkb | but its likely because that is how the api works, it takes all the parameters, treats any unprovided as default and updates the lot | 21:38 |
| *** heyongli has joined #openstack-infra | 21:38 | |
| fungi | is this one of those json-blob neutron api methods? | 21:39 |
| mordred | well, they're all json-blob methods | 21:39 |
| mordred | oh - you mean cli-wise? | 21:40 |
| * mordred does not know | 21:40 | |
| fungi | er, yeah s/methods/commands/ | 21:40 |
| clarkb | in use: 84 says grafana \o/ | 21:40 |
| *** yamamoto has joined #openstack-infra | 21:43 | |
| *** yamamoto has quit IRC | 21:48 | |
| *** heyongli has quit IRC | 21:48 | |
| *** heyongli has joined #openstack-infra | 21:49 | |
| clarkb | no recent errors either | 21:51 |
| *** caphrim007_ has joined #openstack-infra | 21:58 | |
| *** heyongli has quit IRC | 21:59 | |
| *** heyongli has joined #openstack-infra | 21:59 | |
| *** lifeless has joined #openstack-infra | 21:59 | |
| *** myoung is now known as myoung|off | 21:59 | |
| *** lifeless_ has quit IRC | 21:59 | |
| *** caphrim007 has quit IRC | 22:01 | |
| *** dizquierdo has joined #openstack-infra | 22:01 | |
| *** slaweq has quit IRC | 22:04 | |
| *** jrist has quit IRC | 22:05 | |
| *** r-daneel has quit IRC | 22:07 | |
| *** heyongli has quit IRC | 22:09 | |
| *** heyongli has joined #openstack-infra | 22:09 | |
| *** florianf has quit IRC | 22:10 | |
| *** rcernin has joined #openstack-infra | 22:12 | |
| * clarkb pulls up topic:puppet-4 | 22:15 | |
| *** r-daneel has joined #openstack-infra | 22:16 | |
| clarkb | hostvars[inventory_hostname].group_names is the same as just using group_names directly right? | 22:18 |
| clarkb | since hostvars[inventory_hostname] is the set of vars you see by default? | 22:18 |
| *** slaweq has joined #openstack-infra | 22:18 | |
| * clarkb rolls with it and approves https://review.openstack.org/#/c/572861/1 I'll watch puppet_run_all.log after that merges | 22:18 | |
| *** heyongli has quit IRC | 22:19 | |
| *** heyongli has joined #openstack-infra | 22:19 | |
| clarkb | https://review.openstack.org/#/c/564884/ https://review.openstack.org/#/c/564860/ and https://review.openstack.org/#/c/564882/ should be easy ones for other reviewers | 22:19 |
| clarkb | https://review.openstack.org/#/c/567095/ too | 22:20 |
| *** anteaya has joined #openstack-infra | 22:21 | |
| ianw | clarkb: when you're done, one from yesterday https://review.openstack.org/#/c/576351/ ... we ended up writing ~ the same change to debug the traceroute output more. can we decide on one or the other and get it in? | 22:21 |
| *** kzaitsev_pi has quit IRC | 22:23 | |
| clarkb | ianw: sure, on my change people wanted testing. So I did a bunch of work to split things for testing and realized it would require changes to zuul jobs and it sort of died :/ | 22:23 |
| ianw | oh yeah, i don't think the testing story is sorted | 22:23 |
| clarkb | thta was the blocker at the time aiui | 22:24 |
| *** dizquierdo has quit IRC | 22:28 | |
| *** heyongli has quit IRC | 22:29 | |
| *** heyongli has joined #openstack-infra | 22:30 | |
| *** slaweq has quit IRC | 22:33 | |
| openstackgerrit | Merged openstack-infra/system-config master: Add support for turning on the future parser https://review.openstack.org/572861 | 22:37 |
| *** bobh has joined #openstack-infra | 22:39 | |
| *** heyongli has quit IRC | 22:40 | |
| *** heyongli has joined #openstack-infra | 22:40 | |
| *** owalsh_ has joined #openstack-infra | 22:40 | |
| *** owalsh has quit IRC | 22:42 | |
| *** felipemonteiro has joined #openstack-infra | 22:42 | |
| *** owalsh has joined #openstack-infra | 22:42 | |
| *** dklyle has joined #openstack-infra | 22:43 | |
| *** yolanda has quit IRC | 22:43 | |
| *** apetrich has quit IRC | 22:44 | |
| openstackgerrit | Merged openstack-infra/puppet-etherpad_lite master: Fix scope of vhost template variables https://review.openstack.org/564884 | 22:44 |
| *** yamamoto has joined #openstack-infra | 22:44 | |
| *** bobh has quit IRC | 22:44 | |
| *** apetrich has joined #openstack-infra | 22:44 | |
| openstackgerrit | Merged openstack-infra/puppet-ethercalc master: Fix scope of vhost template variables https://review.openstack.org/564882 | 22:45 |
| *** rfolco|off has quit IRC | 22:45 | |
| *** owalsh_ has quit IRC | 22:45 | |
| *** david-lyle has joined #openstack-infra | 22:45 | |
| *** yolanda has joined #openstack-infra | 22:45 | |
| dmsimard | fungi: oi, I'm a verified user on the wiki now \o/ | 22:45 |
| *** caphrim007_ has quit IRC | 22:46 | |
| *** caphrim007 has joined #openstack-infra | 22:46 | |
| fungi | dmsimard: yeah, i noticed that our patrolling volunteers have all gone missing at least 3 months ago (if not longer) | 22:46 |
| clarkb | 2300UTC is when I expect the futureparser option checking will first be used with ansible puppet | 22:47 |
| fungi | so i've been going back, patrolling all the edits for the past 30 days, adding any legit users who have made edits over the past 90 days to the verified users group, blocking any spammers and mass-deleting the pages they've created or rolling back their edits to actual content | 22:48 |
| fungi | a good chunk of my day i expected to spend on other things, unfortunately | 22:48 |
| *** yamamoto has quit IRC | 22:49 | |
| *** heyongli has quit IRC | 22:50 | |
| *** heyongli has joined #openstack-infra | 22:50 | |
| *** rpioso is now known as rpioso|afk | 22:55 | |
| *** eernst has joined #openstack-infra | 22:59 | |
| clarkb | yup it is running now and should pull in the new stuff | 23:00 |
| *** heyongli has quit IRC | 23:00 | |
| *** heyongli has joined #openstack-infra | 23:00 | |
| fungi | https://wiki.openstack.org/w/index.php?title=Special:RecentChanges&hideliu=0&hidepatrolled=1&days=30&limit=500 now only shows deleted spam though i went back the full 90 days of retention we have looking for users to verify or block | 23:01 |
| fungi | hopefully if i can find people interested in patrolling again, it won't be too hard to keep up with | 23:02 |
| clarkb | fungi: thank you for looking at that. Might want to ask the UC about it? | 23:02 |
| fungi | yeah, uc's not a bad angle | 23:02 |
| *** neiloy has quit IRC | 23:03 | |
| *** caphrim007 has quit IRC | 23:03 | |
| anteaya | fungi thanks for cleaning the wiki | 23:03 |
| *** david-lyle has quit IRC | 23:04 | |
| fungi | you're welcome | 23:08 |
| *** anteaya has quit IRC | 23:09 | |
| clarkb | fungi: I've just responded to the openci thread about securing our CI systems. Would be curious to hear your thoughts as well | 23:10 |
| *** heyongli has quit IRC | 23:10 | |
| *** heyongli has joined #openstack-infra | 23:11 | |
| openstackgerrit | Merged openstack-infra/puppet-meetbot master: Fix $manage_index if statement https://review.openstack.org/567095 | 23:11 |
| *** slaweq has joined #openstack-infra | 23:11 | |
| *** felipemonteiro has quit IRC | 23:12 | |
| openstackgerrit | Merged openstack-infra/puppet-cgit master: Add order to cgit package https://review.openstack.org/575852 | 23:12 |
| *** kzaitsev_pi has joined #openstack-infra | 23:12 | |
| *** felipemonteiro has joined #openstack-infra | 23:13 | |
| fungi | yeah, i had travis's initial e-mail for that thread flagged for followup weeks ago but haven't been able to brainstorm anything to suggest yet | 23:15 |
| *** slaweq has quit IRC | 23:15 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/system-config master: Add mqtt connection to zuul https://review.openstack.org/574492 | 23:16 |
| *** hamzy_ has joined #openstack-infra | 23:20 | |
| *** heyongli has quit IRC | 23:21 | |
| *** heyongli has joined #openstack-infra | 23:21 | |
| *** hamzy_ is now known as hamzy | 23:21 | |
| *** boris_42_ has quit IRC | 23:21 | |
| *** Goneri has joined #openstack-infra | 23:22 | |
| ianw | clarkb: so can we decide between 561758 and 576351 ? i'm biased, but i think it's better to stuff the extra stuff into the json retrun, rather than extend the string, as i'm not sure how that error message handles newlines | 23:23 |
| clarkb | ianw: how do we view that data if things break? | 23:24 |
| clarkb | will ara expose it? | 23:24 |
| mnaser | is ovh having some network issues | 23:24 |
| *** rfolco has joined #openstack-infra | 23:25 | |
| mnaser | http://logs.openstack.org/15/576615/2/check/openstack-ansible-deploy-aio_basekit-opensuse-423/dd9eb15/job-output.txt.gz | 23:25 |
| clarkb | ianw: I like yours for being better organized than mine if we can view that data if/when hosts fail | 23:25 |
| clarkb | mnaser: usually I would blame github | 23:26 |
| ianw | clarkb: yep, in the same way it exposes the "**ret" results in the success case, AIUI ; e.g. http://logs.openstack.org/98/575698/1/check/infra-puppet-apply-3-ubuntu-xenial/6dc9d2d/ara-report/result/16f230de-50aa-404b-8380-af5e7d2a9f80/ | 23:26 |
| mnaser | not even bootstrap.pypa.io worked | 23:26 |
| ianw | basically little boxes with each thing | 23:26 |
| mnaser | we fell back to github | 23:26 |
| clarkb | ianw: cool I'll abandon mine | 23:27 |
| clarkb | ianw: before we approve yours, we may want to clarify the testing expectations though | 23:28 |
| mnaser | if we're talking about https://review.openstack.org/#/c/576351/ .. yeah i was worried about breaking stuff | 23:29 |
| clarkb | mnaser: yes | 23:29 |
| clarkb | mnaser: https://twitter.com/OVH_Status/status/1009201673833730052 | 23:30 |
| ianw | do we have a way to test it? | 23:30 |
| mnaser | clarkb: ah good to know | 23:30 |
| mnaser | i'm not sure if your depends-on: works | 23:30 |
| clarkb | ianw: that is what stalled my change, I started working to split our existing tests and roles so that we could test that reliably | 23:30 |
| clarkb | since pinging doesn't need secrets | 23:30 |
| ianw | no it doesn't, i just copied that from clarkb's change. | 23:30 |
| ianw | so moved it into project-config? | 23:30 |
| *** heyongli has quit IRC | 23:31 | |
| *** heyongli has joined #openstack-infra | 23:31 | |
| clarkb | its not in project-config but our base job does the ping test out of there or something meaning it can't be tested as is currently. I'm going through the dpeends on list to page all that back in | 23:31 |
| clarkb | https://review.openstack.org/#/c/563790/1/roles/validate-host/tasks/main.yaml thats the issue since that requires privilege to run on the executor but the rest doesn't | 23:32 |
| clarkb | I guess deprecation is what was asked for | 23:32 |
| clarkb | I think theproblem is/was we don't hvae a process for that? | 23:32 |
| mnaser | yeah thinking about other users.. | 23:33 |
| mnaser | what about using base-test ? | 23:33 |
| clarkb | mnaser: that is what this does | 23:33 |
| clarkb | mnaser: problem is the diff above, you can only run that if it is in a privileged/trusted repo | 23:34 |
| mnaser | ah | 23:34 |
| clarkb | so in order to run validate-host in base-test it has to come out of that aiui | 23:34 |
| clarkb | otherwise it can't be self testing | 23:34 |
| *** rlandy has quit IRC | 23:34 | |
| *** diablo_rojo has quit IRC | 23:35 | |
| *** r-daneel has quit IRC | 23:37 | |
| ianw | just backing rrrriiiggghttt up for the uninitiated here :) | 23:37 |
| ianw | http://logs.openstack.org/51/576351/2/check/openstack-infra-base-integration-centos-7/e7f5bc1/zuul-info/inventory.yaml | 23:37 |
| ianw | is the file we're talking about? | 23:37 |
| clarkb | yes I think so, and because the delegate to is to localhost that means it has to be special | 23:38 |
| *** ldnunes has quit IRC | 23:38 | |
| ianw | what about the delegate_to in the command above it too? | 23:39 |
| ianw | in https://review.openstack.org/#/c/563790/1/roles/validate-host/tasks/main.yaml | 23:39 |
| clarkb | the source is in the job itself so I think it is ok? | 23:40 |
| clarkb | (this is somewhat fuzzy for myself as it has been a little while) | 23:40 |
| clarkb | I want to say the error is the inventory task reads the inventory file from outside of the job workspace itself. But ansible zuul allows us to write a template into the job workspace | 23:41 |
| ianw | i thought the bigger problem was the zuul-jobs was trusted, so was never pulled into the executors | 23:41 |
| *** heyongli has quit IRC | 23:41 | |
| *** lifeless_ has joined #openstack-infra | 23:41 | |
| *** heyongli has joined #openstack-infra | 23:41 | |
| clarkb | zuul jobs is not trusted, but many of the roles it defines are only used from trusted jobs | 23:42 |
| *** diablo_rojo has joined #openstack-infra | 23:42 | |
| *** lifeless has quit IRC | 23:42 | |
| clarkb | this setup stuff gets tricky due to this relationship between roles in untrusted repos and jobs in trusted repos. The way we consume validate-host is via the base job which is trusted and won't execute validate-host in a pre merge setup | 23:43 |
| ianw | ok, yes i think that's what i meant but not saying it correctly. we have problems testing this series of roles -> http://git.openstack.org/cgit/openstack-infra/project-config/tree/playbooks/base/pre.yaml#n20 | 23:44 |
| clarkb | yup | 23:45 |
| *** tosky has quit IRC | 23:45 | |
| ianw | (c.f. me breaking configure-unbound previously) | 23:45 |
| clarkb | if we uncouple the trusted requiring inventory write we can run https://review.openstack.org/#/c/563788/ | 23:46 |
| clarkb | which funny enough has comments about configure-unbound | 23:46 |
| *** yamamoto has joined #openstack-infra | 23:46 | |
| ianw | ooohhh, so we can pull validate-host into base-test.yaml | 23:48 |
| ianw | which is unprivileged | 23:48 |
| clarkb | if we do https://review.openstack.org/#/c/563790/1/roles/validate-host/tasks/main.yaml | 23:49 |
| ianw | ok ... starting to come together | 23:50 |
| *** r-daneel has joined #openstack-infra | 23:50 | |
| ianw | i'm not sure i'm understanding why we need a deprecation notice | 23:50 |
| clarkb | I think the concern is people will have to update their base jobs to use the new log-inventory role | 23:50 |
| ianw | we're adding it back straight away with https://review.openstack.org/#/c/563787/ | 23:51 |
| clarkb | and until they do that won't get the inventory data | 23:51 |
| *** r-daneel has quit IRC | 23:51 | |
| *** yamamoto has quit IRC | 23:51 | |
| *** heyongli has quit IRC | 23:51 | |
| *** heyongli has joined #openstack-infra | 23:52 | |
| ianw | i feel like the intersection of people who look at this file, and maintain their own jobs is probably small enough that mails to openstack-dev might be a bit of overkill | 23:53 |
| *** bobh has joined #openstack-infra | 23:54 | |
| ianw | it doesn't seem like there's a simple way to raise a deprecation warning from a playbook? | 23:54 |
| *** bobh has quit IRC | 23:55 | |
| *** bobh has joined #openstack-infra | 23:55 | |
| clarkb | I think the concern is for people running their own zuuls that are not us | 23:57 |
| clarkb | but I think that number is probably still small enough that mail to zuul list may be sufficient | 23:57 |
| * SpamapS makes my own jobs, and uses validate-host | 23:58 | |
| SpamapS | and I *absolutely* rely on inventory.yml | 23:59 |
| SpamapS | We hold a lot of nodes | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!