| pabelanger | mordred: when do you start your travels to PTG? | 00:00 |
|---|---|---|
| *** bobh has joined #openstack-infra | 00:01 | |
| mordred | pabelanger: saturday. you? | 00:02 |
| pabelanger | mordred: same | 00:02 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Support IPv6 with zuul_stream https://review.openstack.org/500401 | 00:02 |
| *** Guest46592 has joined #openstack-infra | 00:03 | |
| mordred | pabelanger: WOOT!!!!!!! | 00:05 |
| mordred | pabelanger: the sibling change worked!!!!!! | 00:05 |
| mordred | http://logs.openstack.org/87/500387/1/check/tox-py35/98fc1f1/testr_results.html.gz | 00:05 |
| pabelanger | Niiice | 00:05 |
| mordred | open up a fail and look at the error message after "Error authenticating to keystone" | 00:05 |
| clarkb | if everyone is in a reviewing mood, want to review the parted alignment change in d-g? | 00:05 |
| jamielennox | pabelanger: hey, are you mapping any form of known_hosts into bwrap? | 00:05 |
| pabelanger | jamielennox: ya, should be in ~/.ssh/known_hosts | 00:06 |
| jamielennox | from memory you can publish logs by copying to a mounted dir | 00:06 |
| mordred | but we set it up explicitly in the base job | 00:06 |
| clarkb | are we mapping that in from the host? | 00:06 |
| clarkb | we scan in the job right? | 00:06 |
| mordred | we don't do either | 00:06 |
| mordred | we provide host key in variables and set it explicitly in job content because nodepool already knows the host key | 00:06 |
| clarkb | "knows" | 00:07 |
| clarkb | but ya | 00:07 |
| pabelanger | jamielennox: Ya, you can setup a trusted_rw_paths in zuul.conf, we do that for afs-docs now | 00:07 |
| jamielennox | yea, nodepool provides the host ssh keys | 00:07 |
| jamielennox | but i have an rsync job for log publishing | 00:07 |
| mordred | jamielennox: yah - we have one of those, and do the same thing - pass in hostkey of the server we rsync to ... | 00:07 |
| jamielennox | so i need to have a publishing key and known hosts file for logs.bonny. | 00:07 |
| mordred | jamielennox: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.yaml#n292 | 00:08 |
| pabelanger | Ya, that | 00:08 |
| mordred | jamielennox: that's our secret containing the key as well as the hostname and known_hosts | 00:08 |
| pabelanger | ~/.ssh/known_hosts is just nodes from nodepool in bwrap | 00:08 |
| pabelanger | otherwise, what mordred linked | 00:08 |
| jamielennox | oh, interesting | 00:08 |
| jamielennox | i didn't actually think of using zuul secrets for that | 00:09 |
| mordred | that's used in our base job: http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.yaml#n134 | 00:09 |
| mordred | which runs this: http://git.openstack.org/cgit/openstack-infra/project-config/tree/playbooks/base/post-logs.yaml | 00:09 |
| mordred | the add-fileserver role that uses is in zuul-jobs - so you should be able to re-use it: http://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/add-fileserver | 00:09 |
| *** thorst_afk has joined #openstack-infra | 00:10 | |
| jamielennox | you guys really lapped us there in job definitions | 00:10 |
| mordred | and if you make a secret with the same fields in it - you can just pass the secret to add-fileserver and it'll do the magic for you | 00:10 |
| *** bobh has quit IRC | 00:10 | |
| pabelanger | Magic! | 00:10 |
| mordred | jamielennox: well, we've got to get this openstack thing up and going - it's been a bit of a whirlwind of job writing | 00:11 |
| mordred | jamielennox: but - for the most part we're doing our best to make sure as much of what we're writing is directly consumable and if not that we put it in not-zuul-jobs | 00:11 |
| openstackgerrit | Merged openstack-infra/project-config master: Revert "Turn off vexxhost in v3 for now" https://review.openstack.org/500407 | 00:11 |
| jamielennox | yea, i just took my eye off the ball for too long on the job writing, but i know the idea is to keep that reusable | 00:12 |
| *** thorst_afk has quit IRC | 00:12 | |
| mordred | pabelanger: all three of teh DNM patches showing that sibling installation into tox works passed and did the right thing | 00:13 |
| mordred | the stack of examples ends herE: https://review.openstack.org/#/c/500388 | 00:13 |
| jamielennox | so you've obviously relaxed secrets so they cna be used in the check pipeline for that to work? | 00:14 |
| mordred | nope | 00:14 |
| mordred | project-config is a trusted repo | 00:14 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Remove more invalid settings for nodepoolv3.yaml https://review.openstack.org/500399 | 00:14 |
| mordred | so the job parts from it are never put into a speculative state | 00:14 |
| *** dingyichen has joined #openstack-infra | 00:14 | |
| jamielennox | so the job + secret are bothing coming from trusted is what makes that ok ? | 00:15 |
| mordred | when you run a thing in the check pipeline, you don't ever get proposed project-config changes, only landed ones | 00:15 |
| pabelanger | ya, secrets kinda changed too. they are now tied to playbooks more | 00:15 |
| *** salv-orlando has joined #openstack-infra | 00:15 | |
| mordred | yah- oh - yah, that change is important | 00:15 |
| jamielennox | yea, right, last time i was playing with that stuff it was about the pipeline | 00:15 |
| mordred | the playbooks of a job that consumes a secret are the only ones that get access to the secret now | 00:15 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Use afs_publisher_target for afs-docs https://review.openstack.org/500406 | 00:15 |
| jamielennox | but i need to publish jobs for check as well | 00:16 |
| mordred | yah | 00:16 |
| pabelanger | we do too, they are just trusted playbooks | 00:16 |
| mordred | that's why the secrets change was made | 00:16 |
| jamielennox | oh, nice - that's a good change | 00:16 |
| pabelanger | ya | 00:16 |
| mordred | so that we can use secrets for this | 00:16 |
| mordred | that's 100% of our logging there :) | 00:16 |
| pabelanger | mordred: re: 500388 Ya, that is powerful, mind is blown | 00:16 |
| mordred | pabelanger: right? that's possibly the coolest thing I've written in a while | 00:17 |
| jamielennox | ok, i can read code for this next bit then - but what is the key related to then? | 00:17 |
| mordred | that's the private ssh key for the zuul account on the remote fileserver | 00:17 |
| jamielennox | sorry, not what i meant | 00:17 |
| jamielennox | the key that encrypts secrets | 00:17 |
| jamielennox | is there now a per job key? | 00:17 |
| mordred | per project | 00:18 |
| mordred | so you encrypt a secret for, say, project-config | 00:18 |
| mordred | and then jobs in project-config that consume that secret can decrypt it | 00:18 |
| mordred | you can fetch the public key for a project from a URL - and there is a tool in the zuul source tree to help you encryptthings | 00:18 |
| *** Guest46592 has quit IRC | 00:18 | |
| jamielennox | but project-config isn't a job | 00:18 |
| jamielennox | but project-config isn't a project | 00:19 |
| jamielennox | it's (more than) a list of jobs that other projects can import | 00:19 |
| mordred | it is - git.openstack.org/openstack-infra/project-config is a project | 00:19 |
| mordred | (every git repo is a project) | 00:19 |
| jamielennox | ok | 00:20 |
| mordred | in our case, git.openstack.org/openstack-infra/project-config happens to be a config-project, but it doens't matter, every git repo zuul knows about gets a keypair | 00:20 |
| pabelanger | http://zuulv3.openstack.org/keys/gerrit/openstack-infra/project-config.pub would be the public key used | 00:20 |
| *** sflanigan has joined #openstack-infra | 00:20 | |
| *** sflanigan has joined #openstack-infra | 00:20 | |
| jamielennox | ok, i should look more into how those keys get unlocked | 00:20 |
| *** jascott1 has joined #openstack-infra | 00:21 | |
| jamielennox | it's fine that a run could unlock multiple keys, just not how i expected the import to work | 00:21 |
| jamielennox | anyway - thanks, i can work from that | 00:21 |
| mordred | jamielennox: in case the helptext is terse ... | 00:21 |
| mordred | jamielennox: python3 tools/encrypt_secret.py --infile requirements.txt --outfile requirements.txt.yaml http://zuulv3.openstack.org gerrit openstack-infra/project-config | 00:21 |
| jamielennox | yea, i've had to do that manually a few times | 00:22 |
| pabelanger | mordred: Hmm, we should be okay to run bindep for publish-openstack-python-docs, AFS secrets are in bwrap. | 00:22 |
| mordred | jamielennox: is the command line to encrypt the contents of the file requirements.txt into a secret and write it to requirements.txt.yaml for the openstack-infra/project-config project | 00:22 |
| pabelanger | http://logs.openstack.org/ac/9c2783b596e8a40e51703d4ff2d381e0cb4adeac/post/publish-openstack-python-docs/b7e953c/job-output.txt.gz failed since graphviz wasn't installed | 00:22 |
| jamielennox | well manually like against the api, | 00:22 |
| *** salv-orlando has quit IRC | 00:22 | |
| *** salv-orlando has joined #openstack-infra | 00:23 | |
| mordred | pabelanger: oh - yah -we should definitely be running bindep in that job | 00:23 |
| mordred | pabelanger: I thought we were re-using tox base job | 00:23 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Add bindep for publish-openstack-python-docs job https://review.openstack.org/500409 | 00:25 |
| mordred | pabelanger: you can kill those command and fallback lines | 00:26 |
| pabelanger | mordred: Ya, there is some duplicate code here between trusted / untrusted job still. Going to push forward on that tomorrow | 00:26 |
| mordred | pabelanger: ahhhh . yes. I grok now | 00:26 |
| mordred | pabelanger: because we need the publish-openstack-afs parent, not theunittests parent | 00:26 |
| mordred | pabelanger: makes sense | 00:26 |
| *** EricGonczer_ has joined #openstack-infra | 00:27 | |
| pabelanger | mordred: Ya, but I think we'll be able to use roles in project-config, and reduce some overlap | 00:27 |
| mordred | yah | 00:27 |
| pabelanger | but PTG, like to whiteboard some of our jobs and bikeshed a little. | 00:27 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Add bindep for publish-openstack-python-docs job https://review.openstack.org/500409 | 00:28 |
| mordred | pabelanger: yah - we've learned MANY things at this point | 00:28 |
| mordred | pabelanger: oh- also - I noticed earlier that nl01 is running python3.4 ... | 00:30 |
| mordred | pabelanger: so to your question about launchers and stuff for the PTG - we might wnat to spin up some new xenial launchers and decomission nl01 | 00:30 |
| pabelanger | Oh, ya it is trusty | 00:30 |
| pabelanger | ++ | 00:31 |
| pabelanger | we can stand up nl02.o.o as xenial | 00:31 |
| pabelanger | then replace n01 | 00:31 |
| mordred | ++ | 00:31 |
| jeblair | has ze01 been restarted with stream fix? | 00:32 |
| pabelanger | it is ready to be restarted now | 00:32 |
| jeblair | pabelanger: you doing that or should i? | 00:32 |
| pabelanger | jeblair: sure, if you like to | 00:32 |
| jeblair | well, my job has been failing with that for a few hours so yes | 00:33 |
| *** salv-orlando has quit IRC | 00:33 | |
| pabelanger | ack | 00:34 |
| jeblair | mordred: i fixed the syntax error from under you | 00:34 |
| mordred | yah - I see - my patch was mostly so I could poke without stepping on you | 00:34 |
| jeblair | mordred: is etcd the next thing? | 00:35 |
| mordred | ETCD_DOWNLOAD_URL: "http://tarballs.openstack.org/etcd/" is the other thing in there that I learned today | 00:35 |
| mordred | yah | 00:35 |
| mordred | jeblair: although - do please enjoy that things basically disappear while it's downloading etcd | 00:35 |
| jeblair | mordred: whatcha mean? | 00:35 |
| mordred | jeblair: so you might want to not apply that yet so we can poke at why everything stops logging when devstack does the wget | 00:35 |
| mordred | jeblair: multiple times in a row, the log stream got to the wget, and then it just ... stopped | 00:36 |
| jeblair | mordred: okay, i noticed something similar when it was getting cirros, but that was before i fixed the timeout so i assumed it was something with that; also infra cloud is slow and i was unsure there should be output | 00:36 |
| mordred | I would explain more, but I'm baffled by what was going on | 00:36 |
| pabelanger | Oh, if tarballs.o.o/etcd is a thing, we can use proxy cache in devstack too | 00:36 |
| mordred | jeblair: good point | 00:36 |
| pabelanger | then local mirror will have etcd | 00:36 |
| jeblair | mordred: but i did not look into it, so those are all just excuses :) | 00:36 |
| mordred | pabelanger: yah - it's one of the pile of 40 settings d-g makes for devstack that actually *are* meaningful | 00:37 |
| mordred | sorting through them is ... fun | 00:37 |
| clarkb | pabelanger: ya that is what I was saying earlier | 00:37 |
| pabelanger | clarkb: ah, I think I missed that | 00:37 |
| pabelanger | clarkb: see it now | 00:37 |
| mordred | jeblair: https://review.openstack.org/#/c/500365/7/.zuul.yaml ... around line 67 are the other ones I lifted out earlier today | 00:38 |
| mordred | not 100% sure ALL of them are needed, but I think most are (since they're setting network ranges to use, etc) | 00:38 |
| clarkb | pabelanger: willing to review https://review.openstack.org/499314 ? | 00:38 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove more invalid settings for nodepoolv3.yaml https://review.openstack.org/500399 | 00:39 |
| jeblair | mordred: how's the devstack_local thing working out? that make sense? | 00:39 |
| mordred | jeblair: yes - I like it all a lot | 00:39 |
| mordred | jeblair: it was easy for me to write all of the jobs shade needs | 00:39 |
| jeblair | mordred: minor thing -- do you like 'post-config: "$CINDER_CONF": ...' or would you prefer '"post-config|$CINDER_CONF": ...' ? | 00:40 |
| mordred | jeblair: the hardest bit is figuring out what the actual devstack settings are for some of the d-g flags | 00:40 |
| mordred | jeblair: either is fine with me tbh - needing to quote the key name is the only strange bit, but works fine | 00:40 |
| jeblair | i was going back and forth on that -- whether to base in on the underlying construct or be more in tune with the devstack implementation. | 00:40 |
| mordred | I think I like post_config: "$CINDER_CONF" though | 00:40 |
| jeblair | mordred: yeah, and that'd be necessary either way | 00:40 |
| jeblair | cool | 00:41 |
| mordred | jeblair: oh - don't know if you saw in the scrollback - sibling tox installs from git totally works now | 00:41 |
| jeblair | mordred: \o/ | 00:42 |
| jeblair | mordred: that's going to be awesome | 00:42 |
| jeblair | http://logs.openstack.org/02/500202/14/check/devstack-lxc/514a7ce/job-output.txt.gz#_2017-09-04_00_36_36_112028 | 00:42 |
| jeblair | that's a new devstack error | 00:42 |
| mordred | jeblair: there is a stack of patches in shade thatend with https://review.openstack.org/#/c/500388 that show it working | 00:42 |
| jeblair | mordred: mind if i look at them when i have better brains? | 00:42 |
| mordred | oh - totally | 00:43 |
| mordred | no rush | 00:43 |
| mordred | jeblair: I think that new error might be related to the set of things from https://review.openstack.org/#/c/500365/7/.zuul.yaml line 67 | 00:43 |
| mordred | jeblair: since we have to tell devstack about its network information | 00:43 |
| mordred | oh - we also do: devstack-vm-gate.sh: localrc_set "$localrc_file" "HOST_IP" "$primary_node" | 00:43 |
| jeblair | welp. there we go. | 00:44 |
| jeblair | wonder why the detection fails | 00:44 |
| mordred | dunno - and it looks like all of the places we set HOST_IP are for subnodes | 00:45 |
| *** slaweq has joined #openstack-infra | 00:45 | |
| clarkb | mordred: you should be able to diff local.conf files from run jobs to see what needs adding | 00:45 |
| mordred | clarkb: ++ | 00:45 |
| jeblair | private_ipv4: 10.0.1.79 public_ipv4: 77.81.189.94 | 00:46 |
| jeblair | i'm guessing the 10. confuses devstack? | 00:46 |
| *** markvoelker has joined #openstack-infra | 00:47 | |
| clarkb | we explicitly tell devstack to use 10.1.0.0/20 to fix that | 00:47 |
| jeblair | mordred: mind if i just incorporate your etcd line along with host_ip in my next patchset? | 00:47 |
| mordred | jeblair: please do - I'd go ahead and grab the otherlines too though | 00:47 |
| mordred | with FIXED_RANGE and IPV4_ADDRS_SAFE_TO_USE and FLOATING_RANGE and whatnot | 00:47 |
| jeblair | mordred: this set: https://etherpad.openstack.org/p/By9Rw7nNTK | 00:49 |
| jeblair | clarkb: ^ | 00:49 |
| * jeblair works on host_ip | 00:49 | |
| mordred | yes - I missed one ... one sec: | 00:49 |
| mordred | added | 00:50 |
| openstackgerrit | Merged openstack-infra/project-config master: Add bindep for publish-openstack-python-docs job https://review.openstack.org/500409 | 00:50 |
| jeblair | oh, hrm, host_ip is going to need to be special | 00:50 |
| mordred | jeblair: I think we actualy don't set HOST_IP for single-node runs - only for multinode - | 00:50 |
| jeblair | mordred: hrm, i think you're right... but then that's how it failed | 00:51 |
| *** slaweq has quit IRC | 00:51 | |
| pabelanger | okay, rackspace is working great now | 00:51 |
| jeblair | mordred: do you think it's all the other vars that make it work? | 00:51 |
| mordred | jeblair: yes, I do | 00:52 |
| pabelanger | guess we'll just wait for vexxhost jobs now | 00:52 |
| *** mat128 has joined #openstack-infra | 00:52 | |
| jeblair | mordred: indeed, as clarkb suggested, i checked local.conf on a current runand HOST_IP is not there | 00:52 |
| jeblair | i will also add ERROR_ON_CLONE :) | 00:53 |
| mordred | jeblair: $(get_default_host_ip "$FIXED_RANGE" "$FLOATING_RANGE" "$HOST_IP_IFACE" "$HOST_IP" "inet") | 00:54 |
| mordred | jeblair: sorry - bad paste: | 00:54 |
| mordred | jeblair: HOST_IP=$(get_default_host_ip "$FIXED_RANGE" "$FLOATING_RANGE" "$HOST_IP_IFACE" "$HOST_IP" "inet") | 00:54 |
| openstackgerrit | James E. Blair proposed openstack-infra/devstack-gate master: Zuul v3: run devstack https://review.openstack.org/500202 | 00:54 |
| mordred | jeblair: that's from stackrc - so yeah, I think the other things will make it work | 00:54 |
| jeblair | mordred: got it -- fixed_range etc make it work | 00:54 |
| mordred | jeblair: fingers crossed | 00:54 |
| mordred | jeblair: and now I shall dinner | 00:54 |
| jeblair | mordred: erm, me too. | 00:55 |
| *** ijw_ has joined #openstack-infra | 00:55 | |
| *** ijw has quit IRC | 00:58 | |
| pabelanger | ya, I'm out too | 00:58 |
| openstackgerrit | Ian Wienand proposed openstack-infra/devstack-gate master: Ansiblise ephemeral/swap setup https://review.openstack.org/499467 | 00:59 |
| *** ijw has joined #openstack-infra | 01:04 | |
| *** ijw_ has quit IRC | 01:07 | |
| *** zhurong has joined #openstack-infra | 01:09 | |
| openstackgerrit | James E. Blair proposed openstack-infra/devstack-gate master: Zuul v3: run devstack https://review.openstack.org/500202 | 01:17 |
| *** liujiong has joined #openstack-infra | 01:18 | |
| *** markvoelker has quit IRC | 01:19 | |
| *** dims has quit IRC | 01:21 | |
| *** ijw has quit IRC | 01:29 | |
| *** adisky__ has joined #openstack-infra | 01:30 | |
| *** dims has joined #openstack-infra | 01:33 | |
| *** gcb has joined #openstack-infra | 01:35 | |
| *** abelur_ has joined #openstack-infra | 01:38 | |
| *** stakeda has joined #openstack-infra | 01:43 | |
| openstackgerrit | James E. Blair proposed openstack-infra/devstack-gate master: Zuul v3: run devstack https://review.openstack.org/500202 | 02:02 |
| *** liusheng has quit IRC | 02:12 | |
| *** liusheng has joined #openstack-infra | 02:12 | |
| *** markvoelker has joined #openstack-infra | 02:16 | |
| zhongjun | Hi everyone. Since we merged many patches [1] about api-ref doc in manila 3 weeks ago. But it didn't display in the Manila developer doc [2]. Do we need to do anything else or just waiting for automatic update? | 02:17 |
| zhongjun | [1] https://review.openstack.org/#/c/491661/ | 02:18 |
| zhongjun | [2] https://developer.openstack.org/api-ref/shared-file-systems/#consistency-groups-since-api-v2-4 | 02:18 |
| *** xinliang has quit IRC | 02:22 | |
| ianw | mordred: i'm going to assume you are ok with me uploading changes over 500375 :) | 02:24 |
| *** bobh has joined #openstack-infra | 02:33 | |
| *** xinliang has joined #openstack-infra | 02:34 | |
| *** xinliang has quit IRC | 02:34 | |
| *** xinliang has joined #openstack-infra | 02:34 | |
| *** mwarad has joined #openstack-infra | 02:37 | |
| *** EricGonczer_ has quit IRC | 02:41 | |
| *** mwarad has quit IRC | 02:42 | |
| *** ianychoi_ has joined #openstack-infra | 02:43 | |
| *** ianychoi has quit IRC | 02:46 | |
| yamamoto | can anyone take care of lib/neutron? https://review.openstack.org/#/c/437774/ | 02:47 |
| yamamoto | oops, i guess it should be openstack-qa. sorry | 02:49 |
| *** markvoelker has quit IRC | 02:50 | |
| *** mat128 has quit IRC | 02:52 | |
| *** mat128 has joined #openstack-infra | 02:53 | |
| *** bobh has quit IRC | 02:58 | |
| *** chenying has joined #openstack-infra | 02:58 | |
| *** ramishra has joined #openstack-infra | 03:01 | |
| *** thorst_afk has joined #openstack-infra | 03:01 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: DNM: test wget https://review.openstack.org/500426 | 03:07 |
| *** thorst_afk has quit IRC | 03:15 | |
| *** udesale has joined #openstack-infra | 03:19 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: DNM: test wget https://review.openstack.org/500426 | 03:22 |
| *** zhurong_ has joined #openstack-infra | 03:22 | |
| *** ykarel has joined #openstack-infra | 03:23 | |
| *** zhurong_ has quit IRC | 03:25 | |
| *** yamahata has joined #openstack-infra | 03:30 | |
| *** nicolasbock has quit IRC | 03:41 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul feature/zuulv3: sql: normalize logger name https://review.openstack.org/500431 | 03:42 |
| *** markvoelker has joined #openstack-infra | 03:47 | |
| *** Dinesh_Bhor has joined #openstack-infra | 03:53 | |
| *** zhurong has quit IRC | 03:56 | |
| *** claudiub|2 has joined #openstack-infra | 04:13 | |
| *** links has joined #openstack-infra | 04:19 | |
| *** markvoelker has quit IRC | 04:20 | |
| *** persia has quit IRC | 04:20 | |
| *** mat128 has quit IRC | 04:21 | |
| *** hongbin has quit IRC | 04:27 | |
| *** sree_ has joined #openstack-infra | 04:28 | |
| *** psachin has joined #openstack-infra | 04:28 | |
| *** sree_ is now known as Guest50425 | 04:28 | |
| yamamoto | is there a way to rename LP "series"? i thought Q was queen but it's actually queens. | 04:31 |
| *** jaosorior has joined #openstack-infra | 04:36 | |
| *** bhavik1 has joined #openstack-infra | 04:53 | |
| *** andreww is now known as xarses | 05:06 | |
| *** shu-mutou-AWAY is now known as shu-mutou | 05:12 | |
| *** gcb has quit IRC | 05:15 | |
| *** markvoelker has joined #openstack-infra | 05:17 | |
| *** gcb has joined #openstack-infra | 05:29 | |
| *** dhajare has joined #openstack-infra | 05:35 | |
| *** gouthamr has quit IRC | 05:41 | |
| *** markvoelker has quit IRC | 05:49 | |
| *** slaweq has joined #openstack-infra | 05:51 | |
| *** slaweq has quit IRC | 05:56 | |
| *** florianf has joined #openstack-infra | 06:03 | |
| *** LindaWang has joined #openstack-infra | 06:08 | |
| *** ccamacho has joined #openstack-infra | 06:08 | |
| *** zhurong has joined #openstack-infra | 06:09 | |
| *** anthonyper has quit IRC | 06:11 | |
| *** anthonyper has joined #openstack-infra | 06:13 | |
| *** hashar has joined #openstack-infra | 06:18 | |
| *** hashar is now known as hasharAway | 06:18 | |
| *** HeOS has joined #openstack-infra | 06:22 | |
| *** pgadiya has joined #openstack-infra | 06:23 | |
| *** ijw has joined #openstack-infra | 06:30 | |
| *** ijw has quit IRC | 06:35 | |
| *** HeOS has quit IRC | 06:38 | |
| *** Guest50425 is now known as Sree | 06:42 | |
| *** markvoelker has joined #openstack-infra | 06:47 | |
| *** jascott1 has quit IRC | 06:49 | |
| *** rcernin has quit IRC | 06:52 | |
| *** rcernin has joined #openstack-infra | 06:52 | |
| *** slaweq has joined #openstack-infra | 06:52 | |
| *** slaweq has quit IRC | 06:57 | |
| ttx | yamamoto: maybe when on the series page, click "Change details" ? | 06:58 |
| ttx | (haven't tried) | 06:58 |
| *** jpena|off is now known as jpena | 06:59 | |
| yamamoto | ttx: it worked. thank you! | 06:59 |
| ttx | yamamoto: \o/ | 07:00 |
| *** e0ne has joined #openstack-infra | 07:02 | |
| *** slaweq has joined #openstack-infra | 07:03 | |
| *** coolsvap has joined #openstack-infra | 07:07 | |
| *** yamahata has quit IRC | 07:07 | |
| *** zhurong has quit IRC | 07:08 | |
| *** tesseract has joined #openstack-infra | 07:16 | |
| *** markvoelker has quit IRC | 07:20 | |
| openstackgerrit | Merged openstack-infra/devstack-gate master: Update Ansible from 2.2.0.0 to 2.3.2.0 https://review.openstack.org/496356 | 07:22 |
| *** jtomasek has joined #openstack-infra | 07:23 | |
| *** aarefiev_afk is now known as aarefiev | 07:24 | |
| openstackgerrit | Jamie Lennox proposed openstack-infra/zuul-jobs master: Allow overriding the workspace directory in prepare-workspace https://review.openstack.org/500466 | 07:25 |
| *** dingyichen has quit IRC | 07:31 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: upload-logs/bindep: fix indentation https://review.openstack.org/500339 | 07:32 |
| *** dtantsur|afk is now known as dtantsur | 07:34 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add Neutron API job to networking-ovn https://review.openstack.org/498466 | 07:34 |
| *** jpich has joined #openstack-infra | 07:35 | |
| *** pcaruana has joined #openstack-infra | 07:36 | |
| *** pcaruana has quit IRC | 07:36 | |
| *** pcaruana has joined #openstack-infra | 07:37 | |
| *** lucas-afk is now known as lucasagomes | 07:41 | |
| *** rcernin has quit IRC | 07:41 | |
| *** s-shiono has joined #openstack-infra | 07:44 | |
| *** egonzalez has joined #openstack-infra | 07:45 | |
| *** HeOS has joined #openstack-infra | 07:48 | |
| *** jascott1 has joined #openstack-infra | 07:50 | |
| *** shardy has joined #openstack-infra | 07:50 | |
| *** shu-mutou has quit IRC | 07:50 | |
| *** oomichi has quit IRC | 07:51 | |
| *** oomichi has joined #openstack-infra | 07:52 | |
| *** ralonsoh has joined #openstack-infra | 07:53 | |
| *** andreas_s has joined #openstack-infra | 07:56 | |
| *** alexchadin has joined #openstack-infra | 07:59 | |
| *** andreas_s has quit IRC | 07:59 | |
| *** slaweq has quit IRC | 08:09 | |
| *** slaweq has joined #openstack-infra | 08:09 | |
| *** ykarel is now known as ykarel|lunch | 08:10 | |
| *** markvoelker has joined #openstack-infra | 08:18 | |
| *** slaweq has quit IRC | 08:20 | |
| *** HeOS has quit IRC | 08:21 | |
| openstackgerrit | Jamie Lennox proposed openstack-infra/zuul feature/zuulv3: Print SIGTERM logging to debug https://review.openstack.org/500476 | 08:23 |
| *** Sree is now known as sreeram_V | 08:26 | |
| *** sreeram_V is now known as sree | 08:26 | |
| *** derekh has joined #openstack-infra | 08:26 | |
| *** sree is now known as Sree | 08:27 | |
| *** yamamoto has quit IRC | 08:35 | |
| *** bauwser is now known as bauzas | 08:40 | |
| *** hasharAway is now known as hashar | 08:42 | |
| *** yamamoto has joined #openstack-infra | 08:46 | |
| *** markvoelker has quit IRC | 08:51 | |
| *** sshnaidm|off is now known as sshnaidm | 08:54 | |
| *** nicolasbock has joined #openstack-infra | 08:55 | |
| *** electrofelix has joined #openstack-infra | 08:56 | |
| openstackgerrit | Gael Chamoulaud proposed openstack-infra/tripleo-ci master: Enable tripleo-validations tests https://review.openstack.org/481080 | 08:57 |
| *** yamamoto has quit IRC | 08:59 | |
| *** jokke_ has joined #openstack-infra | 09:04 | |
| *** vponomaryov has joined #openstack-infra | 09:11 | |
| *** yamamoto has joined #openstack-infra | 09:11 | |
| *** yamamoto has quit IRC | 09:13 | |
| openstackgerrit | yolanda.robla proposed openstack-infra/project-config master: Move working centos jobs in bifrost to check queue https://review.openstack.org/499525 | 09:15 |
| *** dtantsur is now known as dtantsur|bbl | 09:21 | |
| *** slaweq has joined #openstack-infra | 09:21 | |
| *** tosky has joined #openstack-infra | 09:22 | |
| *** slaweq has quit IRC | 09:26 | |
| *** liujiong has quit IRC | 09:35 | |
| *** liujiong has joined #openstack-infra | 09:35 | |
| *** ykarel_ has joined #openstack-infra | 09:36 | |
| openstackgerrit | Jens Harbott (frickler) proposed openstack-infra/elastic-recheck master: Add query for bug 1714901 https://review.openstack.org/500493 | 09:37 |
| openstack | bug 1714901 in neutron "gate-rally-dsvm-neutron-neutron-ubuntu-xenial failing 100% in NeutronTrunks.create_and_list_trunk_subports" [Undecided,New] https://launchpad.net/bugs/1714901 | 09:37 |
| *** ykarel|lunch has quit IRC | 09:38 | |
| *** nicolasbock has quit IRC | 09:39 | |
| openstackgerrit | Kevin Benton proposed openstack-infra/project-config master: Enable Neuron trunk plugin for rally job https://review.openstack.org/500496 | 09:46 |
| kevinbenton | hi, can we get this merged to unblock the neutron queue? ^^ | 09:46 |
| *** markvoelker has joined #openstack-infra | 09:48 | |
| *** baoli has joined #openstack-infra | 09:48 | |
| *** stakeda has quit IRC | 09:51 | |
| *** lucasagomes is now known as lucas-brb | 09:52 | |
| *** gcb has quit IRC | 09:52 | |
| *** baoli has quit IRC | 09:57 | |
| *** yamamoto has joined #openstack-infra | 10:04 | |
| *** s-shiono has quit IRC | 10:04 | |
| *** tosky has quit IRC | 10:05 | |
| *** HeOS has joined #openstack-infra | 10:06 | |
| *** alexchadin has quit IRC | 10:09 | |
| *** alexchadin has joined #openstack-infra | 10:09 | |
| *** markvoelker has quit IRC | 10:21 | |
| *** slaweq has joined #openstack-infra | 10:22 | |
| *** slaweq has quit IRC | 10:27 | |
| *** pgadiya has quit IRC | 10:31 | |
| *** thorst_afk has joined #openstack-infra | 10:36 | |
| *** ykarel__ has joined #openstack-infra | 10:38 | |
| *** alexchadin has quit IRC | 10:38 | |
| *** pblaho has joined #openstack-infra | 10:38 | |
| *** dhajare has quit IRC | 10:38 | |
| *** adisky__ has quit IRC | 10:39 | |
| *** alexchadin has joined #openstack-infra | 10:39 | |
| *** ykarel_ has quit IRC | 10:41 | |
| *** Sree has quit IRC | 10:42 | |
| *** ykarel__ is now known as ykarel | 10:45 | |
| *** udesale has quit IRC | 10:46 | |
| *** pgadiya has joined #openstack-infra | 10:47 | |
| *** lucas-brb is now known as lucasagomes | 10:48 | |
| *** thorst_afk has quit IRC | 10:54 | |
| *** liujiong_lj has joined #openstack-infra | 10:57 | |
| *** liujiong has quit IRC | 11:00 | |
| *** bhavik1 has quit IRC | 11:01 | |
| *** alexchadin has quit IRC | 11:06 | |
| *** tinwood has quit IRC | 11:07 | |
| *** links has quit IRC | 11:07 | |
| *** tinwood has joined #openstack-infra | 11:08 | |
| *** shardy is now known as shardy_lunch | 11:10 | |
| *** wolverineav has joined #openstack-infra | 11:11 | |
| *** pkovar has joined #openstack-infra | 11:11 | |
| *** pgadiya has quit IRC | 11:15 | |
| *** markvoelker has joined #openstack-infra | 11:19 | |
| *** links has joined #openstack-infra | 11:20 | |
| *** pbourke has quit IRC | 11:29 | |
| *** pbourke has joined #openstack-infra | 11:31 | |
| *** gabor_antal_km has joined #openstack-infra | 11:33 | |
| gabor_antal_km | Hey there! Would someone please let in out company modification patch in stackaltics? https://review.openstack.org/#/c/499764/ | 11:34 |
| *** _mugsie_ has joined #openstack-infra | 11:39 | |
| *** slaweq has joined #openstack-infra | 11:41 | |
| *** links has quit IRC | 11:42 | |
| *** lucasagomes is now known as lucas-hungry | 11:43 | |
| *** liujiong_lj has quit IRC | 11:44 | |
| *** shardy_lunch is now known as shardy | 11:44 | |
| *** mugsie is now known as mugsie__ | 11:45 | |
| *** _mugsie_ is now known as mugsie | 11:45 | |
| *** mugsie has quit IRC | 11:45 | |
| *** mugsie has joined #openstack-infra | 11:45 | |
| *** mugsie__ has quit IRC | 11:46 | |
| *** slaweq has quit IRC | 11:48 | |
| *** jpena is now known as jpena|lunch | 11:51 | |
| *** jkilpatr has joined #openstack-infra | 11:51 | |
| *** markvoelker has quit IRC | 11:51 | |
| *** dtantsur|bbl is now known as dtantsur | 11:52 | |
| *** baoli has joined #openstack-infra | 11:54 | |
| *** links has joined #openstack-infra | 11:56 | |
| *** baoli has quit IRC | 11:59 | |
| ttx | gabor_antal_km: you need someone from stackalytics-core to process the patch: https://review.openstack.org/#/admin/groups/183,members | 12:04 |
| *** alexchadin has joined #openstack-infra | 12:07 | |
| openstackgerrit | Lingxian Kong proposed openstack-infra/project-config master: Change Qinling devstack job https://review.openstack.org/500525 | 12:07 |
| *** alexchadin has quit IRC | 12:11 | |
| *** erlon has joined #openstack-infra | 12:14 | |
| openstackgerrit | Marton Kiss proposed openstack-infra/puppet-askbot master: Add missing www-browser dependency for apachectl status https://review.openstack.org/500528 | 12:25 |
| *** lucas-hungry is now known as lucasagomes | 12:27 | |
| *** udesale has joined #openstack-infra | 12:28 | |
| *** jaypipes has joined #openstack-infra | 12:33 | |
| *** alexchadin has joined #openstack-infra | 12:39 | |
| *** LindaWang has quit IRC | 12:42 | |
| *** EricGonczer_ has joined #openstack-infra | 12:43 | |
| *** EricGonczer_ has quit IRC | 12:44 | |
| mordred | ianw: always | 12:48 |
| *** markvoelker has joined #openstack-infra | 12:48 | |
| *** pcrews has quit IRC | 12:56 | |
| *** gcb has joined #openstack-infra | 13:00 | |
| *** jpena|lunch is now known as jpena | 13:01 | |
| sshnaidm | hi, can anybody tell exactly how do we mirror docker - by apache reverse proxy or docker mirror (https://docs.docker.com/registry/recipes/mirror/) ? | 13:01 |
| sshnaidm | pabelanger, ^^ | 13:02 |
| *** tosky has joined #openstack-infra | 13:05 | |
| *** nicolasbock has joined #openstack-infra | 13:10 | |
| *** Sree has joined #openstack-infra | 13:13 | |
| *** baoli has joined #openstack-infra | 13:14 | |
| gabor_antal_km | ttx: thanks, I hoped someone is here from them, but i'm unlucky it seems | 13:15 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: WIP Force keystone v2 admin call to admin interface https://review.openstack.org/500540 | 13:17 |
| pabelanger | sshnaidm: reverse proxy cache http://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/templates/mirror.vhost.erb#n198 | 13:18 |
| pabelanger | sshnaidm: we don't have any docker binaries running on our control plane | 13:18 |
| sshnaidm | pabelanger, thanks, so do we configure docker right for using it? | 13:19 |
| pabelanger | sshnaidm: yes, tripleo specifically has been using it for the last month | 13:20 |
| *** markvoelker has quit IRC | 13:21 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: sql: normalize logger name https://review.openstack.org/500431 | 13:30 |
| *** sshnaidm is now known as sshnaidm|mtg | 13:31 | |
| *** LindaWang has joined #openstack-infra | 13:33 | |
| *** ldnunes has joined #openstack-infra | 13:33 | |
| *** bobh has joined #openstack-infra | 13:37 | |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack-infra/shade master: De-client-ify Check User in Group https://review.openstack.org/499357 | 13:39 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack-infra/shade master: De-client-ify Check User in Group https://review.openstack.org/499357 | 13:42 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack-infra/shade master: De-client-ify Remove User from Group https://review.openstack.org/499360 | 13:43 |
| *** pkovar has left #openstack-infra | 13:48 | |
| *** hashar is now known as hasharAway | 13:49 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Add python backport PPA during stream test https://review.openstack.org/500549 | 13:50 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Set ansible_python_interpreter in stream test https://review.openstack.org/500550 | 13:50 |
| *** slaweq has joined #openstack-infra | 13:50 | |
| *** slaweq has quit IRC | 13:55 | |
| *** bobh has quit IRC | 13:57 | |
| mnaser | morning folks -- any idea why https://review.openstack.org/#/c/500335/ is not starting gate jobs? | 13:58 |
| fungi | most common causes are unapproved git parents, unmerged cross-repo dependencies, or dependency loops (often in children of the change in question) | 14:00 |
| fungi | the first two don't seem to be the issue, the third is much harder to track down | 14:00 |
| fungi | do you happen to know whether there are any changes in other repos declaring a cross-repo dependency on this one? | 14:01 |
| fungi | oh, fourth common cause is bad/greedy skip-if rules in the zuul config skipping all jobs for a particular set of files in a change | 14:01 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Remove use of legacy keystone client in functional tests https://review.openstack.org/500558 | 14:02 |
| fungi | one more common cause is approving before adding any code review +2, which also doesn't seem to be the case here | 14:02 |
| openstackgerrit | Samuel de Medeiros Queiroz proposed openstack-infra/shade master: Remove improper exc handling in is_user_in_group https://review.openstack.org/500560 | 14:03 |
| mordred | morning fungi! | 14:03 |
| pabelanger | puppet jobs also depend a lot on FileMatcher, so maybe patch doesn't match any jobs now? | 14:04 |
| mordred | fungi: feel like a reasonably easy review? https://review.openstack.org/#/c/500226 | 14:04 |
| fungi | morning mordred. how are you enjoying your canadian holiday? | 14:04 |
| mordred | fungi: I feel very canadian | 14:04 |
| *** udesale has quit IRC | 14:05 | |
| mordred | fungi: I'm wearing moose antlers and drinking a nice large cup of maple syrup while holding a hockey stick - that's the right way to celebrate today ... right pabelanger ? | 14:05 |
| pabelanger | indeed! I'm hanging out in the igloo this morning too | 14:05 |
| mordred | fungi: if you feel slightly more adventurous - the rest of that stack gets us all the way to having the wheel mirror builders being run out of zuulv3 and not in zuulv2 | 14:06 |
| *** udesale has joined #openstack-infra | 14:07 | |
| mordred | fungi: oh - also, I don't know if you were around when we realized it - but we actually don't need those long-lived builders - we DON'T reuse cache on them - two more patches after those and we can delete those servers | 14:08 |
| fungi | mordred: yep, caught that discussion. excellent news! | 14:08 |
| *** efoley has joined #openstack-infra | 14:10 | |
| *** efoley has quit IRC | 14:10 | |
| *** efoley has joined #openstack-infra | 14:10 | |
| *** efoley has quit IRC | 14:11 | |
| *** efoley has joined #openstack-infra | 14:11 | |
| fungi | mnaser: EmilienM: bingo... "Exception: Dependency cycle detected: 500318 in ['500318', u'500317']" | 14:12 |
| mnaser | did i break somethingggg | 14:12 |
| mnaser | i mean i can drop that depends-on | 14:12 |
| fungi | er, well maybe... i'm running those down but the timestamp from that traceback in the log was right at the same time the change in question got approved | 14:12 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add trusty images to nodepool v3 https://review.openstack.org/500564 | 14:12 |
| mordred | fungi, pabelanger: ^^ patch #1 for no-more-long-lived-wheel-mirrors | 14:13 |
| mnaser | ah crap | 14:13 |
| mnaser | https://review.openstack.org/#/c/500318/ | 14:13 |
| mnaser | the depends-on in that one is wrong | 14:13 |
| mnaser | err | 14:13 |
| mnaser | https://review.openstack.org/#/c/500317/ | 14:13 |
| mnaser | i wanted to have it depend on the project-config change | 14:13 |
| *** Sree has quit IRC | 14:14 | |
| mnaser | https://review.openstack.org/#/c/500334/ one specifically | 14:14 |
| mnaser | wait, it does depend on it | 14:14 |
| pabelanger | mordred: looking | 14:14 |
| * mnaser is confused | 14:14 | |
| fungi | mnaser: i'm trying to trace back through the dep tree from that cycle to 500335 | 14:15 |
| mnaser | https://review.openstack.org/#/c/500335/ (change that is not merging) depends on https://review.openstack.org/#/c/500334/ (already merged!) which does not depend on anything | 14:15 |
| mnaser | it was a bit of a mess i admit so the logs might have been complicated to go through | 14:15 |
| *** efoley has quit IRC | 14:15 | |
| fungi | mnaser: yes, but the issue can also be in changes which depend _on_ 500335 having a dependency cycle which causes zuul to give up trying to figure out what order things stack in | 14:16 |
| pabelanger | mordred: fungi: +2 | 14:16 |
| fungi | even if 500335 is closer to head than the dependency loop | 14:16 |
| EmilienM | hello | 14:16 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Move ara output generation to post playbook https://review.openstack.org/500403 | 14:16 |
| mnaser | i guess maybe better off avoiding fancy depends-on's :p | 14:17 |
| * EmilienM happy to not being alone today :) | 14:17 | |
| mordred | pabelanger: piddle. getting afs client on the nodes will be harder - will need to wait til after ptg - because we need to make a role equiv to puppet-openafs to handle centos ... OH WAIT - I just had a very very evil realization | 14:17 |
| mordred | pabelanger: I'm going to spend 10 minutes on a patch - then I think you're going to laugh | 14:18 |
| fungi | mnaser: 500317 depends-on 500318 which depends-on (500317 and 500335) | 14:18 |
| pabelanger | mordred: Right, we don't have openafs installed... kk waiting | 14:18 |
| fungi | mnaser: and so 500335 isn't being tested now | 14:18 |
| mnaser | ok the depends-on in https://review.openstack.org/#/c/500317/ is wrong | 14:18 |
| mnaser | that's the reason | 14:19 |
| mnaser | it should have been the project-config change, sorry | 14:19 |
| openstackgerrit | Merged openstack-infra/project-config master: tripleo: remove centos-7-nonha-multinode-updates job https://review.openstack.org/500249 | 14:19 |
| mnaser | fungi i believe everything should be correct now and 500317 has the right depends-on (which already merged) | 14:20 |
| mnaser | sorry for the noise :< | 14:20 |
| *** ykarel has quit IRC | 14:21 | |
| mnaser | fungi would a 'recheck' on the cinder job be needed now or it will be automatically retriggered | 14:22 |
| mnaser | s/job/change/ | 14:22 |
| openstackgerrit | Jakub Libosvar proposed openstack-infra/project-config master: neutron: Make grenade-neutron-dvr-multinode job non-voting https://review.openstack.org/500567 | 14:25 |
| fungi | mnaser: 500335 will need another approval (either EmilienM removing and readding his or you adding a second approval vote) | 14:26 |
| fungi | rechecking won't help since that change already has a verify +1 | 14:26 |
| * mnaser can't approve things yet | 14:26 | |
| EmilienM | ok let me approve it | 14:28 |
| EmilienM | it's moving in gate now | 14:28 |
| mnaser | yay, thanks EmilienM ! | 14:29 |
| EmilienM | cool | 14:29 |
| EmilienM | thanks for your work :) | 14:29 |
| *** gcb has quit IRC | 14:29 | |
| mnaser | np :> | 14:29 |
| * mnaser goes back to neutron wsgi fun | 14:29 | |
| fungi | mnaser: you're the ptl but not a core reviewer? that's pretty awesome division of labor | 14:29 |
| openstackgerrit | Jakub Libosvar proposed openstack-infra/project-config master: neutron: Make grenade-neutron-dvr-multinode job non-voting https://review.openstack.org/500567 | 14:29 |
| mnaser | fungi i believe the change in groups/whatnot will be done at some point, but i'm in no rush, plenty of folks who can help approve things :) | 14:30 |
| fungi | granted a little confusing since core reviewers are actually considered appointees of the ptl | 14:30 |
| fungi | mordred: okay, reviewing now | 14:31 |
| clarkb | thats a neat trick | 14:31 |
| *** pcrews has joined #openstack-infra | 14:33 | |
| *** amotoki is now known as amotoki__away | 14:34 | |
| *** udesale has quit IRC | 14:35 | |
| *** ccamacho has quit IRC | 14:36 | |
| *** ccamacho has joined #openstack-infra | 14:37 | |
| *** links has quit IRC | 14:40 | |
| *** armax has joined #openstack-infra | 14:40 | |
| mnaser | is zm04.openstack.org okay? | 14:41 |
| mnaser | http://logs.openstack.org/17/500317/12/check/gate-puppet-ceph-puppet-syntax-4-centos-7/9507183/console.html | 14:41 |
| fungi | clarkb: were you planning to review 497699 or are we safe to go ahead there (it's got several +2s and the logs look fine other than that job timeout is really tight in infracloud) | 14:41 |
| mnaser | stderr: 'fatal: unable to access 'http://zm04.openstack.org/p/openstack/puppet-ceph/': The requested URL returned error: 500' | 14:41 |
| fungi | mnaser: i suspect that's continued network connectivity issues manifesting in citycloud-sto2 | 14:42 |
| mnaser | wouldn't that be a timeout or something instead of a 500 (unless zm04 is hosted there) | 14:43 |
| fungi | ahh yep that's a 500 error | 14:43 |
| clarkb | fungi: I think you cam go ahead with it. Not sure I will get around to that today | 14:43 |
| fungi | clarkb: okay, i just didn't know whether we were waiting for you to have an opportunity to look at it | 14:44 |
| openstackgerrit | Merged openstack-infra/shade master: De-client-ify User Update https://review.openstack.org/499284 | 14:45 |
| fungi | mnaser: i do see a fairly large outbound traffic spike on zm04 right at that time, though with 5-minute granularity from snmp polling it's hard to say how high it really went | 14:46 |
| fungi | corresponding spike in system cpu utilization as well | 14:46 |
| fungi | [Mon Sep 04 14:22:23.730541 2017] [cgid:error] [pid 2540:tid 139758755895040] [client 188.240.223.225:45816] End of script output before headers: git-http-backend | 14:48 |
| mnaser | probably timed out or crashed in relation to that cpu usage | 14:48 |
| fungi | confirmed that client ip address matches the fip in our ansible inventory logged from that job | 14:49 |
| fungi | nothing related in dmesg/syslog so that doesn't seem to have been from widespread distress on the system anyway | 14:50 |
| *** slaweq has joined #openstack-infra | 14:51 | |
| *** ianychoi_ is now known as ianychoi | 14:52 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: DNM: test wget https://review.openstack.org/500426 | 14:52 |
| fungi | zm04 is reasonably busy right now but doesn't look overtaxed | 14:52 |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: Update to using zuulv3 variables https://review.openstack.org/500229 | 14:53 |
| *** sshnaidm|mtg is now known as sshnaidm | 14:53 | |
| *** udesale has joined #openstack-infra | 14:54 | |
| *** slaweq has quit IRC | 14:56 | |
| *** baoli has quit IRC | 14:57 | |
| fungi | mordred: seems like it would make sense for the periodic pipeline to be considered post-merge... it doesn't run against proposed changes | 14:57 |
| *** udesale has quit IRC | 14:58 | |
| *** aarefiev is now known as aarefiev_afk | 14:59 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: Update to using zuulv3 variables https://review.openstack.org/500229 | 14:59 |
| *** nicolasbock has quit IRC | 15:00 | |
| *** EricGonczer_ has joined #openstack-infra | 15:01 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: DNM: test wget https://review.openstack.org/500426 | 15:02 |
| *** ykarel has joined #openstack-infra | 15:04 | |
| *** alexchadin has quit IRC | 15:05 | |
| *** alexchadin has joined #openstack-infra | 15:05 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: Update to using zuulv3 variables https://review.openstack.org/500229 | 15:06 |
| *** alexchadin has quit IRC | 15:06 | |
| *** thorst_afk has joined #openstack-infra | 15:07 | |
| pabelanger | fungi: clarkb: looks like tox 2.8.1 is out, did we want to start the unpinning process? | 15:10 |
| mordred | fungi: good point | 15:15 |
| clarkb | pabelanger: probably start with constraints and make sure tempest py35 works first | 15:15 |
| pabelanger | mordred: https://review.openstack.org/500229/ is green now. checking against zuulv3 and shade to see if role works | 15:15 |
| pabelanger | clarkb: ya, good idea | 15:16 |
| mrmartin | fungi: I made this patch to enable the apachectl status on ask.o.o sites, so we can get a detailed worker status when apache fails to serve the requests: https://review.openstack.org/#/c/500528/ | 15:17 |
| mordred | pabelanger: I think we should promote nodepool/elements/nodepool-base/install.d/06-record-builddate into an element in base nodepool - I was just writing a little piece of doc about it, and it seems like a thing that's just handy to have for folks using dib for images | 15:20 |
| *** thorst_afk has quit IRC | 15:21 | |
| pabelanger | mordred: ah, I always thought that was by default. Seems like a good idea | 15:21 |
| pabelanger | clarkb: remote: https://review.openstack.org/500583 Revert "Add tox to upper-constraints to help pin the gate" | 15:22 |
| mordred | pabelanger: also nodepool/elements/nodepool-base/install.d/05-record-details - and it might not be totally stupid to add one that records elements used too | 15:22 |
| *** thorst_afk has joined #openstack-infra | 15:23 | |
| *** armax_ has joined #openstack-infra | 15:23 | |
| *** gmann has quit IRC | 15:24 | |
| *** gmann has joined #openstack-infra | 15:24 | |
| pabelanger | mordred: maybe move them into /etc/dib-manifests folder if we are reorging, but agree with move upstream | 15:24 |
| fungi | mrmartin: at the moment you can just pull up https://ask.openstack.org/server-status in a browser even (it doesn't appear to be restricted to access only by localhost) | 15:24 |
| mordred | pabelanger, jeblair: I also just realized there is a hole in our validate_hosts role (similar to one jeblair brought up when we were looking at secrets reorg) - fixing now | 15:25 |
| *** armax has quit IRC | 15:25 | |
| *** armax_ is now known as armax | 15:25 | |
| mrmartin | fungi: oh, don't we like to limit this access to localhost only? | 15:25 |
| *** thorst_afk has quit IRC | 15:25 | |
| fungi | mrmartin: we try to elsewhere, though at the moment we're running into issues getting it to work correctly on multi-vhost servers (particularly files01.o.o and static.o.o) | 15:26 |
| openstackgerrit | Paul Belanger proposed openstack-infra/shade master: Add openstack-doc-build to shade https://review.openstack.org/500201 | 15:27 |
| *** bobh has joined #openstack-infra | 15:34 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/openstack-zuul-jobs master: Update to using zuulv3 variables https://review.openstack.org/500229 | 15:35 |
| *** coolsvap has quit IRC | 15:37 | |
| *** gouthamr has joined #openstack-infra | 15:37 | |
| *** LindaWang has quit IRC | 15:38 | |
| *** ccamacho has quit IRC | 15:38 | |
| fungi | mrmartin: if you look at the default /etc/apache2/mods-enabled/status.conf provided by the system package you'll see it attempts to do a "Require local" in <Location /server-status> but something in some of our vhost definitions is then overriding that and allowing full public access instead. as of yet i haven't been able to spend enough time to work out what's doing that | 15:39 |
| pabelanger | mordred: woot | 15:40 |
| pabelanger | http://logs.openstack.org/36/500236/2/check/openstack-doc-build/a6ad32a/html/ | 15:40 |
| *** thorst_afk has joined #openstack-infra | 15:41 | |
| *** e0ne has quit IRC | 15:41 | |
| mrmartin | fungi: ok, I'll check it. | 15:41 |
| pabelanger | mordred: until we can use zuul_return with success-url, clicking on jobs results won't take you directly to sphinx html page for openstack-doc-build on stable or tags. But it does mean, publishing to afs, works correctly | 15:42 |
| pabelanger | mordred: in fact, I think we can refactor some of that logic in zuulv3, but I think this is best effort for PTG now | 15:42 |
| fungi | maybe like something with "satisfy any" in combination with "require all granted" for the docroot directory block in the vhost is overriding the location restriction for /server-status, but reading the apache 2.4 configuration docs it seems like that's not supposed to happen | 15:42 |
| fungi | mrmartin: ^ | 15:43 |
| pabelanger | mordred: and shade: http://logs.openstack.org/01/500201/3/check/openstack-doc-build/c1b5458/html/ | 15:43 |
| openstackgerrit | David Shaughnessy proposed openstack-infra/irc-meetings master: Adding David Shaughnessy as meeting chair. https://review.openstack.org/500588 | 15:44 |
| *** thorst_afk has quit IRC | 15:45 | |
| *** bobh has quit IRC | 15:47 | |
| *** Krenair has quit IRC | 15:47 | |
| *** hongbin has joined #openstack-infra | 15:48 | |
| fungi | mrmartin: one other possibility i haven't looked into yet is that this could be caused by mod_access_compat | 15:48 |
| *** slaweq has joined #openstack-infra | 15:52 | |
| *** Krenair has joined #openstack-infra | 15:52 | |
| mrmartin | ok, thnx, now worries, I'm going to find out what is causing that. | 15:52 |
| *** davidsha has joined #openstack-infra | 15:55 | |
| *** slaweq has quit IRC | 15:56 | |
| *** lucasagomes is now known as lucas-afk | 15:58 | |
| davidsha | Hey, would someone be able to have a look at this patch: https://review.openstack.org/#/c/500588/1 The chairs are currently on leave and we need someone to be able to start the meeting tomorrow. | 15:59 |
| clarkb | davidsha: anyone can start a meeting | 16:00 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Update publish-openstack-python-docs to use tox role https://review.openstack.org/500591 | 16:00 |
| fungi | davidsha: the chairs list in the meeting definitions is just informational, in case potential attendees want to reach out to the chairs about agenda topics in advance | 16:01 |
| *** dtantsur is now known as dtantsur|bbl | 16:01 | |
| *** pblaho has quit IRC | 16:02 | |
| *** egonzalez has quit IRC | 16:02 | |
| davidsha | clarkb, fungi: Ah, Thanks! I presumed this gave people access to the actions. | 16:02 |
| *** bobh has joined #openstack-infra | 16:02 | |
| mordred | pabelanger: woot! I think we can land the stack ending at https://review.openstack.org/#/c/500229 and shift our v3 projects to publishing from v3 and not in v2 yeah? | 16:04 |
| fungi | davidsha: nope. by default the chair for any meeting is the nick who initiates the #startmeeting command, and then that person can add the nicks of additional chairs if they want to delegate some of the chair-only commands over the course of the meeting or need to drop from the meeting early and want someone else to take over chairing for the remainder of that time | 16:04 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Make validate-host read from site-variables https://review.openstack.org/500592 | 16:05 |
| pabelanger | mordred: Ya, I think so. https://review.openstack.org/500591/ will be the changes needed for publishing to afs with trusted playbooks | 16:05 |
| pabelanger | mordred: it atleasts puts us in a good spot to itterate forward as we find issues | 16:05 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Make validate-host read from site-variables https://review.openstack.org/500592 | 16:06 |
| davidsha | fungi: Ok, I understand now. Thanks! | 16:06 |
| fungi | mordred: pabelanger: why is zuul complaining about extra keys for periodic on 499843? | 16:08 |
| pabelanger | fungi: we need to land 500226 first, since zuul doesn't have periodic in its pipeline configuration | 16:09 |
| fungi | pabelanger: oh, needed a depends-on i guess? | 16:10 |
| fungi | or rebase onto that change | 16:10 |
| fungi | oh, wait, it is based on that change | 16:11 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: DNM: test wget https://review.openstack.org/500426 | 16:11 |
| fungi | pabelanger: so zuul needs to reload with 500226 in place before 499843 will pass? | 16:11 |
| pabelanger | fungi: ya | 16:12 |
| pabelanger | pipelines changes are post-merge only | 16:12 |
| fungi | got it, thanks for your patience ;) | 16:14 |
| pabelanger | np! | 16:14 |
| fungi | and yes, now i remember why that is | 16:14 |
| *** dosaboy has quit IRC | 16:16 | |
| *** dosaboy has joined #openstack-infra | 16:18 | |
| *** Goneri has joined #openstack-infra | 16:18 | |
| *** cloudnull has quit IRC | 16:18 | |
| fungi | basically, safety measure working as intended | 16:20 |
| *** cloudnull has joined #openstack-infra | 16:21 | |
| *** armax has left #openstack-infra | 16:25 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: DNM: test wget https://review.openstack.org/500426 | 16:28 |
| *** davidsha has left #openstack-infra | 16:32 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Adding David Shaughnessy as meeting chair. https://review.openstack.org/500588 | 16:33 |
| *** jpich has quit IRC | 16:45 | |
| *** tesseract has quit IRC | 16:48 | |
| *** bobh has quit IRC | 16:49 | |
| *** slaweq has joined #openstack-infra | 16:53 | |
| *** dtantsur|bbl is now known as dtantsur | 16:54 | |
| *** markvoelker has joined #openstack-infra | 16:54 | |
| *** derekh has quit IRC | 16:57 | |
| *** slaweq has quit IRC | 16:58 | |
| *** Goneri has quit IRC | 17:02 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/devstack-gate master: DNM: tox 2.8.1 testing https://review.openstack.org/500600 | 17:06 |
| *** ralonsoh has quit IRC | 17:08 | |
| *** jkilpatr has quit IRC | 17:17 | |
| *** jamesmcarthur has joined #openstack-infra | 17:17 | |
| *** jpena is now known as jpena|off | 17:19 | |
| *** sshnaidm is now known as sshnaidm|off | 17:22 | |
| *** markvoelker has quit IRC | 17:27 | |
| *** gabor_antal_km has quit IRC | 17:29 | |
| *** tosky has quit IRC | 17:30 | |
| *** psachin has quit IRC | 17:33 | |
| *** EricGonczer_ has quit IRC | 17:34 | |
| *** ramishra has quit IRC | 17:38 | |
| *** electrofelix has quit IRC | 17:42 | |
| *** jamesdenton has quit IRC | 17:48 | |
| *** slaweq has joined #openstack-infra | 17:54 | |
| *** tosky has joined #openstack-infra | 17:55 | |
| *** EricGonczer_ has joined #openstack-infra | 17:56 | |
| *** rcernin has joined #openstack-infra | 17:57 | |
| *** dtantsur is now known as dtantsur|afk | 17:57 | |
| *** slaweq has quit IRC | 17:58 | |
| *** mat128 has joined #openstack-infra | 17:59 | |
| mordred | fungi: yay safety! | 18:00 |
| *** jamesmcarthur has quit IRC | 18:10 | |
| *** EricGonczer_ has quit IRC | 18:11 | |
| *** tinwood has quit IRC | 18:13 | |
| *** ykarel has quit IRC | 18:14 | |
| *** tinwood has joined #openstack-infra | 18:15 | |
| *** gouthamr has quit IRC | 18:19 | |
| *** gouthamr has joined #openstack-infra | 18:20 | |
| *** markvoelker has joined #openstack-infra | 18:24 | |
| *** hongbin has quit IRC | 18:25 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Make validate-host read from site-variables https://review.openstack.org/500592 | 18:25 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Rework upload-logs to enable running on localhost https://review.openstack.org/500611 | 18:25 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add zuul base roles to ease sharing base job content https://review.openstack.org/500612 | 18:25 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Set validate_host parameters in site-variables https://review.openstack.org/500613 | 18:25 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Use rollup base roles in base-test job https://review.openstack.org/500614 | 18:25 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Use zuul base roll-up roles in base jobs https://review.openstack.org/500615 | 18:25 |
| smcginnis | Looks like a POST_FAILURE publishing a tarball for tripleo-heat. | 18:26 |
| *** jascott1 has quit IRC | 18:26 | |
| *** jascott1 has joined #openstack-infra | 18:26 | |
| fungi | smcginnis: we don't seem to be out of space at least... i'll see if i can tell what happened | 18:26 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Add zuul base roles to ease sharing base job content https://review.openstack.org/500612 | 18:27 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Contract back to a single upload-logs tasks file https://review.openstack.org/500618 | 18:27 |
| *** yee379 has joined #openstack-infra | 18:27 | |
| smcginnis | fungi: Thanks | 18:30 |
| fungi | found where it reported the result in zuul's debug log... tracking it down to a launcher now | 18:30 |
| *** simondodsley_ has joined #openstack-infra | 18:30 | |
| *** kmalloc_ has joined #openstack-infra | 18:30 | |
| *** edwarnicke_ has joined #openstack-infra | 18:30 | |
| *** csatari_ has joined #openstack-infra | 18:30 | |
| *** slaweq has joined #openstack-infra | 18:31 | |
| fungi | 2017-09-04 18:17:40,368 DEBUG zuul.Gearman: Build <gear.Job 0x7fa6d7cb30d0 handle: H:127.0.0.1:7116407 name: build:tripleo-heat-templates-tarball unique: 517ee89f29b7417b88c3b4f6b43af602> update {u'url': u'telnet://46.254.15.207:19885', u'manager': u'zl04', u'number': u'517ee89f29b7417b88c3b4f6b43af602'} | 18:32 |
| fungi | so zl04 | 18:32 |
| *** slaweq has quit IRC | 18:32 | |
| *** thorre_se has joined #openstack-infra | 18:33 | |
| *** StevenK_ has joined #openstack-infra | 18:33 | |
| *** mikap has joined #openstack-infra | 18:33 | |
| fungi | 2017-09-04 18:18:56,570 DEBUG zuul.NodeWorker.ubuntu-xenial-citycloud-sto2-10765911: Ansible post output: ERROR! Unexpected Exception: [Errno 17] File exists | 18:35 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Add nl02.openstack.org https://review.openstack.org/500620 | 18:36 |
| smcginnis | Hmm | 18:36 |
| *** evrardjp_ has joined #openstack-infra | 18:36 | |
| *** __Cyclone__ has joined #openstack-infra | 18:36 | |
| *** eventingmonkey_ has joined #openstack-infra | 18:36 | |
| pabelanger | infra-root: ^brings nl02.openstack.org online for xenial | 18:36 |
| *** leifmadsen_ has joined #openstack-infra | 18:36 | |
| pabelanger | nodepool-launcher is currently stopped so we can schedule cutover from nl01 when people are ready | 18:37 |
| *** saneax___ has joined #openstack-infra | 18:37 | |
| *** thorre has quit IRC | 18:37 | |
| *** yee37926 has quit IRC | 18:37 | |
| *** kambiz has quit IRC | 18:37 | |
| *** saneax has quit IRC | 18:37 | |
| *** fried_rice has quit IRC | 18:37 | |
| *** eventingmonkey has quit IRC | 18:37 | |
| *** _Cyclone_ has quit IRC | 18:37 | |
| *** evrardjp has quit IRC | 18:37 | |
| *** mika has quit IRC | 18:37 | |
| *** leifmadsen has quit IRC | 18:37 | |
| *** stephenfin has quit IRC | 18:37 | |
| *** StevenK has quit IRC | 18:37 | |
| *** simondodsley has quit IRC | 18:37 | |
| *** kmalloc has quit IRC | 18:37 | |
| *** csatari has quit IRC | 18:37 | |
| *** edwarnicke has quit IRC | 18:37 | |
| *** mrmartin has quit IRC | 18:37 | |
| *** mikap is now known as mika | 18:37 | |
| *** thorre_se is now known as thorre | 18:37 | |
| *** simondodsley_ is now known as simondodsley | 18:37 | |
| *** csatari_ is now known as csatari | 18:37 | |
| *** edwarnicke_ is now known as edwarnicke | 18:37 | |
| *** kmalloc_ is now known as kmalloc | 18:37 | |
| *** kambiz has joined #openstack-infra | 18:37 | |
| fungi | smcginnis: here's what i've managed to grep out of the launcher log: http://paste.openstack.org/show/620362/ | 18:38 |
| *** stephenfin has joined #openstack-infra | 18:38 | |
| *** jklare has quit IRC | 18:40 | |
| *** isviridov_away has quit IRC | 18:40 | |
| fungi | unfortunately it seems the playbook got cleaned up... it's not entirely clear to me what file it's objecting to existing | 18:41 |
| *** mattoliverau has quit IRC | 18:42 | |
| *** trown has quit IRC | 18:42 | |
| *** migi has quit IRC | 18:42 | |
| *** trown has joined #openstack-infra | 18:42 | |
| *** mattoliverau has joined #openstack-infra | 18:42 | |
| fungi | given all subsequent jobs were skipped, i can reenqueue the tag into the release pipeline and see if it happens again | 18:42 |
| *** migi has joined #openstack-infra | 18:42 | |
| smcginnis | fungi: Sounds good. Hopefully it was just an fluke. | 18:43 |
| *** jklare has joined #openstack-infra | 18:43 | |
| *** mdrabe has quit IRC | 18:44 | |
| *** dims has quit IRC | 18:44 | |
| *** ericyoung has quit IRC | 18:44 | |
| *** fried_rice has joined #openstack-infra | 18:45 | |
| *** oomichi has quit IRC | 18:45 | |
| fungi | smcginnis: okay, it's enqueued and rerunning the tarball job now | 18:46 |
| *** oomichi has joined #openstack-infra | 18:46 | |
| *** mat128 has quit IRC | 18:47 | |
| *** dims has joined #openstack-infra | 18:47 | |
| *** mdrabe has joined #openstack-infra | 18:47 | |
| *** ericyoung has joined #openstack-infra | 18:47 | |
| fungi | job succeeded (already) so waiting to see whether the publish works this time | 18:47 |
| smcginnis | fungi: Yeah, looking good so far. | 18:47 |
| fungi | yep! must have been a fluke... dependent jobs are running now | 18:47 |
| fungi | pypi uploaded seems to have completed | 18:47 |
| smcginnis | And done.. | 18:48 |
| smcginnis | Oh, one last job. | 18:48 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Reduce debug output for repo https://review.openstack.org/500622 | 18:49 |
| *** isviridov_away has joined #openstack-infra | 18:49 | |
| smcginnis | Now done. | 18:50 |
| smcginnis | fungi: Thanks for checking on it. | 18:50 |
| fungi | thanks for bringing it to our attention | 18:50 |
| *** hasharAway is now known as hashar | 18:51 | |
| fungi | my guess is we have some subtle bug in the zuul ansible-launcher (or maybe deeper in ansible itself). if we see similar errors in zuulv3 we should probably dig deeper at that point, but i'm not too inclined to go hunting this down in the deprecated launcher | 18:51 |
| smcginnis | Yeah, probably good to wait and see. | 18:52 |
| *** gaurangt has joined #openstack-infra | 18:52 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: DNM: test wget https://review.openstack.org/500426 | 18:57 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Reduce debug output for repo https://review.openstack.org/500622 | 18:57 |
| *** markvoelker has quit IRC | 18:57 | |
| *** baoli has joined #openstack-infra | 18:59 | |
| frickler | p-c cores: could you please take a look at https://review.openstack.org/500567 to help unblock neutron gate? | 19:01 |
| pabelanger | jeblair: clarkb: fungi: mordred: what do you think about stopping logstash-worker16.o.o to logstash-worker20.o.o. and see how logstash reacts? | 19:02 |
| *** baoli has quit IRC | 19:03 | |
| fungi | pabelanger: seems like a fine experiment | 19:04 |
| *** mrmartin has joined #openstack-infra | 19:09 | |
| mordred | pabelanger: kk | 19:11 |
| mordred | pabelanger, fungi, jeblair: you ready to see something insane? | 19:12 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add some Infra puppet module repos to Zuul v3 https://review.openstack.org/500626 | 19:13 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Make wheel mirror builders normal zuul nodes https://review.openstack.org/500627 | 19:13 |
| mordred | boom | 19:13 |
| mordred | fungi: enjoy https://review.openstack.org/500627 - wherein I use ansible to run puppet in a zuulv3 job to set up the state on the node | 19:14 |
| mordred | fungi: or, by enjoy, I could mean "stab yourself in the eyes and run away screaming" | 19:14 |
| pabelanger | #status log logstash-worker16.o.o to logstash-worker20.o.o services stopped | 19:16 |
| *** jamesmcarthur has joined #openstack-infra | 19:17 | |
| *** pabelanger has quit IRC | 19:17 | |
| *** pabelanger has joined #openstack-infra | 19:17 | |
| pabelanger | #status log logstash-worker16.o.o to logstash-worker20.o.o services stopped | 19:17 |
| openstackstatus | pabelanger: finished logging | 19:17 |
| *** shardy has quit IRC | 19:18 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add post-review flag to periodic pipeline https://review.openstack.org/500628 | 19:18 |
| pabelanger | #status log ubuntu-xenial nodepool-launcher (nl02.o.o) online | 19:18 |
| openstackstatus | pabelanger: finished logging | 19:18 |
| mordred | fungi, pabelanger: ^^ there's the followup patch to the periodic patch to add post-review flag | 19:19 |
| pabelanger | looking | 19:19 |
| *** jamesmcarthur has quit IRC | 19:19 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: De-client-ify Add User to Group https://review.openstack.org/499345 | 19:20 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: De-client-ify Check User in Group https://review.openstack.org/499357 | 19:21 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: De-client-ify Remove User from Group https://review.openstack.org/499360 | 19:21 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Remove use of legacy keystone client in functional tests https://review.openstack.org/500558 | 19:21 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Remove improper exc handling in is_user_in_group https://review.openstack.org/500560 | 19:21 |
| pabelanger | mordred: wow, 500627 is all sorts of crazy! But, don't we still need keytabs in zuulv3? | 19:21 |
| mordred | we do - but what I removed was the ssh keys | 19:26 |
| mordred | pabelanger: so it'll still use secrets to add the keytabs | 19:27 |
| mordred | and it'll add just the build keytab for the build job, and then the admin/release keytab only for the release job | 19:27 |
| pabelanger | mordred: Oh, yes. My mistake | 19:28 |
| mordred | pabelanger: I'm starting to hit my stride in terms of thinking about crazy things the jobs can do now :) | 19:28 |
| openstackgerrit | Merged openstack-infra/project-config master: Add periodic pipeline https://review.openstack.org/500226 | 19:28 |
| pabelanger | mordred: ++ | 19:28 |
| *** EricGonczer_ has joined #openstack-infra | 19:29 | |
| mordred | pabelanger: I should (not today) figure out a post-playbook for things that use the puppet role for log collection | 19:29 |
| pabelanger | mordred: Ya, pretty sure we can write a simple role to deploy one of our services on a node, and that becomes new beaker testing / wheel-mirror deploy | 19:31 |
| pabelanger | so much awesome | 19:31 |
| *** slaweq has joined #openstack-infra | 19:33 | |
| *** benj_ has quit IRC | 19:35 | |
| *** benj_ has joined #openstack-infra | 19:37 | |
| *** slaweq has quit IRC | 19:38 | |
| mnaser | this might be throwing things out there but is it possible to somehow test p-o-i jobs in zuulv3 and try seeing if there are things which are breaking which we can fix in advance? | 19:45 |
| *** markvoelker has joined #openstack-infra | 19:55 | |
| jeblair | mnaser: it's a holiday so you'll have to remind me what you mean by p-o-i :) | 19:55 |
| mnaser | jeblair puppet-openstack-integration jobs :p | 19:55 |
| mnaser | (which pretty much run for most puppet module changes, ensuring that we can deploy openstack successfully with the modules) | 19:57 |
| mnaser | (therefore, i have a feeling that it may be slightly more complicated than the usual tox or devstack based jobs) | 19:57 |
| jeblair | mnaser: ah, not yet. but those sound like good test cases -- once we have the migration script able to create those jobs, we can check them out and make sure they work right. | 19:58 |
| mnaser | jeblair cool, i'd be more than happy to help figure out any issues that come up earlier than later | 19:59 |
| jeblair | thanks! | 19:59 |
| *** [HeOS] has joined #openstack-infra | 20:10 | |
| *** HeOS has quit IRC | 20:11 | |
| fungi | mordred: huh... so that's a 6-node job mixing nodes from three different linux distros/releases? neat! | 20:18 |
| fungi | it had dawned on me not too far back that zuulv3's multinode concept does not preclude us from mixing nodes booted from wholly different images, or on diverse architectures even | 20:19 |
| fungi | so long as the complete set of nodes we'd need could all be satisfied in the same provider, i guess | 20:21 |
| openstackgerrit | Merged openstack-infra/project-config master: Add post-review flag to periodic pipeline https://review.openstack.org/500628 | 20:22 |
| *** ldnunes has quit IRC | 20:26 | |
| *** markvoelker has quit IRC | 20:28 | |
| *** slaweq has joined #openstack-infra | 20:34 | |
| *** [HeOS] has quit IRC | 20:37 | |
| *** slaweq has quit IRC | 20:39 | |
| *** ldnunes has joined #openstack-infra | 20:39 | |
| *** slaweq has joined #openstack-infra | 20:42 | |
| *** [HeOS] has joined #openstack-infra | 20:43 | |
| *** hashar has quit IRC | 20:44 | |
| *** rcernin has quit IRC | 20:45 | |
| *** thorst_afk has joined #openstack-infra | 20:47 | |
| *** thorst_afk has quit IRC | 20:51 | |
| *** hongbin has joined #openstack-infra | 20:59 | |
| *** jaypipes has quit IRC | 21:01 | |
| *** slaweq has quit IRC | 21:02 | |
| mordred | fungi: yup! | 21:03 |
| *** rkukura has joined #openstack-infra | 21:03 | |
| openstackgerrit | Merged openstack-infra/shade master: De-client-ify Add User to Group https://review.openstack.org/499345 | 21:05 |
| *** thorst_afk has joined #openstack-infra | 21:07 | |
| *** florianf has quit IRC | 21:07 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Reduce debug output for repo https://review.openstack.org/500622 | 21:12 |
| *** thorst_afk has quit IRC | 21:12 | |
| *** ldnunes has quit IRC | 21:17 | |
| *** pcaruana has quit IRC | 21:20 | |
| *** baoli has joined #openstack-infra | 21:21 | |
| *** jamesmcarthur has joined #openstack-infra | 21:23 | |
| *** markvoelker has joined #openstack-infra | 21:25 | |
| *** baoli has quit IRC | 21:25 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Make wheel mirror builders normal zuul nodes https://review.openstack.org/500627 | 21:26 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add some Infra puppet module repos to Zuul v3 https://review.openstack.org/500626 | 21:26 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Treat wheel builders more like build nodes https://review.openstack.org/500217 | 21:27 |
| *** jamesmcarthur has quit IRC | 21:27 | |
| *** b4rti has joined #openstack-infra | 21:37 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/elastic-recheck master: Add query for bug 1715029 https://review.openstack.org/500635 | 21:42 |
| openstack | bug 1715029 in tripleo "[overcloud.AllNodesDeploySteps.ControllerDeployment_Step3]: CREATE_FAILED Resource CREATE failed: Operation cancelled" [Undecided,New] https://launchpad.net/bugs/1715029 | 21:42 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-jenkins master: Revert "Pin tox to 2.7.0" https://review.openstack.org/500636 | 21:44 |
| *** jtomasek has quit IRC | 21:44 | |
| *** aeng has joined #openstack-infra | 21:45 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Revert "Pin tox to 2.7.0" https://review.openstack.org/500637 | 21:45 |
| fungi | pabelanger: is http://downloads.asterisk.org/pub/security/AST-2017-006.html anything we need to worry about for pbx.o.o? | 21:49 |
| fungi | not sure whether the app_minivm module is something we're using | 21:49 |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Add query for bug 1715029 https://review.openstack.org/500635 | 21:50 |
| openstack | bug 1715029 in tripleo "[overcloud.AllNodesDeploySteps.ControllerDeployment_Step3]: CREATE_FAILED Resource CREATE failed: Operation cancelled" [Undecided,New] https://launchpad.net/bugs/1715029 | 21:50 |
| pabelanger | fungi: I'll double check, but pretty sure we didn't load that module | 21:57 |
| fungi | pabelanger: thanks, seemed like that might be the case but i don't feel certain of my ability to confirm it conclusively | 21:58 |
| *** markvoelker has quit IRC | 21:58 | |
| pabelanger | fungi: ya, we are not using it | 21:58 |
| pabelanger | fungi: we're also protected from the other 2 recent security, strictrtp (I think) and pjsip | 21:58 |
| fungi | pabelanger: thanks for checking! | 21:59 |
| pabelanger | fungi: also, sudo asterisk -rx | 22:01 |
| pabelanger | gets you to console | 22:01 |
| pabelanger | then you can do | 22:01 |
| pabelanger | *CLI> core show modules | 22:01 |
| pabelanger | that lists what we have loaded | 22:01 |
| fungi | asterisk: option requires an argument -- 'x' | 22:01 |
| pabelanger | doh | 22:01 |
| pabelanger | no x | 22:01 |
| pabelanger | sudo asterisk -r | 22:01 |
| pabelanger | sigh | 22:02 |
| pabelanger | *CLI> modules show | 22:02 |
| fungi | i guess the constant stream of "Wrong password" errors is just internet background radiation? | 22:02 |
| pabelanger | fungi: Yup | 22:02 |
| pabelanger | fungi: we likely could run fail2ban on the server to block them after x attempts | 22:03 |
| pabelanger | otherwise, will just spam logs | 22:03 |
| fungi | No such command 'modules show' (type 'core show help modules show' for other possible commands) | 22:03 |
| pabelanger | sorry | 22:03 |
| pabelanger | module show | 22:03 |
| pabelanger | will get the commands right | 22:03 |
| fungi | that works great, thanks!!! | 22:04 |
| fungi | does a use count of 0 mean it's not loaded, or just not being used by any clients at the moment? | 22:04 |
| pabelanger | so, if you see the module listed, it is loaded into memory. Count is how many channels are interacting with said module | 22:05 |
| pabelanger | or threads | 22:05 |
| pabelanger | can never remember | 22:05 |
| fungi | thanks, that makes sense | 22:05 |
| openstackgerrit | Jamie Lennox proposed openstack-infra/zuul feature/zuulv3: Print SIGTERM logging to debug https://review.openstack.org/500476 | 22:05 |
| pabelanger | fungi: *CLI> core show channels | 22:06 |
| fungi | interesting that the interactive shell seems to want ctrl-c (interrupt) to exit rather than ctrl-d (end of file) | 22:06 |
| pabelanger | will show open channels | 22:06 |
| pabelanger | looks like people are in conference | 22:06 |
| pabelanger | could be hung channels are people actually using it | 22:06 |
| *** wolverineav has quit IRC | 22:10 | |
| pabelanger | 2441141 calls processed | 22:10 |
| pabelanger | still impressive is just spammers | 22:10 |
| *** thorst_afk has joined #openstack-infra | 22:13 | |
| *** bobh has joined #openstack-infra | 22:17 | |
| *** thorst_afk has quit IRC | 22:18 | |
| fungi | constant load-testing provided free by "the internet" | 22:22 |
| *** bobh has quit IRC | 22:22 | |
| pabelanger | indeed | 22:23 |
| *** thorst_afk has joined #openstack-infra | 22:34 | |
| *** EricGonczer_ has quit IRC | 22:37 | |
| *** thorst_afk has quit IRC | 22:39 | |
| *** markvoelker has joined #openstack-infra | 22:54 | |
| *** rwsu has joined #openstack-infra | 22:59 | |
| openstackgerrit | Merged openstack-infra/shade master: De-client-ify Check User in Group https://review.openstack.org/499357 | 22:59 |
| *** aeng has quit IRC | 23:03 | |
| *** aeng has joined #openstack-infra | 23:06 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul-jobs master: Ignore errors from ara generate https://review.openstack.org/500645 | 23:09 |
| *** rama_y has joined #openstack-infra | 23:21 | |
| *** tosky has quit IRC | 23:23 | |
| *** markvoelker has quit IRC | 23:29 | |
| *** ijw has joined #openstack-infra | 23:33 | |
| *** jamesmcarthur has joined #openstack-infra | 23:38 | |
| *** jamesmcarthur has quit IRC | 23:42 | |
| *** ijw has quit IRC | 23:44 | |
| *** hongbin has quit IRC | 23:44 | |
| *** EricGonczer_ has joined #openstack-infra | 23:49 | |
| *** thorst_afk has joined #openstack-infra | 23:50 | |
| *** ijw has joined #openstack-infra | 23:51 | |
| *** baoli has joined #openstack-infra | 23:51 | |
| *** thorst_afk has quit IRC | 23:55 | |
| *** baoli has quit IRC | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!