| jeblair | fungi, jlvillal: yes, though i'm more confused by why it's complaining at all. like, how did this start enforcing.... hrm | 00:00 |
|---|---|---|
| jlvillal | jeblair: Got me. | 00:00 |
| jeblair | jlvillal: did it actually reject that push? or was it just a warning message? | 00:00 |
| jlvillal | jeblair: Rejected as far as I could tell | 00:00 |
| *** dingyichen has joined #openstack-infra | 00:01 | |
| jlvillal | jeblair: No, I was wrong. I think | 00:01 |
| jlvillal | http://paste.openstack.org/show/617191/ | 00:01 |
| jlvillal | I think it is a warning, but not the real reason it was rejected. | 00:01 |
| jeblair | jlvillal: agreed | 00:02 |
| cloudnull | fungi: so the host running volume mirror.regionone.osic-cloud1.openstack.org/main01 is down | 00:03 |
| cloudnull | locked up | 00:03 |
| cloudnull | I have to cycle it | 00:03 |
| jeblair | jlvillal: do you maybe want to unabandon 427395 and push your cherry-pick on top of that? | 00:03 |
| cloudnull | volume ID 975d5bd9-37f5-4cfe-965a-a917056de961 | 00:03 |
| jlvillal | jeblair: Yeah, I may have to unabandon about 40 patches... | 00:03 |
| cloudnull | I assume that the two volumes are in a RAID 1 ? | 00:04 |
| jlvillal | jeblair: trying to decide what I want to do. New change-ids or unabandon patches... | 00:04 |
| fungi | cloudnull: that would explain things, thanks! and no, the two volumes are separate | 00:04 |
| cloudnull | ah ok . | 00:04 |
| cloudnull | i'll try to get it back asap | 00:04 |
| jeblair | jlvillal: right. that's the alternative. | 00:04 |
| fungi | cloudnull: there's no hurry, it's non-impacting at the moment. i could also just create a new volume | 00:05 |
| clarkb | cloudnull: we've avoided raiding cinder volumes in part because there is no indication of whether or not youd' get a volume from different devices or even hosts | 00:05 |
| fungi | if it becomes urgent | 00:05 |
| clarkb | cloudnull: so its a false sense of security | 00:05 |
| fungi | yeah, without an actual anti-affinity api to cinder volume placement, software raid strikes me as silly | 00:05 |
| clarkb | though I guess our logs filesystem is basically a raid0 | 00:05 |
| *** vhosakot has quit IRC | 00:06 | |
| fungi | more like a jbod | 00:06 |
| fungi | we don't stripe writes | 00:06 |
| *** pahuang has quit IRC | 00:06 | |
| cloudnull | fungi: what ever is easiest for everyone. | 00:07 |
| cloudnull | if nuking and re-building is simple and fast by all means go ahead | 00:07 |
| cloudnull | I'm going to try and get the host back online regardless. | 00:07 |
| fungi | cloudnull: if i delete the volume, will that be likely to get the instance out of an error state? or am i likely to need to boot a replacement instance too? | 00:07 |
| cloudnull | however it's possible this is due to some hardware failure and its finally just giving up | 00:08 |
| cloudnull | I can reset the state of the instance | 00:08 |
| fungi | if you're going to poke at it anyway, then i'll wait | 00:08 |
| cloudnull | ok | 00:08 |
| fungi | if you decide it's not worth getting it running, lmk and i'll happily rebuild the broken bits on our end | 00:08 |
| *** jkilpatr has quit IRC | 00:08 | |
| cloudnull | ok | 00:09 |
| cloudnull | it'll be a few | 00:09 |
| fungi | no rush, and thanks for whatever you come up with! | 00:09 |
| cloudnull | these hp nodes take ~20 min to cycle | 00:09 |
| fungi | yeah, tell me about it. i used to deal with their earlier incarnations i expect | 00:10 |
| fungi | though i also used to deal with mainframes that took hours to boot, so by comparison... | 00:10 |
| fungi | could be worse! | 00:10 |
| fungi | i need to go do some chores for a few minutes so i can settle in before the tc office hour starts in 45 minutes, but will be back shortly | 00:12 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Wrap handle_keys with debug statement https://review.openstack.org/489777 | 00:13 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Fix webapp path parsing https://review.openstack.org/489779 | 00:13 |
| dmsimard | mordred: re - pre-release, so what I want is "{name}-pypi-both-upload" and in release too, I guess ? | 00:13 |
| dmsimard | I've been publishing releases to pypi manually after tagging a release | 00:14 |
| dmsimard | but openstack is already authorized to push on pypi for ara | 00:14 |
| *** tosky has quit IRC | 00:15 | |
| openstackgerrit | Merged openstack-infra/infra-manual master: Add zuul v3 migration doc https://review.openstack.org/489677 | 00:16 |
| *** gmann has quit IRC | 00:16 | |
| *** gmann has joined #openstack-infra | 00:16 | |
| clarkb | dmsimard: yes that sounds right | 00:17 |
| clarkb | dmsimard: you also need the appropriate artifact build jobs | 00:17 |
| clarkb | dmsimard: there should be examples in the openstack release job group | 00:17 |
| dmsimard | clarkb: thanks! I'll look around. | 00:18 |
| fungi | there were a number of years where we only uploaded wheels for pre-releases but sdists and wheels for full release versions. i think a year or so ago we basically stopped doing the wheel-only uploads to pypi for pre-releases and just upload both regardless now | 00:18 |
| dmsimard | fungi: yeah it turns out it's safe to upload pre-releases to pypi, I didn't out of precaution so that pre-releases would not get installed due to higher version number | 00:19 |
| dmsimard | so knowing that now, I'll leave my fate in the hands of zuul and automated releasing \o/ | 00:19 |
| clarkb | dmsimard: it wasn't safe once upon a time but that was many years ago and many pip releases | 00:19 |
| *** mat128 has quit IRC | 00:19 | |
| fungi | older versions of pip didn't know any better (maybe pip<=1.3?) but it was also the same point where pip learned to find and install wheels | 00:20 |
| fungi | hence our workaround, if your pip was new enough to find the wheel, it was also new enough to know to prefer release versions unless you told it pre-releases were okay | 00:20 |
| *** jamesmcarthur has joined #openstack-infra | 00:20 | |
| clarkb | jlvillal: are we essentially force pushing some external history into the ironic tempest plugin repo? | 00:21 |
| * clarkb wonders how we ended up with so many changes all at once for it | 00:21 | |
| fungi | clarkb: he said something about backports? | 00:21 |
| fungi | guessing a mass backport | 00:21 |
| clarkb | these appear to be against master though | 00:21 |
| fungi | strange | 00:22 |
| fungi | oh, i bet these are split out of another repo, and we didn't import the initial repo state from a git filter-branch of those files | 00:22 |
| fungi | now that everyone is required to put their tempest plugins in separate repos from other stuff | 00:23 |
| *** pahuang has joined #openstack-infra | 00:23 | |
| *** jamesmcarthur has quit IRC | 00:25 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] LVM support for dib-block-device https://review.openstack.org/472065 | 00:26 |
| *** slaweq has joined #openstack-infra | 00:30 | |
| *** s-shiono has joined #openstack-infra | 00:31 | |
| *** rloo has quit IRC | 00:31 | |
| *** DFFlanders has quit IRC | 00:33 | |
| dmsimard | ianw: is f26 live yet ? | 00:35 |
| *** baoli has joined #openstack-infra | 00:35 | |
| dmsimard | not impatient, just wondering :) | 00:35 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config master: Add pre-release and release automation for ARA https://review.openstack.org/489817 | 00:35 |
| *** slaweq has quit IRC | 00:35 | |
| *** baoli has quit IRC | 00:35 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config master: Add pre-release and release automation for ARA https://review.openstack.org/489817 | 00:35 |
| *** DFFlanders has joined #openstack-infra | 00:36 | |
| dmsimard | clarkb: ^ I think that should do it, thanks for pointing me in the right direction :D | 00:36 |
| *** baoli has joined #openstack-infra | 00:36 | |
| ianw | dmsimard: i haven't started the builds, but we could | 00:36 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/project-config master: Add pre-release and release automation for ARA https://review.openstack.org/489817 | 00:37 |
| *** sree_ has joined #openstack-infra | 00:37 | |
| dmsimard | ianw: ok! | 00:37 |
| *** sree_ is now known as Guest40523 | 00:37 | |
| *** thorst_afk has joined #openstack-infra | 00:40 | |
| *** Guest40523 has quit IRC | 00:41 | |
| openstackgerrit | Jesse Keating proposed openstack-infra/zuul feature/zuulv3: Simplify github status descriptions https://review.openstack.org/489767 | 00:42 |
| *** thorst_afk has quit IRC | 00:43 | |
| *** thorst_afk has joined #openstack-infra | 00:43 | |
| *** srwilkers_ has joined #openstack-infra | 00:47 | |
| *** https_GK1wmSU has joined #openstack-infra | 00:47 | |
| *** thorst_afk has quit IRC | 00:47 | |
| *** https_GK1wmSU has left #openstack-infra | 00:48 | |
| *** ramishra has joined #openstack-infra | 00:51 | |
| *** cuongnv has joined #openstack-infra | 00:51 | |
| *** lrossetti_ has quit IRC | 00:55 | |
| *** ijw has quit IRC | 00:56 | |
| *** ijw has joined #openstack-infra | 00:57 | |
| *** jamesdenton has quit IRC | 00:57 | |
| *** zeih has joined #openstack-infra | 00:57 | |
| *** jamesdenton has joined #openstack-infra | 00:58 | |
| *** jkilpatr has joined #openstack-infra | 01:00 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Add Fedora 26 builds https://review.openstack.org/489819 | 01:00 |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Initial Fedora 26 testing https://review.openstack.org/489820 | 01:00 |
| *** gyee has quit IRC | 01:00 | |
| *** hongbin has joined #openstack-infra | 01:01 | |
| *** zeih has quit IRC | 01:02 | |
| *** EricGonczer_ has joined #openstack-infra | 01:03 | |
| *** samP has quit IRC | 01:04 | |
| *** bobh has joined #openstack-infra | 01:05 | |
| *** yushb has joined #openstack-infra | 01:10 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/nodepool master: Support fedora-26 for nodepool dsvm job https://review.openstack.org/482942 | 01:13 |
| *** EricGonczer_ has quit IRC | 01:14 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Make github ssl verification configurable https://review.openstack.org/489573 | 01:14 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add max-nodes-per-job tenant setting https://review.openstack.org/489481 | 01:14 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Use zuul attributes in nodeset section https://review.openstack.org/489506 | 01:14 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add a short name to the project in the inventory https://review.openstack.org/488877 | 01:15 |
| *** Apoorva has joined #openstack-infra | 01:16 | |
| *** jkilpatr has quit IRC | 01:17 | |
| *** yamahata has quit IRC | 01:20 | |
| *** Apoorva_ has quit IRC | 01:20 | |
| jeblair | zuul/nodepool don't look quite right | 01:20 |
| *** Apoorva has quit IRC | 01:21 | |
| *** annegentle has joined #openstack-infra | 01:22 | |
| fungi | jeblair: should we merge 489776 and take nodepool.o.o back out of the emergency disable list, or will that confuse troubleshooting further? | 01:22 |
| *** https_GK1wmSU has joined #openstack-infra | 01:22 | |
| jeblair | fungi: i'm not sure we can merge anything at this point | 01:23 |
| jeblair | zuul and nodepool look like they are busy idling | 01:23 |
| fungi | erg | 01:23 |
| *** liujiong has joined #openstack-infra | 01:23 | |
| *** https_GK1wmSU has left #openstack-infra | 01:24 | |
| *** EricGonczer_ has joined #openstack-infra | 01:24 | |
| jeblair | oh, 4689 merge jobs pending | 01:26 |
| jeblair | zuul *is* processing merge jobs | 01:26 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Use project object in _uploadPack in gerrit driver https://review.openstack.org/488560 | 01:27 |
| fungi | so just... too busy to start any non-merge jobs for a bit | 01:28 |
| jeblair | i think something may be awry with the ironic-tempest-plugin stuff | 01:28 |
| *** EricGonczer_ has quit IRC | 01:28 | |
| jeblair | looking at zm01, it seems to be merging a *very* long stack of changes | 01:28 |
| fungi | yeah, i get the impression that instead of doing a git filter-branch and importing that at repo creation, they decided to push in all the commit history one change at a time or something | 01:29 |
| cloudnull | fungi: it's dead jim | 01:29 |
| cloudnull | the node refuses to come back . | 01:29 |
| fungi | cloudnull: thanks for trying anyway! i'll work on a replacement, though maybe not until tomrrow | 01:30 |
| cloudnull | and i don't think dcops will prioritize getting the drives swapped | 01:30 |
| cloudnull | so i suspect that this instance is a loss. | 01:30 |
| cloudnull | sorry. | 01:30 |
| *** mriedem has quit IRC | 01:30 | |
| fungi | cloudnull: is what it is, not hard to replace on our end but thanks for looking! | 01:30 |
| cloudnull | np. | 01:31 |
| cloudnull | sorry for the hassel | 01:31 |
| fungi | no apology needed, really | 01:31 |
| jeblair | 4605 merge jobs | 01:31 |
| jeblair | so, erm, i reckon it'll work its way out eventually | 01:31 |
| fungi | jlvillal: if you're still around, what's the story with the really long chain of dependent changes for the tempest plugin? | 01:31 |
| *** ijw has quit IRC | 01:31 | |
| mtreinish | fungi: my guess is history preservation | 01:32 |
| *** ijw has joined #openstack-infra | 01:32 | |
| mtreinish | fungi: if you run the scripts to do that history migration without after the repo is already in gerrit, that's a lot of patches | 01:32 |
| fungi | that was my guess as well, though we have some less impactful solutions to that | 01:33 |
| jeblair | fungi: i'm done looking at things, feel free to change nodepool at will; i have to run now | 01:33 |
| fungi | have a good evening jeblair, and thanks! | 01:34 |
| *** tuanluong has joined #openstack-infra | 01:38 | |
| *** gouthamr has joined #openstack-infra | 01:42 | |
| *** Goneri has quit IRC | 01:43 | |
| *** thorst_afk has joined #openstack-infra | 01:44 | |
| mnaser | old patch which would make things prettier if we can get a +2 :) https://review.openstack.org/#/c/457823/ | 01:46 |
| *** pahuang has quit IRC | 01:48 | |
| *** dingyichen has quit IRC | 01:50 | |
| *** aeng has quit IRC | 01:50 | |
| *** aeng_ has joined #openstack-infra | 01:50 | |
| *** dingyichen has joined #openstack-infra | 01:50 | |
| *** abelur_ has joined #openstack-infra | 01:51 | |
| *** LindaWang has joined #openstack-infra | 01:51 | |
| *** dchen has joined #openstack-infra | 01:52 | |
| *** pahuang has joined #openstack-infra | 01:52 | |
| *** pahuang has quit IRC | 01:52 | |
| fungi | yeah, would be nice to get another infra-root to review it | 01:52 |
| fungi | i'm not going to single-core approve is as i'm sponsored by the same organization as the author | 01:53 |
| *** markvoelker has joined #openstack-infra | 01:54 | |
| *** dingyichen has quit IRC | 01:55 | |
| mnaser | fungi np :) i left a screenshot with the change applied to hopefully helpout the review | 01:56 |
| fungi | i saw! thanks so much for testing that | 01:57 |
| *** esberglu has quit IRC | 01:59 | |
| *** slagle has quit IRC | 01:59 | |
| *** pahuang has joined #openstack-infra | 02:00 | |
| *** thorst_afk has quit IRC | 02:00 | |
| *** baoli has quit IRC | 02:02 | |
| *** baoli has joined #openstack-infra | 02:02 | |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/project-config master: networking-midonet: Make v2 grenade job non-voting https://review.openstack.org/489827 | 02:05 |
| *** slaweq has joined #openstack-infra | 02:07 | |
| *** slaweq has quit IRC | 02:11 | |
| *** slagle has joined #openstack-infra | 02:12 | |
| *** gildub has joined #openstack-infra | 02:15 | |
| jlvillal | fungi, As mtreinish said, history preservation. | 02:15 |
| jlvillal | fungi, Luckily it is only doing pep8 checks. | 02:16 |
| *** dingyichen has joined #openstack-infra | 02:22 | |
| *** bobh has quit IRC | 02:23 | |
| *** markvoelker has quit IRC | 02:23 | |
| *** yamamoto has quit IRC | 02:24 | |
| *** ijw has quit IRC | 02:25 | |
| *** slaweq has joined #openstack-infra | 02:25 | |
| clarkb | jlvillal: was importing history when the project was created considered? (our project creation process does that for free) | 02:27 |
| *** yamamoto has joined #openstack-infra | 02:30 | |
| *** slaweq has quit IRC | 02:30 | |
| *** yamamoto has quit IRC | 02:35 | |
| *** yamamoto has joined #openstack-infra | 02:36 | |
| *** esberglu has joined #openstack-infra | 02:38 | |
| *** dave-mcc_ has joined #openstack-infra | 02:40 | |
| *** yamamoto has quit IRC | 02:41 | |
| *** esberglu has quit IRC | 02:42 | |
| *** dave-mccowan has quit IRC | 02:42 | |
| *** baoli has quit IRC | 02:43 | |
| *** panda has quit IRC | 02:43 | |
| *** aeng_ is now known as aeng | 02:44 | |
| *** yamamoto has joined #openstack-infra | 02:45 | |
| *** panda has joined #openstack-infra | 02:48 | |
| *** ramishra has quit IRC | 02:53 | |
| *** nicolasbock has joined #openstack-infra | 02:57 | |
| *** zeih has joined #openstack-infra | 02:58 | |
| *** markvoelker has joined #openstack-infra | 02:59 | |
| openstackgerrit | Merged openstack-infra/subunit2sql master: Add a unit test with a simple sql injection ala xkcd https://review.openstack.org/487188 | 02:59 |
| jlvillal | clarkb, No, but we asked afterwards and people said "too late"... | 03:02 |
| *** zeih has quit IRC | 03:02 | |
| *** bobh has joined #openstack-infra | 03:04 | |
| *** rkukura has quit IRC | 03:12 | |
| *** dave-mccowan has joined #openstack-infra | 03:12 | |
| *** dave-mcc_ has quit IRC | 03:14 | |
| *** bobh has quit IRC | 03:18 | |
| *** gtrxcb has joined #openstack-infra | 03:21 | |
| *** esikachev has joined #openstack-infra | 03:23 | |
| *** thorst_afk has joined #openstack-infra | 03:23 | |
| *** esikachev has quit IRC | 03:27 | |
| *** thorst_afk has quit IRC | 03:27 | |
| *** aeng has quit IRC | 03:29 | |
| *** hongbin has quit IRC | 03:30 | |
| *** dchen has quit IRC | 03:31 | |
| *** dchen has joined #openstack-infra | 03:32 | |
| *** gildub has quit IRC | 03:32 | |
| *** yamamoto has quit IRC | 03:33 | |
| *** dave-mccowan has quit IRC | 03:33 | |
| *** rodrigopaiva has joined #openstack-infra | 03:37 | |
| *** rodrigopaiva has quit IRC | 03:41 | |
| *** rodrigopaiva has joined #openstack-infra | 03:46 | |
| *** stakeda has joined #openstack-infra | 03:47 | |
| *** links has joined #openstack-infra | 03:48 | |
| *** rodrigopaiva has quit IRC | 03:48 | |
| *** yamamoto has joined #openstack-infra | 03:49 | |
| pabelanger | jeblair: fungi: clarkb: mordred: So, apparently we generate .md5 and .sha1 files for tarballs and wheels, but don't do anything with them. EG: upload them to tarballs.o.o. Do we actually want to start uploading them? | 03:59 |
| pabelanger | working on ansible playbook for tox-tarball job | 03:59 |
| *** sree has joined #openstack-infra | 04:00 | |
| *** slaweq has joined #openstack-infra | 04:03 | |
| *** yamamoto has quit IRC | 04:05 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Initial Fedora 26 testing https://review.openstack.org/489820 | 04:05 |
| *** gouthamr has quit IRC | 04:06 | |
| *** gouthamr has joined #openstack-infra | 04:06 | |
| clarkb | pabelanger: I think we record them in the logs but thsts it | 04:07 |
| pabelanger | clarkb: right, that is what I see too | 04:07 |
| *** slaweq has quit IRC | 04:07 | |
| *** yamamoto has joined #openstack-infra | 04:10 | |
| *** gongysh has joined #openstack-infra | 04:13 | |
| *** winggundamth has joined #openstack-infra | 04:16 | |
| *** yushb has quit IRC | 04:18 | |
| *** gildub has joined #openstack-infra | 04:22 | |
| *** adisky__ has joined #openstack-infra | 04:24 | |
| *** sree has quit IRC | 04:28 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Support fedora-26 for nodepool dsvm job https://review.openstack.org/482942 | 04:30 |
| *** ykarel has joined #openstack-infra | 04:30 | |
| *** gildub has quit IRC | 04:31 | |
| *** kornicameister has quit IRC | 04:31 | |
| *** yushb has joined #openstack-infra | 04:32 | |
| *** kornicameister has joined #openstack-infra | 04:32 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/project-config master: Disable swift for Vitrage https://review.openstack.org/489115 | 04:40 |
| *** ijw has joined #openstack-infra | 04:40 | |
| *** gildub has joined #openstack-infra | 04:42 | |
| *** esikachev has joined #openstack-infra | 04:44 | |
| *** ijw has quit IRC | 04:45 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add pre-release and release automation for ARA https://review.openstack.org/489817 | 04:46 |
| openstackgerrit | Merged openstack-infra/project-config master: tripleo: move container upgrade jobs to check https://review.openstack.org/489702 | 04:49 |
| *** yamamoto has quit IRC | 04:50 | |
| *** sree has joined #openstack-infra | 04:51 | |
| *** pahuang has quit IRC | 04:51 | |
| *** ijw has joined #openstack-infra | 04:52 | |
| openstackgerrit | Merged openstack-infra/project-config master: [Zun] Add py35 dsvm non-voting job https://review.openstack.org/489424 | 04:55 |
| *** aeng has joined #openstack-infra | 04:55 | |
| *** dchen has quit IRC | 04:56 | |
| *** dchen has joined #openstack-infra | 04:56 | |
| *** esikachev has quit IRC | 04:59 | |
| *** zeih has joined #openstack-infra | 04:59 | |
| openstackgerrit | Merged openstack-infra/project-config master: Disable swift for Vitrage https://review.openstack.org/489115 | 04:59 |
| *** ijw has quit IRC | 05:00 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/puppet-jenkins master: Don't install ccache for RPM distros https://review.openstack.org/439294 | 05:03 |
| *** zeih has quit IRC | 05:03 | |
| *** yamamoto has joined #openstack-infra | 05:07 | |
| *** ramishra has joined #openstack-infra | 05:12 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-manual master: Add internal link for job naming https://review.openstack.org/489854 | 05:13 |
| *** jtomasek has joined #openstack-infra | 05:16 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Don't ignore inexistent jobs in config https://review.openstack.org/488758 | 05:18 |
| *** psachin has joined #openstack-infra | 05:20 | |
| *** cuongnv has quit IRC | 05:23 | |
| *** thorst_afk has joined #openstack-infra | 05:23 | |
| *** slaweq has joined #openstack-infra | 05:25 | |
| *** ramishra has quit IRC | 05:26 | |
| *** ramishra has joined #openstack-infra | 05:28 | |
| *** thorst_afk has quit IRC | 05:28 | |
| *** cuongnv has joined #openstack-infra | 05:29 | |
| *** slaweq has quit IRC | 05:30 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Simplify github status descriptions https://review.openstack.org/489767 | 05:31 |
| *** shu-mutou is now known as shu-mutou-AWAY | 05:32 | |
| *** yushb has quit IRC | 05:39 | |
| *** esikachev has joined #openstack-infra | 05:42 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Simplify run tox task https://review.openstack.org/487551 | 05:44 |
| *** pgadiya has joined #openstack-infra | 05:48 | |
| *** lathiat_ has quit IRC | 05:52 | |
| *** lathiat has joined #openstack-infra | 05:52 | |
| *** gouthamr has quit IRC | 05:53 | |
| *** cshastri has joined #openstack-infra | 05:59 | |
| *** yushb has joined #openstack-infra | 06:00 | |
| *** e0ne has joined #openstack-infra | 06:01 | |
| *** e0ne has quit IRC | 06:03 | |
| *** slaweq has joined #openstack-infra | 06:15 | |
| *** rlandy has joined #openstack-infra | 06:18 | |
| *** mpaolino has joined #openstack-infra | 06:19 | |
| *** jaosorior has quit IRC | 06:25 | |
| *** gildub has quit IRC | 06:27 | |
| *** rwsu has quit IRC | 06:30 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Rename Node.hold_reason to 'comment' https://review.openstack.org/489717 | 06:33 |
| *** ccamacho has left #openstack-infra | 06:33 | |
| *** jaosorior has joined #openstack-infra | 06:34 | |
| *** pcaruana has joined #openstack-infra | 06:34 | |
| *** markus_z has joined #openstack-infra | 06:36 | |
| *** jamesmcarthur has joined #openstack-infra | 06:37 | |
| *** jamesmcarthur has quit IRC | 06:41 | |
| *** panda has quit IRC | 06:43 | |
| *** rcernin has joined #openstack-infra | 06:46 | |
| *** panda has joined #openstack-infra | 06:47 | |
| *** esberglu has joined #openstack-infra | 06:48 | |
| *** sflanigan has quit IRC | 06:48 | |
| *** kjackal_ has quit IRC | 06:51 | |
| *** markvoelker has quit IRC | 06:51 | |
| *** esberglu has quit IRC | 06:52 | |
| *** andreas_s has joined #openstack-infra | 06:55 | |
| *** sflanigan has joined #openstack-infra | 06:57 | |
| *** sflanigan has joined #openstack-infra | 06:57 | |
| *** namnh has joined #openstack-infra | 06:58 | |
| *** prateek has joined #openstack-infra | 06:59 | |
| *** zeih has joined #openstack-infra | 07:00 | |
| *** zeih has quit IRC | 07:05 | |
| *** slaweq_ has joined #openstack-infra | 07:05 | |
| *** aarefiev_afk is now known as aarefiev | 07:07 | |
| *** sflanigan has quit IRC | 07:07 | |
| *** apetrich has quit IRC | 07:08 | |
| *** slaweq_ has quit IRC | 07:10 | |
| *** zeih has joined #openstack-infra | 07:12 | |
| *** apetrich has joined #openstack-infra | 07:13 | |
| *** https_GK1wmSU has joined #openstack-infra | 07:14 | |
| *** kjackal_ has joined #openstack-infra | 07:15 | |
| *** zeih has quit IRC | 07:16 | |
| *** https_GK1wmSU has left #openstack-infra | 07:16 | |
| *** aviau has quit IRC | 07:19 | |
| *** aviau has joined #openstack-infra | 07:19 | |
| *** sflanigan has joined #openstack-infra | 07:21 | |
| *** sflanigan has joined #openstack-infra | 07:21 | |
| *** thorst_afk has joined #openstack-infra | 07:24 | |
| *** sree has quit IRC | 07:28 | |
| *** sree has joined #openstack-infra | 07:28 | |
| *** sree has quit IRC | 07:29 | |
| *** thorst_afk has quit IRC | 07:29 | |
| *** esikachev has quit IRC | 07:31 | |
| *** sree has joined #openstack-infra | 07:31 | |
| openstackgerrit | Gergely Csatari proposed openstack-infra/project-config master: Removal of installguide-cookiecutter https://review.openstack.org/489898 | 07:32 |
| *** annegentle has quit IRC | 07:34 | |
| *** dizquierdo has joined #openstack-infra | 07:34 | |
| *** hashar has joined #openstack-infra | 07:35 | |
| *** electrofelix has joined #openstack-infra | 07:35 | |
| *** Hal has joined #openstack-infra | 07:36 | |
| *** Hal is now known as Guest83693 | 07:36 | |
| openstackgerrit | Merged openstack-infra/project-config master: Enable Django 1.10 test for Sahara UI only from Pike+ https://review.openstack.org/489674 | 07:39 |
| *** Guest44789 has quit IRC | 07:39 | |
| openstackgerrit | Merged openstack-infra/project-config master: networking-midonet: Make v2 grenade job non-voting https://review.openstack.org/489827 | 07:42 |
| *** gtrxcb has quit IRC | 07:44 | |
| *** aeng has quit IRC | 07:45 | |
| *** ralonsoh has joined #openstack-infra | 07:46 | |
| *** rlandy is now known as rlandy|ci_mtgs | 07:46 | |
| *** gcb_ has quit IRC | 07:48 | |
| *** tesseract has joined #openstack-infra | 07:48 | |
| *** pblaho has joined #openstack-infra | 07:49 | |
| *** ccamacho has joined #openstack-infra | 07:52 | |
| *** alexchadin has joined #openstack-infra | 07:53 | |
| *** egonzalez has joined #openstack-infra | 07:53 | |
| *** cuongnv has quit IRC | 07:58 | |
| *** kjackal__ has joined #openstack-infra | 08:03 | |
| *** kjackal_ has quit IRC | 08:04 | |
| *** kjackal___ has joined #openstack-infra | 08:04 | |
| *** kjackal__ has quit IRC | 08:08 | |
| *** esikachev has joined #openstack-infra | 08:09 | |
| openstackgerrit | Merged openstack-infra/project-config master: Removal of installguide-cookiecutter https://review.openstack.org/489898 | 08:09 |
| *** apetrich has quit IRC | 08:12 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add Fedora 26 builds https://review.openstack.org/489819 | 08:12 |
| *** ralonsoh has quit IRC | 08:12 | |
| *** ralonsoh has joined #openstack-infra | 08:13 | |
| openstackgerrit | Merged openstack-infra/project-config master: Add python and client libray for Rack Scale Design https://review.openstack.org/487601 | 08:14 |
| *** https_GK1wmSU has joined #openstack-infra | 08:14 | |
| *** derekh has joined #openstack-infra | 08:15 | |
| *** ralonsoh has quit IRC | 08:15 | |
| *** apetrich has joined #openstack-infra | 08:16 | |
| *** https_GK1wmSU has left #openstack-infra | 08:16 | |
| *** cuongnv has joined #openstack-infra | 08:16 | |
| *** ralonsoh has joined #openstack-infra | 08:18 | |
| *** pgadiya has quit IRC | 08:25 | |
| *** jpena has joined #openstack-infra | 08:26 | |
| *** efoley has joined #openstack-infra | 08:27 | |
| *** s-shiono has quit IRC | 08:27 | |
| *** efoley has quit IRC | 08:28 | |
| *** efoley_ has joined #openstack-infra | 08:28 | |
| rm_work | hey, anyone have any tips on why our devstack logs stopped having color enabled? (octavia) | 08:29 |
| rm_work | is there a common cause for that I can check? | 08:29 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Fix github dependent pipeline with merge https://review.openstack.org/489925 | 08:30 |
| *** sree has quit IRC | 08:30 | |
| *** dizquierdo has quit IRC | 08:32 | |
| *** lucas-afk is now known as lucasagomes | 08:32 | |
| *** openstackgerrit has quit IRC | 08:33 | |
| yuval | AJaeger: again :\ | 08:36 |
| yuval | AJaeger: yolanda: Infra cores: same as yesterday and two days ago, gate jobs remain queued | 08:37 |
| yolanda | really? :( | 08:37 |
| *** yushb has quit IRC | 08:37 | |
| yuval | seems so | 08:37 |
| yolanda | ok looking, if that happened at the same hour again, there is a problem for sure | 08:37 |
| yuval | yolanda: thanks :) | 08:38 |
| yolanda | launcher is active today | 08:38 |
| *** ralonsoh_ has joined #openstack-infra | 08:40 | |
| *** ralonsoh has quit IRC | 08:40 | |
| yuval | maybe it was stuck and 'released' | 08:40 |
| *** pgadiya has joined #openstack-infra | 08:41 | |
| *** wolverineav has joined #openstack-infra | 08:42 | |
| *** wolverineav has quit IRC | 08:42 | |
| *** wolverineav has joined #openstack-infra | 08:43 | |
| *** martinkopec has joined #openstack-infra | 08:43 | |
| *** openstackgerrit has joined #openstack-infra | 08:47 | |
| openstackgerrit | Merged openstack-infra/tripleo-ci master: Let the pre-upgrade hook stop services in pike+ https://review.openstack.org/484012 | 08:47 |
| ianw | yuval / yolanda / AJaeger : i don't think things are stuck at the moment. things are quite busy though because we turned of osic and previously there was a huge merge backlog | 08:48 |
| yolanda | yep, in this case, i'm not seeing errors in nodepool | 08:49 |
| yuval | ok, good to hear | 08:49 |
| ianw | fungi will look at restoring the mirror for osic tomorrow (his time) i believe | 08:50 |
| *** dizquierdo has joined #openstack-infra | 08:52 | |
| *** dchen has quit IRC | 08:55 | |
| *** dingyichen has quit IRC | 08:55 | |
| *** winggundamth has quit IRC | 08:56 | |
| *** slaweq_ has joined #openstack-infra | 08:59 | |
| *** markus_z has quit IRC | 09:00 | |
| *** slaweq_ has quit IRC | 09:05 | |
| *** kjackal_ has joined #openstack-infra | 09:11 | |
| *** kjackal__ has joined #openstack-infra | 09:13 | |
| *** kjackal___ has quit IRC | 09:13 | |
| *** markus_z has joined #openstack-infra | 09:15 | |
| *** lrossetti has joined #openstack-infra | 09:16 | |
| *** kjackal_ has quit IRC | 09:16 | |
| *** dtantsur|afk is now known as dtantsur | 09:17 | |
| *** lrossetti has quit IRC | 09:19 | |
| *** nmagnezi has joined #openstack-infra | 09:19 | |
| *** lrossetti has joined #openstack-infra | 09:20 | |
| *** slaweq_ has joined #openstack-infra | 09:22 | |
| *** priteau has joined #openstack-infra | 09:24 | |
| *** thorst_afk has joined #openstack-infra | 09:25 | |
| *** slaweq_ has quit IRC | 09:27 | |
| *** gcb has joined #openstack-infra | 09:28 | |
| *** esberglu has joined #openstack-infra | 09:30 | |
| *** thorst_afk has quit IRC | 09:30 | |
| openstackgerrit | James Page proposed openstack-infra/project-config master: Add Gnocchi charm and associated interfaces https://review.openstack.org/489946 | 09:31 |
| *** sdague has joined #openstack-infra | 09:32 | |
| *** esberglu has quit IRC | 09:34 | |
| openstackgerrit | James Page proposed openstack-infra/project-config master: Add Gnocchi charm and associated interfaces https://review.openstack.org/489946 | 09:35 |
| *** sree has joined #openstack-infra | 09:39 | |
| AJaeger | yolanda: could you put https://review.openstack.org/#/c/489669/ on your review queue, please? | 09:40 |
| *** aviau has quit IRC | 09:41 | |
| *** aviau has joined #openstack-infra | 09:41 | |
| *** yushb has joined #openstack-infra | 09:41 | |
| yolanda | sure | 09:41 |
| AJaeger | thanks | 09:42 |
| *** esikachev has quit IRC | 09:42 | |
| *** sree has quit IRC | 09:43 | |
| openstackgerrit | Dinesh Bhor proposed openstack-infra/project-config master: Add Python3.5 job to masakari-* projects https://review.openstack.org/489950 | 09:43 |
| *** digv has joined #openstack-infra | 09:44 | |
| digv | hi.. is there a way to trigger a job with specific branch of the project by editing layout.yaml? | 09:44 |
| AJaeger | digv: please explain very specific what you want | 09:45 |
| digv | I want to run patch which comes only on master branch of cinder | 09:46 |
| AJaeger | digv: if you push a change to gerrit, it runs on the branch you use, so if you run on master, the master tests are run. | 09:46 |
| openstackgerrit | James Page proposed openstack-infra/project-config master: Add Gnocchi charm and associated interfaces https://review.openstack.org/489946 | 09:47 |
| *** tosky has joined #openstack-infra | 09:48 | |
| digv | I want to avoid the patch coming on branches other than master, since the ci env setting for other branches are different... | 09:48 |
| digv | so whenever patch based on ocata get proposed on gerrit, I want to avoid it & redirect it to other slave vms in my ci env | 09:49 |
| *** dizquierdo has quit IRC | 09:49 | |
| openstackgerrit | James Page proposed openstack-infra/project-config master: Add Gnocchi charm and associated interfaces https://review.openstack.org/489946 | 09:50 |
| digv | here is conf details from layout.yaml -> http://paste.openstack.org/show/617233/ | 09:51 |
| openstackgerrit | Merged openstack-infra/project-config master: Push translated pages to docs-draft https://review.openstack.org/489669 | 09:51 |
| *** kjackal_ has joined #openstack-infra | 09:52 | |
| *** kjackal__ has quit IRC | 09:52 | |
| *** kjackal__ has joined #openstack-infra | 09:52 | |
| digv | is there a way to distinguish the patch depending on the branch & run then run to specific slave | 09:52 |
| *** e0ne has joined #openstack-infra | 09:53 | |
| AJaeger | digv: you can specify that a specific job only run for changes against some branches. Look at the branch conditions in the layout file | 09:56 |
| *** kjackal_ has quit IRC | 09:56 | |
| AJaeger | digv: Ah, you're asking for third-party CI? Then best ask on the thirdparty CI operator list. | 09:56 |
| AJaeger | channel I mean - channel #openstack-third-party-ci | 09:57 |
| AJaeger | digv: you can define that your jobs should only run if the change is for master | 09:57 |
| digv | AJaeger: How? by only defining "branch: master"? | 10:01 |
| *** cuongnv has quit IRC | 10:02 | |
| *** namnh has quit IRC | 10:06 | |
| *** hrubi has quit IRC | 10:06 | |
| *** yushb has quit IRC | 10:06 | |
| *** hrubi has joined #openstack-infra | 10:06 | |
| AJaeger | digv: together wit hthe job name | 10:07 |
| AJaeger | see the examples in layout.yaml | 10:07 |
| *** tuanluong has quit IRC | 10:07 | |
| AJaeger | sorry, lunch time now... | 10:07 |
| Dinesh_Bhor | AJaeger, yolanda: Hi, Please check this when you get time: https://review.openstack.org/#/c/489950/ | 10:09 |
| *** sambetts|afk is now known as sambetts | 10:09 | |
| *** jaosorior has quit IRC | 10:10 | |
| *** liujiong has quit IRC | 10:16 | |
| *** ramishra has quit IRC | 10:19 | |
| *** ramishra has joined #openstack-infra | 10:21 | |
| *** slaweq_ has joined #openstack-infra | 10:23 | |
| *** esikachev has joined #openstack-infra | 10:26 | |
| digv | AJaeger" that helped, thanks | 10:26 |
| *** slaweq_ has quit IRC | 10:27 | |
| *** jascott1 has quit IRC | 10:36 | |
| *** jascott1 has joined #openstack-infra | 10:37 | |
| *** jascott1 has quit IRC | 10:41 | |
| *** alexchadin has quit IRC | 10:43 | |
| *** zhurong has quit IRC | 10:43 | |
| *** panda has quit IRC | 10:43 | |
| *** panda has joined #openstack-infra | 10:44 | |
| *** rossella_s has quit IRC | 10:49 | |
| *** rossella_s has joined #openstack-infra | 10:49 | |
| *** esikachev has quit IRC | 10:50 | |
| *** markvoelker has joined #openstack-infra | 10:52 | |
| *** esikachev has joined #openstack-infra | 10:54 | |
| *** rossella_s has quit IRC | 10:54 | |
| *** pgadiya has quit IRC | 10:56 | |
| *** markvoelker has quit IRC | 10:57 | |
| *** rossella_s has joined #openstack-infra | 10:58 | |
| *** jaosorior has joined #openstack-infra | 11:02 | |
| *** dizquierdo has joined #openstack-infra | 11:03 | |
| *** pgadiya has joined #openstack-infra | 11:08 | |
| *** kaisers1 has quit IRC | 11:15 | |
| *** gongysh has quit IRC | 11:15 | |
| *** LindaWang has quit IRC | 11:15 | |
| *** digv has quit IRC | 11:16 | |
| *** esberglu has joined #openstack-infra | 11:18 | |
| *** jaypipes has joined #openstack-infra | 11:18 | |
| *** kaisers1 has joined #openstack-infra | 11:21 | |
| *** thorst_afk has joined #openstack-infra | 11:21 | |
| *** jaypipes is now known as jaypipes-jurydut | 11:22 | |
| *** jaypipes-jurydut is now known as jaypipesjuryduty | 11:22 | |
| *** esberglu has quit IRC | 11:23 | |
| *** _ari_|gone is now known as _ari_ | 11:23 | |
| *** jkilpatr has joined #openstack-infra | 11:24 | |
| *** rhallisey has joined #openstack-infra | 11:26 | |
| *** gildub has joined #openstack-infra | 11:27 | |
| *** esikachev has quit IRC | 11:28 | |
| *** ldnunes has joined #openstack-infra | 11:29 | |
| AJaeger | Dinesh_Bhor: could you have others from your team look over it first, please? | 11:31 |
| *** baoli has joined #openstack-infra | 11:32 | |
| *** baoli has quit IRC | 11:32 | |
| *** baoli has joined #openstack-infra | 11:32 | |
| *** baoli has quit IRC | 11:33 | |
| *** jcoufal has joined #openstack-infra | 11:34 | |
| *** nmagnezi has left #openstack-infra | 11:36 | |
| *** lucasagomes is now known as lucas-hungry | 11:37 | |
| *** kjackal_ has joined #openstack-infra | 11:37 | |
| *** kjackal___ has joined #openstack-infra | 11:38 | |
| *** kjackal__ has quit IRC | 11:39 | |
| *** sree has joined #openstack-infra | 11:40 | |
| *** baoli has joined #openstack-infra | 11:40 | |
| electrofelix | for zuulv3 is it intended to allow multiple sources (gerrit, github, etc) into the same pipeline? or will pipelines from different sources always need to be separate? | 11:40 |
| *** nicolasbock_ has joined #openstack-infra | 11:41 | |
| electrofelix | thought it was going to be the first one as that's where the issue around cross-repo-dependencies mentioned in an email before, but can't track down anything definitive | 11:41 |
| *** ramishra has quit IRC | 11:41 | |
| *** nicolasbock__ has joined #openstack-infra | 11:42 | |
| *** kjackal_ has quit IRC | 11:42 | |
| *** kjackal___ has quit IRC | 11:44 | |
| *** nicolasbock has quit IRC | 11:44 | |
| *** LindaWang has joined #openstack-infra | 11:44 | |
| *** sree has quit IRC | 11:45 | |
| *** nicolasbock has joined #openstack-infra | 11:45 | |
| *** baoli has quit IRC | 11:46 | |
| *** nicolasbock_ has quit IRC | 11:46 | |
| *** nicolasbock__ has quit IRC | 11:47 | |
| *** kaisers1 has quit IRC | 11:48 | |
| *** markvoelker has joined #openstack-infra | 11:53 | |
| *** dprince has joined #openstack-infra | 11:54 | |
| *** kaisers1 has joined #openstack-infra | 12:02 | |
| *** baoli has joined #openstack-infra | 12:02 | |
| Dinesh_Bhor | AJaeger: sure, I will request them to give vote | 12:04 |
| *** rlandy|ci_mtgs has quit IRC | 12:04 | |
| *** jpena is now known as jpena|lunch | 12:06 | |
| openstackgerrit | Merged openstack-infra/system-config master: Use new OpenStack logo on Gerrit sites https://review.openstack.org/457823 | 12:06 |
| *** kjackal_ has joined #openstack-infra | 12:07 | |
| *** gildub has quit IRC | 12:07 | |
| *** stakeda has quit IRC | 12:08 | |
| openstackgerrit | Merged openstack-infra/yaml2ical master: Add Adhoc frequency https://review.openstack.org/489659 | 12:09 |
| fungi | pabelanger: the md5 and sha1 we echo in the logs are to some extent an artifact from before we started making detached signatures and uploading those | 12:09 |
| fungi | now that we're signing the files with a private key as a separate step, the checksums aren't as useful | 12:10 |
| *** jcoufal has quit IRC | 12:11 | |
| fungi | rm_work: colorization where? when you run devstack locally, or browsing on the logs.openstack.org site? | 12:12 |
| *** alexchadin has joined #openstack-infra | 12:15 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/diskimage-builder master: Updated from global requirements https://review.openstack.org/487989 | 12:16 |
| *** LindaWang has quit IRC | 12:18 | |
| fungi | electrofelix: you might try asking in #zuul as there may be more people in there who know how cross-project deps are supposed to work with mixing different triggers. i got the impression from the decision to switch to dependency urls that it was expected to be a feature, but am unsure how it's supposed to work since i thought pipelines had exclusive triggers | 12:18 |
| *** markvoelker has quit IRC | 12:22 | |
| *** markvoelker has joined #openstack-infra | 12:22 | |
| *** slaweq_ has joined #openstack-infra | 12:25 | |
| electrofelix | fungi: thanks, I keep forgetting there is now a dedicated channel, need to add it to my autojoins | 12:25 |
| *** ramishra has joined #openstack-infra | 12:26 | |
| *** slaweq_ has quit IRC | 12:30 | |
| *** coreyob has quit IRC | 12:31 | |
| fungi | electrofelix: yeah, most of the zuul v3 discussion in here has been focused on migrating the openstack community ci, and so on maintaining feature parity with v2 for our specific use cases (which up to now, non-gerrit services have not been) | 12:35 |
| *** baoli has quit IRC | 12:36 | |
| *** lucas-hungry is now known as lucasagomes | 12:39 | |
| dhellmann | did anything change in the pipermail setup recently? about every other time I visit a page on lists.openstack.org/pipermail lately the CSS fails to load. Reloading the page usually fixes it. | 12:40 |
| dhellmann | the behavior feels like a loadbalancer that's out of balance | 12:40 |
| dhellmann | I started noticing the issue yesterday. I suppose it could be a browser cache issue on my side, too | 12:41 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Optionally limit github to protected branches https://review.openstack.org/490009 | 12:42 |
| *** baoli has joined #openstack-infra | 12:43 | |
| *** sdague_ has joined #openstack-infra | 12:43 | |
| fungi | hasn't been happening to me (yet anyway) | 12:44 |
| *** gordc has joined #openstack-infra | 12:45 | |
| fungi | we're still getting slammed with attempts to spam some of the -owner addresses there (>100k of them yesterday) but system load remains low | 12:46 |
| dhellmann | I'll restart my browser and see if it stops | 12:49 |
| *** baoli has quit IRC | 12:51 | |
| *** Goneri has joined #openstack-infra | 12:52 | |
| *** gouthamr has joined #openstack-infra | 12:56 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Optionally limit github to protected branches https://review.openstack.org/490009 | 12:57 |
| mnaser | yay the logo update merged | 12:57 |
| mnaser | :D | 12:57 |
| mordred | dhellmann: I read that weirdly (still coffeeing) and thought for a brief second you were going to restart your browser to see if the spam flood stopped | 12:58 |
| dhellmann | mordred : that would be something, wouldn't it? | 12:58 |
| mordred | dhellmann: I mean, sadly, it's not unpossible | 12:59 |
| fungi | i'm going to restart my browser and see if it starts my day over | 13:00 |
| fungi | that could be a handy trick if i can make it work | 13:00 |
| *** camunoz has joined #openstack-infra | 13:00 | |
| *** yamamoto has quit IRC | 13:02 | |
| *** xyang1 has joined #openstack-infra | 13:04 | |
| *** jpena|lunch is now known as jpena | 13:06 | |
| *** jcoufal has joined #openstack-infra | 13:07 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test https://review.openstack.org/475747 | 13:08 |
| *** gordc has quit IRC | 13:09 | |
| *** esberglu has joined #openstack-infra | 13:11 | |
| *** abelur_ has quit IRC | 13:14 | |
| *** bobh has joined #openstack-infra | 13:14 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add --detail option to nodepool list command https://review.openstack.org/489720 | 13:17 |
| *** yamamoto has joined #openstack-infra | 13:18 | |
| pabelanger | fungi: ya. I think for now I'll look to just print the info into logs. | 13:20 |
| mnaser | looks like infracloud is having issues by any chance? | 13:20 |
| pabelanger | what are you seeing? | 13:21 |
| *** kgiusti has joined #openstack-infra | 13:21 | |
| mnaser | http://logs.openstack.org/46/488246/1/gate/gate-puppet-nova-puppet-unit-4.5-centos-7/b1d6752/console.html | 13:21 |
| mnaser | mirrors unable to connect | 13:21 |
| mnaser | i saw this on 3 builds | 13:21 |
| pabelanger | wonder if the mirror is under load | 13:22 |
| pabelanger | will osic gone, it will get more jobs scheduled to it | 13:23 |
| pabelanger | http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=554 | 13:23 |
| pabelanger | looks doen | 13:23 |
| pabelanger | down* | 13:23 |
| pabelanger | unable to SSH to mirror.regionone.infracloud-vanilla.openstack.org too | 13:24 |
| *** LindaWang has joined #openstack-infra | 13:24 | |
| *** adarazs has left #openstack-infra | 13:26 | |
| pabelanger | mnaser: do you want to propose a patch to project config setting max-server for infracloud-vanilla to 0. Until we get the mirror back online | 13:27 |
| mnaser | pabelanger sure np | 13:27 |
| mnaser | 1 sec | 13:27 |
| *** sree has joined #openstack-infra | 13:27 | |
| *** sree has quit IRC | 13:28 | |
| *** prateek has quit IRC | 13:28 | |
| *** sree has joined #openstack-infra | 13:29 | |
| *** pgadiya has quit IRC | 13:29 | |
| openstackgerrit | Mohammed Naser proposed openstack-infra/project-config master: Set infracloud-vanilla servers to zero. https://review.openstack.org/490025 | 13:29 |
| mnaser | pabelanger ^ | 13:29 |
| mnaser | i think i did it right? :p | 13:29 |
| pabelanger | mnaser: yup, thanks. | 13:31 |
| pabelanger | just looking at logs on controller to see if I can see the reason | 13:31 |
| *** felipemonteiro has joined #openstack-infra | 13:31 | |
| *** baoli has joined #openstack-infra | 13:32 | |
| *** jistr is now known as jistr|mtg | 13:32 | |
| *** xinliang has quit IRC | 13:32 | |
| *** felipemonteiro has quit IRC | 13:37 | |
| pabelanger | ya compute015.vanilla.ic.openstack.org looks to be down | 13:37 |
| pabelanger | and mirror lives on that | 13:37 |
| fungi | that's unfortunate | 13:38 |
| pabelanger | just +3 490025 for now | 13:38 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test https://review.openstack.org/475747 | 13:38 |
| pabelanger | will see if I can get into ilo and reboot | 13:38 |
| *** rwsu has joined #openstack-infra | 13:41 | |
| *** mriedem has joined #openstack-infra | 13:41 | |
| fungi | want me to manually apply that on nodepool.o.o for now? i have it in emergency disable since the change to stop using osic never got approved | 13:42 |
| pabelanger | fungi: ah, ya please. Looks like your review is in patch conflict now | 13:43 |
| fungi | i just went ahead and updated it on the server | 13:43 |
| pabelanger | ty | 13:43 |
| fungi | also want me to delete all nodes in that region so we don't have more test failures? | 13:43 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add --detail option to nodepool list command https://review.openstack.org/489720 | 13:43 |
| *** sdague has quit IRC | 13:43 | |
| pabelanger | fungi: might as well, they will fail | 13:44 |
| *** xinliang has joined #openstack-infra | 13:44 | |
| fungi | this way at least zuul will hopefully requeue most of the ones that hadn't finished yet | 13:44 |
| pabelanger | ilo looks down too | 13:45 |
| pabelanger | no route | 13:45 |
| *** alexchadin has quit IRC | 13:45 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add hold job to nodepool list output https://review.openstack.org/489721 | 13:48 |
| pabelanger | I'll rebuild the mirror shortly | 13:48 |
| pabelanger | need to top up on coffee first | 13:48 |
| fungi | yeah, just finished doing the same, starting work on the osic mirror rebuild now | 13:51 |
| openstackgerrit | Merged openstack-infra/project-config master: Set infracloud-vanilla servers to zero. https://review.openstack.org/490025 | 13:51 |
| *** aarefiev is now known as aarefiev_afk | 13:51 | |
| *** jistr|mtg is now known as jistr | 13:52 | |
| mnaser | should we run rechecks (or will you make a small announcement when we're ready to recheck?) | 13:52 |
| pabelanger | mnaser: ya, let me notice now | 13:53 |
| cloudnull | fungi: if you need anything in the osic to help get the mirror online, let me know. | 13:54 |
| cloudnull | I did check in on the old node, its still dead | 13:54 |
| cloudnull | and the cinder volume node has been powered down | 13:54 |
| Odd_Bloke | zxiiro: o/ I have a project which would be made much easier with https://review.openstack.org/#/c/489414/; I was wondering if I could get a high-level indication of whether or not that include syntax is acceptable; if it is, I'm happy to work against my fork of JJB whilst the details of getting it merged are ironed out but I don't want to be depending on something that has no chance of getting | 13:54 |
| Odd_Bloke | merged. :) | 13:54 |
| *** yamamoto has quit IRC | 13:55 | |
| fungi | cloudnull: nope, just need to run a few canned commands/scripts to launch the replacement, create and attach the new volume, and update dns | 13:55 |
| fungi | cloudnull: but thanks! | 13:55 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Enable repo for python-os-testr https://review.openstack.org/490032 | 13:55 |
| pabelanger | #status notice We have disable infracloud-vanilla due to the compute host running mirror.regionone.infracloud-vanilla.o.o being offline. Please recheck your failed jobs to schedule them to another cloud. | 13:55 |
| openstackstatus | pabelanger: sending notice | 13:55 |
| zxiiro | Odd_Bloke: please join #openstack-jjb I'd defer that question to some of the others since I'm not familiar with that part of the code. | 13:55 |
| *** sdague has joined #openstack-infra | 13:56 | |
| -openstackstatus- NOTICE: We have disable infracloud-vanilla due to the compute host running mirror.regionone.infracloud-vanilla.o.o being offline. Please recheck your failed jobs to schedule them to another cloud. | 13:57 | |
| *** zhurong has joined #openstack-infra | 13:57 | |
| Odd_Bloke | zxiiro: Oh, nice, didn't know JJB had its own channel. Have joined and posted there. :) | 13:58 |
| openstackstatus | pabelanger: finished sending notice | 13:59 |
| zxiiro | Odd_Bloke: yep it was created recently. makes it easier for us to communicate | 13:59 |
| fungi | mnaser: looks like gerrit was smart enough to apply the theme change without a restart. hurrah! | 14:03 |
| *** nicolasbock_ has joined #openstack-infra | 14:04 | |
| *** nicolasbock__ has joined #openstack-infra | 14:07 | |
| fungi | cloudnull: as suspected, can't delete the server instance, nor detach/delete the volumes from it, while the instance is still in an error state | 14:07 |
| *** nicolasbock has quit IRC | 14:07 | |
| fungi | cloudnull: i don't suppose there's an easy trick to cleaning that up? | 14:07 |
| fungi | the api accepts the server delete call but just seems to queue it, because the instance sticks around still in an error state | 14:08 |
| mnaser | fungi: yes i notice, looks nice :) | 14:08 |
| mnaser | fungi do you have admin access to that cloud? | 14:08 |
| mnaser | you can do a force detach with cinder | 14:09 |
| fungi | mnaser: nope, which is why i'm pestering the generous admin | 14:09 |
| mnaser | oh okay :p | 14:09 |
| *** andreas_s has quit IRC | 14:09 | |
| fungi | doesn't look like openstack server remove volume --force is a thing, at least for me | 14:10 |
| EmilienM | pabelanger: really awesome for the buildlogs mirror, thanks | 14:10 |
| *** nicolasbock_ has quit IRC | 14:10 | |
| *** hongbin has joined #openstack-infra | 14:10 | |
| pabelanger | EmilienM: thanks, https://review.openstack.org/489673 will help also. | 14:13 |
| fungi | cloudnull: i found a sufficient workaround in the interim... renamed the instance and volumes to broken, broken1 and broken2 so i don't have to worry about confusion in our ansible inventory | 14:14 |
| pabelanger | fungi: ^ mind a +3 on NODEPOOL_BUILDLOGS_CENTOS_PROXY variable? | 14:14 |
| EmilienM | pabelanger: I'm already preparing the patches to use the mirror in puppet & tripleo CI | 14:14 |
| *** slaweq has quit IRC | 14:14 | |
| *** mriedem1 has joined #openstack-infra | 14:14 | |
| pabelanger | EmilienM: ack | 14:15 |
| *** slaweq has joined #openstack-infra | 14:15 | |
| *** esikachev has joined #openstack-infra | 14:15 | |
| pabelanger | 15.184.65.187 | 14:15 |
| pabelanger | new mirror IP address for infracloud-vanilla | 14:16 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test https://review.openstack.org/475747 | 14:16 |
| pabelanger | working on DNS changes now | 14:16 |
| *** mriedem has quit IRC | 14:16 | |
| pabelanger | #status log mirror.regionone.infracloud-vanilla.openstack.org DNS updated to 15.184.65.187 | 14:18 |
| openstackstatus | pabelanger: finished logging | 14:18 |
| *** jcoufal has quit IRC | 14:18 | |
| pabelanger | fungi: mnaser: http://mirror.regionone.infracloud-vanilla.openstack.org/ should hit new server in 60min, current DNS TTL | 14:19 |
| *** slaweq has quit IRC | 14:19 | |
| pabelanger | fungi: mnaser: new nodepool VM should get it out of box | 14:19 |
| openstackgerrit | James Slagle proposed openstack-infra/tripleo-ci master: Default $NODEPOOL_PROVIDER https://review.openstack.org/490037 | 14:20 |
| *** prateek has joined #openstack-infra | 14:20 | |
| *** mriedem1 is now known as mriedem | 14:20 | |
| mnaser | pabelanger so should we revert the patch? | 14:21 |
| mnaser | (max-servers) | 14:21 |
| pabelanger | mnaser: ya | 14:21 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/project-config master: Revert "Set infracloud-vanilla servers to zero." https://review.openstack.org/490039 | 14:21 |
| *** nicolasbock_ has joined #openstack-infra | 14:21 | |
| fungi | pabelanger: any recollection of which network we should pass the launch script for building in osic? we have a GATEWAY_NET and a GATEWAY_NET_V6 there | 14:21 |
| fungi | i suppose i could just try one... i have a 50/50 chance of picking correctly | 14:22 |
| openstackgerrit | Merged openstack-infra/project-config master: Add buildlogs.centos to nodepool variables https://review.openstack.org/489673 | 14:22 |
| *** rloo has joined #openstack-infra | 14:22 | |
| *** eharney has joined #openstack-infra | 14:22 | |
| pabelanger | fungi: I believe GATEWAY_NET_V6 | 14:23 |
| fungi | darn, i picked the more generic of the two | 14:23 |
| *** annegentle has joined #openstack-infra | 14:23 | |
| fungi | restarted with your recommendation now | 14:24 |
| *** nicolasbock__ has quit IRC | 14:25 | |
| pabelanger | going to revert change on nodepool.o.o now for infracloud | 14:27 |
| fungi | the dns change has already propagated for that? | 14:27 |
| *** slaweq has joined #openstack-infra | 14:27 | |
| fungi | i thought you said the ttl was an hour | 14:27 |
| pabelanger | fungi: it has a TTL of 60mins, but new server should be fine right? | 14:27 |
| pabelanger | since they wouldn't have cached the request | 14:28 |
| fungi | only if it's not forwarding dns queries to an external recursor | 14:28 |
| fungi | it's caching at the external recursor you have to worry about there | 14:28 |
| pabelanger | Right, was assuming local unbound server would look it up first, before passing to google / opendns | 14:28 |
| fungi | (opendns, google, et cetera) | 14:28 |
| pabelanger | ya, okay. I'll hold off for the 60mins | 14:28 |
| pabelanger | gate isn't that bad atm | 14:29 |
| fungi | local unbound server is configured as a caching forwarder, which means if it doesn't find the record in its local cache it forwards the request to another recursove resolver | 14:29 |
| *** rwsu has quit IRC | 14:29 | |
| fungi | er, recursive | 14:29 |
| * mnaser thought unbound was doing recursive lookups | 14:29 | |
| mnaser | so i thought thats why it was ok for CI instances | 14:29 |
| mnaser | is there a way to manage gerrit email notifications? i keep a close eye on it all the time so getting all the emails is a bit redundant, i looked thought settings but couldnt find much | 14:30 |
| mnaser | (and while i was there i found out you can create your own menu, pretty cool) | 14:30 |
| *** jcoufal has joined #openstack-infra | 14:30 | |
| *** rbrndt has joined #openstack-infra | 14:32 | |
| *** slaweq has quit IRC | 14:32 | |
| *** bobh has quit IRC | 14:32 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Fix github dependent pipeline with merge https://review.openstack.org/489925 | 14:32 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Remove getProjectBranches from FakeGithubConnection https://review.openstack.org/490043 | 14:32 |
| fungi | mnaser: pabelanger: http://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/elements/nodepool-base/finalise.d/89-unbound sets up query forwarding | 14:34 |
| fungi | with forward-addr directives in the forward-zone block | 14:35 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Update tox-tarball to use tox role https://review.openstack.org/489705 | 14:35 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Log file stats for tox tarball / wheels https://review.openstack.org/490045 | 14:35 |
| mnaser | fungi oh gotcha | 14:36 |
| *** ykarel has quit IRC | 14:36 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Use py35 tox envlist for documentation https://review.openstack.org/489707 | 14:37 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test https://review.openstack.org/475747 | 14:37 |
| openstackgerrit | Merged openstack-infra/project-config master: Revert "Set infracloud-vanilla servers to zero." https://review.openstack.org/490039 | 14:38 |
| *** zhurong has quit IRC | 14:38 | |
| mnaser | is there a way of hooking into job timeouts (or an example of a project that does it)? unfortunately a timed out job has no data helpful to do much with | 14:39 |
| fungi | mnaser: gerrit's e-mail notification features are pretty inflexible. you can "watch" specific projects for certain events (new changes, new patch sets, all comments, submitted changes, abandoned changes) but any changes you own, comment on or are added to as a reviewer by someone else will get notification for all those event types unless you remove yourself as a reviewer on the individually and any | 14:39 |
| fungi | votes as well (and if you're the change owner you're stuck regardless) | 14:39 |
| *** ralonsoh_ has quit IRC | 14:39 | |
| *** rwsu has joined #openstack-infra | 14:39 | |
| fungi | mnaser: i have some semi-complex server-side filters on my personal mta to change which inbox certain gerrit event notifications go into, in an effort to wrangle the madness | 14:39 |
| mnaser | fungi: well, i was going to do that anyways but i figured less work for gerrit rather than me filtering it | 14:40 |
| *** xarses has joined #openstack-infra | 14:40 | |
| *** gordc has joined #openstack-infra | 14:41 | |
| dtantsur | folks, how do I bother with gerrit problems? there are two accounts associated with my email: https://review.openstack.org/accounts/?q=divius.inside@gmail.com. The 2nd one (ID 10239) is what I use, the first one is what the election scripts get.. | 14:41 |
| dtantsur | ending up with an non-sense full name (and potentially other issues) | 14:42 |
| fungi | yeah, i have gerrit set not to watch any projects, and then my exim filter looks like: http://paste.openstack.org/show/617282/ (some of which may be obsolete/redundant these days as i haven't touched it in years now) | 14:42 |
| fungi | mnaser: ^ | 14:42 |
| *** LindaWang1 has joined #openstack-infra | 14:43 | |
| *** panda has quit IRC | 14:43 | |
| mnaser | fair enoguh | 14:43 |
| * mnaser creates filters | 14:43 | |
| fungi | dtantsur: i can try to work out which account to disable here in a bit | 14:43 |
| dtantsur | thanks! | 14:43 |
| *** panda has joined #openstack-infra | 14:44 | |
| *** LindaWang has quit IRC | 14:45 | |
| *** prateek has quit IRC | 14:45 | |
| *** LindaWang1 is now known as LindaWang | 14:45 | |
| *** jamesmcarthur has joined #openstack-infra | 14:46 | |
| fungi | pabelanger: when i tried to launch with --network=GATEWAY_NET_V6 it never managed to connect to do the puppet apply steps... when i used --network=GATEWAY_NET it worked, but the instance has no ipv6 address and no eth1 interface attached... am i supposed to pass the launch script multiple networks? | 14:48 |
| *** bauzas has quit IRC | 14:50 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Log file stats for tox tarball / wheels https://review.openstack.org/490045 | 14:51 |
| clarkb | fungi: assuming osic I believe you need two interfaces one for each network | 14:52 |
| fungi | yeah, just trying to work out how to pass that through launch-node.py | 14:52 |
| fungi | doesn't look like the --network option takes repetition nor any sort of iteration over the value provided | 14:53 |
| *** armax_ has joined #openstack-infra | 14:53 | |
| *** armax has quit IRC | 14:53 | |
| *** armax_ is now known as armax | 14:53 | |
| pabelanger | fungi: clarkb: with --network=GATEWAY_NET_V6 did it just get an IPV6 address? or ipv4 too? | 14:54 |
| fungi | pabelanger: dunno, the script just kept complaining that it couldn't ssh into the server | 14:54 |
| clarkb | it gets both but I dont think you can fip on that network | 14:55 |
| pabelanger | fungi: I'd try again, and look at openstack server show $FQDN | 14:55 |
| *** nunchuck has joined #openstack-infra | 14:55 | |
| fungi | --network=GATEWAY_NET,GATEWAY_NET_V6 gives me "Network ['GATEWAY_NET,GATEWAY_NET_V6'] is not a valid network in openstackci-osic-cloud1:RegionOne" so i'm testing with --network=GATEWAY_NET --network=GATEWAY_NET_V6 now but suspect it's just taking the second one | 14:55 |
| clarkb | so you need the other network too to fip (and other network may be our normal private thing in neutron) | 14:55 |
| fungi | While testing ssh access: [Errno -2] Name or service not known | 14:55 |
| *** Douhet has quit IRC | 14:55 | |
| *** yamamoto has joined #openstack-infra | 14:55 | |
| fungi | is all i'm getting, same as when i only used --network=GATEWAY_NET_V6 | 14:56 |
| pabelanger | --config-drive flag too? | 14:56 |
| fungi | wondering if someone booted the previous instance bespoke instead of with our script | 14:56 |
| pabelanger | so cloud-init is setup | 14:56 |
| fungi | no idea... the readme doesn't say anything about needing --config-drive | 14:56 |
| *** Douhet has joined #openstack-infra | 14:57 | |
| pabelanger | Ya, usually for rax we don't | 14:57 |
| pabelanger | but other clouds I've needed to use it | 14:57 |
| pabelanger | so, I just always pass it now | 14:57 |
| dmsimard | I ran a pre-release tag to test the release jobs for ARA and it looks like it adds an extra ".0" digit to the versioned file on tarballs.o.o ? | 14:57 |
| dmsimard | So the signing and pypi upload jobs failed on http://logs.openstack.org/ec/ecdcd483a752c4227e0bf8393e87a49e5315016d/pre-release/ara-tarball-signing/4eac59e/console.html#_2017-08-02_14_55_19_679754 | 14:58 |
| dmsimard | They are trying to get https://tarballs.openstack.org/ara/ara-0.14.0rc3.tar.gz but the file is https://tarballs.openstack.org/ara/ara-0.14.0.0rc3.tar.gz (extra zero at the end) | 14:58 |
| ttx | fungi: looking up ptgbot status. I found the website, but the bot doesn't seem to be running ? Also if we do an index page, would you rather have it live in puppet-ptgbot or the ptgbot repository ? Probably puppet-ptgbot since it needs to change at each event | 14:58 |
| fungi | dmsimard: you have to make sure your tags follow pbr's notion of semver+pep440 version string normalization | 14:58 |
| fungi | ttx: i have a change proposed for the index page for a few weeks already, just waiting on infra reviewers to have bandwidth to review it | 14:59 |
| clarkb | fungi: I think we largely replaced the script since osic wa first booted | 14:59 |
| ttx | ok, will review it | 14:59 |
| fungi | ttx: i can look into what's going on with the service in a bit, but backlogged on several other problems this morning | 14:59 |
| ttx | np | 14:59 |
| *** felipemonteiro has joined #openstack-infra | 15:00 | |
| * ttx adds puppet-ptgbot to watch lists | 15:00 | |
| fungi | ttx: the change in question should be using topic:ptgbot if memory serves | 15:00 |
| *** yamamoto has quit IRC | 15:00 | |
| *** slaweq has joined #openstack-infra | 15:01 | |
| dmsimard | fungi: ara uses pbr and I have no problem with it, I think. Tag is 0.14.0rc3 | 15:01 |
| *** ykarel has joined #openstack-infra | 15:01 | |
| *** marst has joined #openstack-infra | 15:01 | |
| *** links has quit IRC | 15:01 | |
| ttx | found it @ https://review.openstack.org/#/c/484798/ | 15:01 |
| *** felipemonteiro_ has joined #openstack-infra | 15:04 | |
| *** felipemonteiro has quit IRC | 15:04 | |
| *** felipemonteiro__ has joined #openstack-infra | 15:04 | |
| *** jamesmcarthur has quit IRC | 15:05 | |
| *** jistr is now known as jistr|mtg | 15:06 | |
| *** jamesmcarthur has joined #openstack-infra | 15:06 | |
| *** felipemonteiro__ has quit IRC | 15:06 | |
| *** felipemonteiro__ has joined #openstack-infra | 15:07 | |
| *** felipemonteiro_ has quit IRC | 15:08 | |
| clarkb | fungi: pabelanger I want to say you can't attach floating IPs to the ipv6 external network and you can't direct attach a VM to the external ipv4 network. So that forced us to have two interfaces one for ipv6 with private v4 and a second with private v4 that we could attach fips to | 15:08 |
| clarkb | fungi: pabelanger also we may have added ipv6 interface later as osic ipv6 didn't exist when we first deployed into that region so we may not have booted it with two networks but just one and added a network later | 15:09 |
| fungi | dmsimard: right, 0.14.0.0rc3 is the pep440 normalized release candidate version string. that's what you should use in your tag | 15:09 |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Add query for missing host mapping bug 1708039 https://review.openstack.org/489771 | 15:11 |
| openstack | bug 1708039 in grenade "gate-grenade-dsvm-neutron-multinode-ubuntu-xenial fails with "No host-to-cell mapping found for selected host"" [Undecided,New] https://launchpad.net/bugs/1708039 | 15:11 |
| fungi | dmsimard: 0.14.0 is the release it's working toward, and 0rc3 is the prerelease version. prerelease version components are separate from the 3 semver version components (because the semver spec doesn't incorporate the notion of prereleases) but pep440 normalization needs version components to start with a digit | 15:11 |
| fungi | dmsimard: best way to double-check in the future is before pushing the tag try running `tox -e venv python setup.py sdist` and look at what version number it gives the tarball | 15:12 |
| fungi | if it doesn't match, git tag -d and reapply the correct string | 15:12 |
| *** thorst_afk has quit IRC | 15:14 | |
| *** esikachev has quit IRC | 15:14 | |
| *** dprince has quit IRC | 15:14 | |
| fungi | clarkb: pabelanger: trying with --network=GATEWAY_NET --config-drive now to see where that goes | 15:15 |
| dmsimard | fungi: so I guess what I really want is 0.14rc3 then ? | 15:15 |
| *** e0ne has quit IRC | 15:16 | |
| clarkb | fungi: I think that may fail due to not being able to direct attach to the external ipv4 network. I think we have a router, network,subnet for v4 private networking that you then attach floating IPs to from GATEWAY_NET | 15:16 |
| clarkb | fungi: but give it a go, neutron/nova should error quick if I am right | 15:16 |
| fungi | dmsimard: 0.14rc3 will also normalize to 0.14.0.0rc3 | 15:16 |
| dmsimard | T_T | 15:17 |
| *** esikachev has joined #openstack-infra | 15:17 | |
| fungi | dmsimard: pbr uses normalized versions in the form of <major>.<minor>.<patch>.<pre> where major, minor and patch can only be digits with leading zeroes omitted if there's more than one digit in the component, and pre starts with 0 followed by a, b or rc plus one or more digits (i believe also with leading zeroes removed) | 15:18 |
| * fungi goes to find the pbr docs for this | 15:19 | |
| fungi | dmsimard: https://docs.openstack.org/pbr/latest/user/semver.html is the dissertation | 15:19 |
| dmsimard | Well I'm sure pbr is doing the right thing, it just goes against what I felt was ok :) | 15:19 |
| *** armax has quit IRC | 15:19 | |
| dmsimard | For me, this is a rc of 0.14.0, so 0.14.0.0 feels out of place | 15:20 |
| dmsimard | But what do I know, I'm not a software guy | 15:20 |
| *** cshastri has quit IRC | 15:20 | |
| fungi | dmsimard: well, it's not 0.14.0.0 it's 0.14.0 release candidate 3 | 15:20 |
| *** nicolasbock__ has joined #openstack-infra | 15:21 | |
| fungi | the release candidate 3 part is encoded as a prerelease version component "0rc3" | 15:21 |
| dmsimard | Right, so 0.14.0rc3, not 0.14.0.0rc3 ? | 15:21 |
| dmsimard | Bah | 15:21 |
| fungi | 0rc3 is not a semver-compliant release version component | 15:21 |
| fungi | so it has to be the fourth component (semver only covers the first three components) | 15:22 |
| *** esikachev has quit IRC | 15:22 | |
| *** psachin has quit IRC | 15:22 | |
| fungi | also that documentation predates a minor change to setuptools pep440 version normalization so says 0.14.0.rc3 when in actuality setuptools turns that into 0.14.0.0rc3 | 15:22 |
| *** LindaWang has quit IRC | 15:23 | |
| *** electrofelix has quit IRC | 15:23 | |
| fungi | because setuptools interpretation of pep440 is that version components must start with a digit and so normalizes anything starting with a letter by prepending a 0 | 15:23 |
| dmsimard | I'll try poking around with sdist and figure out if I can find something I'm happy with :D | 15:24 |
| cloudnull | fungi: sorry was in a meeting. | 15:24 |
| dmsimard | Thanks for your help | 15:24 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: Add hold job to nodepool list output https://review.openstack.org/489721 | 15:24 |
| cloudnull | yes the node is in error, I can nuke it whenever | 15:24 |
| *** nicolasbock_ has quit IRC | 15:24 | |
| cloudnull | just let me know when | 15:24 |
| fungi | cloudnull: no worries, it finally seems to have deleted on its own | 15:24 |
| fungi | and one of the two volumes detached so i was able to delete it | 15:25 |
| cloudnull | cool | 15:25 |
| fungi | the other was still showing attached to the nonexistent instance id last time i looked though, so wouldn't allow me to detach or delete | 15:25 |
| clarkb | if you haven't deleted teh node yet you should be able to show it to get the network setup info | 15:26 |
| clarkb | fungi: ^ | 15:26 |
| fungi | yeah, volume list says volume 975d5bd9-37f5-4cfe-965a-a917056de961 is in-use attached to instance 8b5b85b7-aaca-45a7-9e1a-5c35903f3079 but server remove volume says instance 8b5b85b7-aaca-45a7-9e1a-5c35903f3079 doesn't exist | 15:26 |
| fungi | schizophrenic | 15:26 |
| fungi | clarkb: well, it booted and puppeted similar to when i didn't pass --config-drive but it only has an interface on GATEWAY_NET so far | 15:27 |
| *** annegentle has quit IRC | 15:28 | |
| fungi | clarkb: presumably i just need to work out the openstackclient command necessary to attach GATEWAY_NET_V6 on a second interface? | 15:28 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/elastic-recheck master: Add query for troveclient AttributeError bug 1708190 https://review.openstack.org/490058 | 15:28 |
| openstack | bug 1708190 in python-troveclient "AttributeError: 'module' object has no attribute '_logger'" [Undecided,Confirmed] https://launchpad.net/bugs/1708190 | 15:28 |
| *** armax has joined #openstack-infra | 15:28 | |
| clarkb | fungi: ya that is what I am beginning to think we did because osic was up and running ipv4 only for a month or two before we added ipv6. Guessing we just tacked on the interface and didn't boot a new instance | 15:29 |
| fungi | do we have a cheat sheet for the syntax? otherwise i'll poke around in context help for a but | 15:29 |
| fungi | s/but/bit/ | 15:29 |
| clarkb | I don't know that we do | 15:30 |
| clarkb | pabelanger: ^ I think you may have added ipv6 to osic, do you recall? | 15:30 |
| clarkb | we sort of played with it in cityclodu too but that was mostly how to make new network/subnet using subnet pools | 15:30 |
| cloudnull | clarkb: fungi: you could use the IPv6 net and then assign a FIP to the v4 subnet if that makes life easier for folks | 15:30 |
| *** ralonsoh has joined #openstack-infra | 15:31 | |
| *** martinkopec has quit IRC | 15:31 | |
| clarkb | cloudnull: I think fungi was having problems with ssh working initially on that network | 15:32 |
| fungi | clarkb: i think the issue there is our launch-node.py considers whatever you provide via --network as the thing you want attached to eth0 | 15:32 |
| fungi | and then the instance gets booted with only ipv6 and (something) breaks | 15:33 |
| morgan | did we change the openstack logo on gerrit? | 15:33 |
| *** esikachev has joined #openstack-infra | 15:33 | |
| morgan | it looks... funky now | 15:33 |
| fungi | morgan: yes | 15:33 |
| *** armax has quit IRC | 15:33 | |
| morgan | ftr, i think this version isn't as nice. the O is really fat and the color on "openstack text" is an odd grey. (totally personal opinion) | 15:34 |
| morgan | the wide-ness of the O logo thing is feeling weird. it might be fine once i'm used to it | 15:35 |
| morgan | (it's really... square) | 15:35 |
| *** Sukhdev has joined #openstack-infra | 15:35 | |
| fungi | morgan: if there's a different one available at https://www.openstack.org/brand/openstack-logo/logo-download/ we should use instead, that can probably be arranged. otherwise you want to talk to the design team at the foundation | 15:36 |
| morgan | yeah it's so much just a personal view, i just was voicing my thoughts ;) | 15:36 |
| morgan | prrrrooooobably not worth changing | 15:36 |
| *** annegentle has joined #openstack-infra | 15:36 | |
| morgan | it's about the only place i see the logo :P | 15:37 |
| *** dizquierdo has quit IRC | 15:37 | |
| morgan | so, it surprised me to see a different one | 15:37 |
| *** ym200 has joined #openstack-infra | 15:37 | |
| pabelanger | clarkb: fungi: I cannot remember. I _think_ we created eth1.cfg after the server booted for ipv6 | 15:37 |
| clarkb | morgan: I want to say this logo was first used in austin so its been cooking for a while | 15:39 |
| fungi | (the newton summit in austin, not the austin release) | 15:40 |
| morgan | clarkb: hm. maybe? i also havent' been to any summits really | 15:40 |
| morgan | lately | 15:40 |
| *** vhosakot has joined #openstack-infra | 15:41 | |
| fungi | but yeah, that version of the logo has been on the www site for around a year i think, and on wiki.o.o for a while, and the updated docs.o.o theme for at least a few months | 15:41 |
| Sukhdev | fungi - can I bother you with one question regarding building Third party CI for a community project? | 15:42 |
| fungi | morgan: and the change to update it in the gerrit webui was proposed back in april, we just didn't get around to approving it until a few hours ago | 15:42 |
| clarkb | pabelanger: that sounds right | 15:42 |
| morgan | fungi: heheh | 15:42 |
| fungi | Sukhdev: i may not know the answer but doesn't hurt to ask | 15:42 |
| clarkb | fungi: pabelanger so ya we probably want whatever neutron incantation says attach network to existing server. Then configure it quickly | 15:43 |
| *** gyee has joined #openstack-infra | 15:43 | |
| fungi | Sukhdev: there's also a #openstack-third-party-ci irc channel which may be more appropriate depending on your question | 15:43 |
| *** sdague_ has quit IRC | 15:43 | |
| Sukhdev | fungi : thanks - I am considering building a CI system and was told that for community related projects I could use nodepool infra to build it - | 15:43 |
| Sukhdev | fungi : was looking for some docs/pointers | 15:44 |
| mnaser | clarkb https://review.openstack.org/#/c/458200/ -- if you have the branch locally still handy and can do a quick git add -- i can show the edits in screenshot so git.o.o cna look nice and pretty as well :> | 15:44 |
| Sukhdev | fungi : let me ping on other channel as well | 15:44 |
| fungi | Sukhdev: please provide details; it's unclear to me whether you're talking about building a third-party ci system with a nodepool server, or configuring jobs to run in our upstream ci system | 15:44 |
| clarkb | fungi: pabelanger `openstack server add fixed ip` maybe? | 15:45 |
| Sukhdev | fungi : build the CI system for networking-opencontrail project | 15:45 |
| clarkb | mnaser: thats a good question, let me go look | 15:46 |
| *** e0ne has joined #openstack-infra | 15:46 | |
| fungi | Sukhdev: not the detail i was looking for. do you want to build a ci system for the networking-opencontrail project, or configure jobs in our ci system to test changes to the networking-opencontrail project? | 15:46 |
| *** mtanino_ has joined #openstack-infra | 15:46 | |
| Sukhdev | fungi: the former | 15:46 |
| pabelanger | clarkb: don't believe so. IIRC, we rebuilt using GATEWAY_V6, and both ipv4 and ipv6 attached to the server. But we needed to manually create eth1.cfg on disk | 15:47 |
| fungi | Sukhdev: maybe a pointer to where you had the original conversation about "for community related projects I could use nodepool infra to build it" would help me better figure out what you're asking for | 15:47 |
| pabelanger | I don't think I ran any openstack commands for networking | 15:47 |
| *** ralonsoh has quit IRC | 15:47 | |
| Sukhdev | fungi : In order to kick off the tests for the CI system, I need nodes to launch the jobs on - I was told I could use nodepools for that | 15:47 |
| *** ralonsoh has joined #openstack-infra | 15:47 | |
| clarkb | mnaser: I think it got committed in an earlier change | 15:48 |
| clarkb | mnaser: https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/files/openstack.svg | 15:48 |
| fungi | Sukhdev: we do maintain software for a suite of services which will allow you to dynamically boot and delete a pool of server instances in an one or more openstack clouds to which you obtain access | 15:48 |
| Sukhdev | fungi : actually somebody mentioned this during a discussion at Boston summit - hence, I started to look for some pointers and did not find anything | 15:48 |
| clarkb | mnaser: so its using the same svg file that we used on the gerrit update | 15:49 |
| fungi | Sukhdev: are you asking if you can run that software? if so, the answer is yes, we distribute it under a free and open license so anyone who wants to do so can run a copy of it | 15:49 |
| fungi | Sukhdev: and many third-party ci operators are running nodepool to manage their server resources for jobs, as i understand it | 15:49 |
| Sukhdev | fungi : yes - that is exactly what somebody mentioned - where can I get information to read up on it so that I can replicate what these guys are doing | 15:50 |
| fungi | Sukhdev: but the fact that you seem to be asking whether or not you're allowed to run free software we publish seems odd, so i have to assume that's not really what you mean to ask | 15:50 |
| clarkb | mnaser: but looks like I have to rebase to address merge conflicts anywayts | 15:50 |
| mnaser | clarkb oooh gotcha | 15:50 |
| clarkb | mnaser: so let me push up a rebase really quickly | 15:50 |
| mnaser | yeah | 15:50 |
| mnaser | once you do that i'll do the same for the other patch so reviewers dont have to go through that :p | 15:50 |
| fungi | Sukhdev: start reading https://docs.openstack.org/infra/openstackci/ and see if the documentation there answers your questions | 15:51 |
| fungi | Sukhdev: it has a section on nodepool specifically, but that's just one component of the suite of ci infrastructure described in that documentation | 15:52 |
| clarkb | mnaser: oh it doesn't need a rebase it needs to figure out why http://logs.openstack.org/24/457824/3/gate/gate-infra-puppet-apply-3-centos-7/ba3ac41/puppetapplytest29.final.out.FAILED failed | 15:52 |
| clarkb | fungi: ^ that looks related to ssh key cleanup, is that a known problem? | 15:52 |
| Sukhdev | fungi : thanks - this is exactly I need. This will get me going. | 15:52 |
| Sukhdev | fungi : sorry I confused you a bit the way I asked the question - but, now we on the same page - and I think this documentation will answer my questions | 15:53 |
| clarkb | I've rechecked it on the off chance we have a conflict between what the instance booted with and what is in puppet | 15:54 |
| *** bauzas has joined #openstack-infra | 15:55 | |
| *** rkukura has joined #openstack-infra | 15:55 | |
| fungi | Sukhdev: yep, your specific mention of nodepool was what was confusing me i think, since you were looking for documentation/process about setting up a ci system in whole | 15:55 |
| *** Apoorva has joined #openstack-infra | 15:56 | |
| Sukhdev | fungi : thanks a ton | 15:56 |
| fungi | Sukhdev: if you're interested in the details of nodepool itself, we have detailed documentation about its configuration and services at https://docs.openstack.org/infra/nodepool/ but as i said that's just one component of a larger suite of ci software | 15:57 |
| mnaser | clarkb looks like we'll have to do a little patch for the openstack logo | 15:57 |
| mnaser | https://review.openstack.org/gitweb?p=openstack/magnum.git;a=commitdiff;h=ccb6bfa46e902aa394783515a1f00f7b2e610be3 | 15:57 |
| *** ykarel has quit IRC | 15:57 | |
| * mnaser is going to grab something to eat | 15:58 | |
| *** Guest83693 has quit IRC | 15:58 | |
| fungi | clarkb: that's a very odd error... the changes to remove and clean up accounts/keys haven't even been approved yet | 15:58 |
| AJaeger | mnaser, clarkb : what a nice "small" logo. | 15:59 |
| *** links has joined #openstack-infra | 15:59 | |
| clarkb | fungi: you know I think we've seen this and attributrd it to nova or glean fails | 16:00 |
| *** lrossetti has quit IRC | 16:00 | |
| *** lrossetti_ has joined #openstack-infra | 16:00 | |
| *** makowals has quit IRC | 16:00 | |
| clarkb | fungi: where we write invalif key data to the file (notice lack of rsa prefix) | 16:00 |
| *** ihrachys has joined #openstack-infra | 16:00 | |
| *** egonzalez has quit IRC | 16:01 | |
| clarkb | mnaser: I guss that is downside to an svg? | 16:01 |
| *** esikachev has quit IRC | 16:02 | |
| *** sdague has quit IRC | 16:03 | |
| *** gyee has quit IRC | 16:03 | |
| jlvillal | clarkb: fungi: and anyone else who cares. Is it a big deal if I do push 60 or so patches that all will cause one pep8 job each to run? Seems to me about the same load as me uploading 3 ironic patches. | 16:03 |
| jlvillal | But I noticed people raising concerns yesterday about it. So wanted to double check. | 16:03 |
| fungi | jlvillal: the issue is more with the fact that zuul needs to calculate pre-merge stacked references the entire way up the chain | 16:04 |
| jlvillal | fungi: Oh. I'm guessing that is bad? /me doesn't know how zuul works very well | 16:05 |
| fungi | so ends up being n+(n-1)+(n-2)+... merge operations | 16:05 |
| fungi | the fact that it's 60 interdependent patches is the resource contention. if it were 60 unrelated patches it would handle it without breaking a sweat | 16:06 |
| fungi | but it ended up basically deadlocking zuul for a few hours yesterday when you did it | 16:06 |
| jlvillal | fungi: Ah. | 16:06 |
| jlvillal | Ouch :( | 16:06 |
| clarkb | one work around is to just push 5 at a time or so until you get them all in | 16:06 |
| jlvillal | So, if I push 5, wait till they all finish their tests. Then push next 5, etc.. | 16:07 |
| jlvillal | Will that final patch that has 60 merge dependencies cause big issues? | 16:07 |
| fungi | alternatively, if you have a git state you need pushed into the repo and don't actually have any need to test them as they go in, then one of our gerrit sysadmins can batch push them straight into the repo and bypass ci (basically to simulate what would have been done at import if you had specified an initial repo state to import) | 16:07 |
| jlvillal | fungi: Let me talk to dtantsur about that. | 16:08 |
| jlvillal | I think we would be willing to do that. Well I am willing to do that :) | 16:08 |
| *** rcernin has quit IRC | 16:08 | |
| * dtantsur reads | 16:09 | |
| jlvillal | dtantsur: This is in regards to ironic-tempest-plugin | 16:09 |
| *** Rockyg has joined #openstack-infra | 16:09 | |
| fungi | we discourage that because we want people to import initial state and save us the work of doing it manually, but that train has sailed so it may be a viable compromise over monopolizing the ci scheduler queue for hours at a time | 16:09 |
| dtantsur | jlvillal, fungi, the plan sounds good to me | 16:10 |
| fungi | though of course if you can wait a few days to upload and dribble your changes in a few at a time, that's also fine from our perspective | 16:10 |
| dtantsur | I also don't care too much about preserving git history, so I'm fine with one patch imorting everything | 16:10 |
| jlvillal | fungi: dtantsur: Thanks. I will work on getting the patch set ready. | 16:11 |
| dtantsur | jlvillal: please don't forget that we agreed to include ironic-inspector tests in the same repo | 16:11 |
| fungi | well, we're talking about preserving git history by pushing the commits straight into gerrit | 16:11 |
| fungi | bypassing code review and testing | 16:11 |
| dtantsur | jlvillal: this may be another patch chain that we somehow merge with this... | 16:11 |
| jeblair | jlvillal: (to answer the earlier question -- no the single patch with 60 parents at the end isn't bad -- it will only monopolize one of the mergers; we have 8 -- as long as some of them are free, other things can keep moving) | 16:11 |
| dtantsur | fungi: right, yeah, I'm talking about another (less favorable) option | 16:11 |
| fungi | so just get us a url to a git repo somewhere with the desired state into it | 16:11 |
| fungi | if you need us to do that | 16:12 |
| jlvillal | fungi: dtantsur: I will work on it. | 16:12 |
| dtantsur | thanks! | 16:12 |
| *** rkukura has quit IRC | 16:13 | |
| e0ne | hi team! does depends-on flag work for client and libs dependencies? | 16:13 |
| openstackgerrit | Clark Boylan proposed openstack-infra/project-config master: Add accessbot to #openstack-ptg https://review.openstack.org/490069 | 16:13 |
| *** pcaruana has quit IRC | 16:14 | |
| clarkb | fungi: ttx ^ that should bootstrap the access we need with chanserv so that you can restrict administrator commands to ptgbot | 16:14 |
| *** slaweq has quit IRC | 16:14 | |
| pabelanger | e0ne: should work for any project in openstack namespace | 16:14 |
| fungi | e0ne: as long as your jobs are written to install those dependencies from source and uses something like zuul-cloner or devstack-gate to obtain the correct refs for them | 16:14 |
| pabelanger | yes, that too | 16:14 |
| e0ne | pabelanger: good to know. how can I check it? | 16:14 |
| *** slaweq has joined #openstack-infra | 16:14 | |
| fungi | e0ne: generally if your project has a need to test against, say, oslo.config from source you will define a specific job that installs it from source instead of using normal release version dependencies for that | 16:15 |
| e0ne | fungi: we've got nova patch that depends on a cinderclient patch | 16:16 |
| fungi | e0ne: and then that job will use the proposed changes you depend on for oslo.config when testing your changes | 16:16 |
| e0ne | fungi: job's logs http://logs.openstack.org/85/330285/110/check/gate-tempest-dsvm-neutron-full-ubuntu-xenial/fd20fc9/console.html | 16:16 |
| fungi | e0ne: so nova's devstack-gate jobs will test with the cinderclient depends-on automatically, that should work | 16:16 |
| mordred | infra-root: gonna send the zuul v3 email now unless anyone has any last-minute comments or concerns | 16:16 |
| openstackgerrit | Merged openstack-infra/system-config master: Use https where possible for status site things https://review.openstack.org/457824 | 16:17 |
| ildikov | fungi: there's a devstack change too in the depends-on list | 16:17 |
| fungi | mordred: can't remember if i said (because i got ratholed on osic mirror breakage when trying to get the related infra manual change to merge) but message looked fine. i left a couple of small edits i think | 16:17 |
| *** panda is now known as panda|bbl | 16:17 | |
| ildikov | fungi: which seems to be needed at this stage to get the system work again | 16:18 |
| ildikov | fungi: does that work out of the box too? | 16:18 |
| e0ne | nova patch https://review.openstack.org/#/c/330285/ which we're talking about | 16:18 |
| jeblair | mordred: can i tweak your mention of the zuulv3 infra manual page real quick | 16:18 |
| *** armax has joined #openstack-infra | 16:18 | |
| fungi | ildikov: e0ne: multiple depends-on should also work for those devstack-gate jobs | 16:18 |
| clarkb | fungi: e0ne keep in mind on most jobs clients and libs are installed from pypi not from source | 16:18 |
| mordred | jeblair: absolutely! | 16:19 |
| clarkb | fungi: e0ne you need the special src jobs that we run against the clients and libs to test with them installed from source | 16:19 |
| *** slaweq has quit IRC | 16:19 | |
| ildikov | clarkb: the logs look like the devstack change is not applied | 16:19 |
| fungi | clarkb: i thought devstack-gate based jobs installed them from source? | 16:19 |
| fungi | wow that's a lot of depends-on footers | 16:19 |
| clarkb | fungi: no that changed a few years back (it changed in devstack) | 16:19 |
| mordred | fungi: not by default, no - they use released versions | 16:19 |
| e0ne | clarkb, fungi: does it mean that cinderclient will be installed from pypi even we've got depends-on flag? | 16:19 |
| ildikov | fungi: my bad, I will clean it up | 16:20 |
| mordred | yah - you have to opt-in to -src jobs | 16:20 |
| fungi | okay, so it's just services which get depends-on handled in normal devstack-gate jobs i guess | 16:20 |
| clarkb | e0ne: yes unless you are running the opt in src job which installs it from source instead | 16:20 |
| jeblair | mordred: okay added a sentence there, look okay? | 16:20 |
| *** gyee has joined #openstack-infra | 16:20 | |
| mordred | jeblair: yup! | 16:20 |
| jeblair | mordred: i'm all set, thanks! | 16:20 |
| fungi | e0ne: anyway, the recommended order is to get features you need in clients/libraries merged and released before you depend on them in services. it would be _bad_ to end up with a nova release depending on an unreleased feature in cinderclient | 16:21 |
| clarkb | e0ne: all of the libs and clients themselves should run that job so you can put them at the tail of a chain to get it all tested together from source | 16:21 |
| *** jistr|mtg is now known as jistr | 16:22 | |
| fungi | yep, and as long as we're gating separately on jobs which require testing with the released dependencies too, we avoid accidentally releasing services depending on unreleased library features | 16:22 |
| ildikov | I cleaned up the dependency list by removing what's merged already | 16:23 |
| *** mpaolino has quit IRC | 16:23 | |
| mordred | fungi: yah - exactly. one job that does from source with depends-on and one that does from releases can show that the new features work end-to-end and also don't break with old library releases | 16:23 |
| openstackgerrit | Alex Schultz proposed openstack-infra/yaml2ical master: Add adhoc to the README https://review.openstack.org/490072 | 16:24 |
| e0ne | ildikov: looks like we can't test this chain with gates now:( | 16:24 |
| *** jascott1 has joined #openstack-infra | 16:24 | |
| fungi | e0ne: you can make a throwaway cinderclient change which depends-on the nova change and look at its install from source devstack-gate jobs | 16:25 |
| fungi | like clarkb suggested | 16:25 |
| fungi | or it's possible nova has some experimental deps-from-source jobs | 16:25 |
| e0ne | fungi: good idea. thanks for the help | 16:25 |
| ildikov | fungi: thanks! | 16:25 |
| e0ne | ildikov, fungi: to be clear: we have to change dependency order. remove depends-on from nova patch and add depends-on to a cinderclient patch | 16:26 |
| ildikov | e0ne: ok | 16:26 |
| ildikov | e0ne: where to put the devstack dependency? | 16:27 |
| *** markus_z has quit IRC | 16:27 | |
| e0ne | ildikov: i have no idea. it seems that we can leave it on nova's side | 16:27 |
| ildikov | e0ne: ok, I'll remove only yours from the list then | 16:28 |
| fungi | e0ne: i was suggesting adding an extra cinderclient change that does something innocuous like adding a blank line in a python file and setting that to depends-on the last change in your current chain (presumably the nova change) | 16:28 |
| *** lucasagomes is now known as lucas-afk | 16:28 | |
| *** sree has quit IRC | 16:28 | |
| fungi | then you abandon that cinderclient change once you're satisfied with the results of the tests from source it ran | 16:28 |
| e0ne | fungi: thanks a lot! | 16:28 |
| fungi | obviously you still can't (and shouldn't) merge the nova change until there's a cinderclient release with the things you need to rely on | 16:29 |
| fungi | at which point nova's normal jobs should work to test your new stuff | 16:29 |
| ildikov | hmm, so we still need the client dependency in the Nova patch? | 16:30 |
| fungi | presumably | 16:30 |
| fungi | if your nova patches are going to rely on it | 16:30 |
| fungi | ildikov: devstack and devstack-gate are always installed from source, so wherever seems logical to put them in the dependency series | 16:30 |
| ildikov | ok, I will put the cinderclient dependency back then and leave the devstack one there too | 16:31 |
| ildikov | sorry, I got a little lost by the end :) | 16:31 |
| e0ne | I'll create cinderclient patch with nova dependency | 16:31 |
| *** baoli has quit IRC | 16:32 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Add bits to support stable/pike https://review.openstack.org/490077 | 16:32 |
| *** baoli has joined #openstack-infra | 16:33 | |
| *** EricGonczer_ has joined #openstack-infra | 16:34 | |
| fungi | clarkb: pabelanger: sorry, got sidetracked with a dozen other things... did we have a cheat sheet including openstackclient commands for creating a port in a specific neutron network and attaching it to an instance as a new interface? | 16:35 |
| clarkb | fungi: no but I think the command is `openstack server add fixed ip` | 16:36 |
| clarkb | fungi: then we'll have to update /etc/network/interfaces on the host | 16:36 |
| fungi | don't need to create the switchport in neutron separately? | 16:36 |
| clarkb | fungi: I don't think so | 16:36 |
| fungi | cool, i think that gets me enough of a starting point to figure the rest out from context help | 16:36 |
| clarkb | fungi: you just give that command a server and a network and it should attach things if I read the help docs properly | 16:36 |
| clarkb | fungi: you can do it the other way I think as well but I don't see a need for that in this case | 16:37 |
| fungi | sweet! Internal Server Error (HTTP 500) | 16:38 |
| fungi | gotta love vague errors from an api | 16:38 |
| clarkb | fungi: I'm glad you find that entertaining :) | 16:38 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Add --detail option to nodepool list command https://review.openstack.org/489720 | 16:39 |
| fungi | i'm going to guess we need to specify some precreated fixed ip here, so just working out how to get a list of them so i can identify it | 16:39 |
| clarkb | fungi: the command help info said network name and server from which it assigns an ip | 16:40 |
| clarkb | but could be bug in osc | 16:40 |
| *** sdague has joined #openstack-infra | 16:40 | |
| fungi | i did `openstack server add fixed ip $FQDN GATEWAY_NET_V6` but maybe i need to use uuids or something | 16:40 |
| clarkb | fungi: you need to provide the server too | 16:41 |
| fungi | that's $FQDN | 16:42 |
| *** dprince has joined #openstack-infra | 16:42 | |
| fungi | passing the network uuid instead of the name seems to have worked | 16:42 |
| *** sree has joined #openstack-infra | 16:42 | |
| clarkb | oh I read that as a single var for some reason. Interesti it wouldn't take the name | 16:42 |
| *** ccamacho has quit IRC | 16:42 | |
| fungi | dtroyer: is there any consistency expected between which subcommands take resource names and which require uuids? | 16:42 |
| clarkb | osc help says name or id | 16:42 |
| *** ykarel has joined #openstack-infra | 16:42 | |
| fungi | yeah, maybe if i'd retried it would have "just worked" too | 16:43 |
| *** rossella_s has quit IRC | 16:45 | |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Add hold job to nodepool list output https://review.openstack.org/489721 | 16:47 |
| dtroyer | fungi: everything possible that can take a name does (ie, where there is a name to use) | 16:47 |
| dtroyer | any that do not are bugs | 16:48 |
| fungi | dtroyer: cool, so it's more likely that the cloud in question had trouble reversing the network name to a uuid, or simply choked on that call for unrelated reasons and the fact that it worked when i retried with the uuid was mere coincidence | 16:48 |
| fungi | thanks! | 16:48 |
| fungi | "Internal Server Error (HTTP 500)" left a lot of room for interpretation | 16:49 |
| dtroyer | yeah… we need to do a better job to return the body in those cases | 16:49 |
| fungi | also possible we're not using super new openstackclient | 16:50 |
| dtroyer | if it's repeatable, —debug will give you the server's error body text in the traceback | 16:50 |
| *** ykarel has quit IRC | 16:50 | |
| fungi | thanks, i'll give that a shot next time | 16:50 |
| dtroyer | I can't say why it would fail one time then succeed another, except that something may not have been in the right state yet on the server | 16:50 |
| clarkb | ya could be an error on the cloud side | 16:51 |
| openstackgerrit | Attila Darazs proposed openstack-infra/project-config master: Add "check experimental" jobs to mistral https://review.openstack.org/490086 | 16:52 |
| *** LindaWang has joined #openstack-infra | 16:54 | |
| *** LindaWang has quit IRC | 16:55 | |
| *** jascott1 has quit IRC | 16:55 | |
| *** thorst_afk has joined #openstack-infra | 16:55 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Add bits to support stable/pike https://review.openstack.org/490077 | 16:56 |
| *** e0ne has quit IRC | 16:57 | |
| *** e0ne has joined #openstack-infra | 16:58 | |
| *** e0ne has quit IRC | 16:58 | |
| *** esikachev has joined #openstack-infra | 16:59 | |
| *** derekh has quit IRC | 16:59 | |
| *** ralonsoh has quit IRC | 17:00 | |
| clarkb | mnaser: I think the two base changes will be merged shortly then you can poke at the last one (https://review.openstack.org/#/c/458200/2) | 17:00 |
| *** dtantsur is now known as dtantsur|afk | 17:00 | |
| *** ramishra has quit IRC | 17:00 | |
| openstackgerrit | Rico Lin proposed openstack-infra/project-config master: Avoid py35 and docs ocata gate for heat-agents https://review.openstack.org/490092 | 17:00 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: Add bits to deploy TripleO with stable/pike https://review.openstack.org/490093 | 17:02 |
| *** esikachev has quit IRC | 17:03 | |
| *** ym200 has quit IRC | 17:05 | |
| *** sree has quit IRC | 17:06 | |
| *** tesseract has quit IRC | 17:07 | |
| *** hashar has quit IRC | 17:08 | |
| *** harlowja has joined #openstack-infra | 17:13 | |
| *** EricGonczer_ has quit IRC | 17:13 | |
| *** efoley_ has quit IRC | 17:15 | |
| rm_work | fungi: re: colorization, browsing via gerrit log links | 17:15 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Add bits to support stable/pike https://review.openstack.org/490077 | 17:16 |
| rm_work | fungi: the neutron and nova logs are colorized, and ours USED to be | 17:16 |
| rm_work | i am sure there's some setting we have to do or something I just don't know where to start looking | 17:16 |
| clarkb | rm_work: your logs have to be in a known format and whitelisted by os-loganalyze. If the name of the file changed that could break it | 17:19 |
| rm_work | ah | 17:19 |
| clarkb | rm_work: os-loganalyze is where I would check first | 17:19 |
| rm_work | ok | 17:19 |
| rm_work | thanks I'll look | 17:20 |
| *** prateek has joined #openstack-infra | 17:22 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Add bits to support stable/pike https://review.openstack.org/490077 | 17:25 |
| pabelanger | clarkb: fungi: I think we are ready to bring infracloud-vanilla back online. DNS should be updated by now | 17:26 |
| *** rkukura has joined #openstack-infra | 17:26 | |
| clarkb | andreaf: mtreinish sdague thoughts on giving https://review.openstack.org/#/c/483761/ a go? There is a chance it will result in more reliable access to the mirrors. Possibly also quicker too particularly in europe. I don't expect it to be any less reliable. The big gotcha is going to be cache expiration (max time allowed in cache is 24 hours) | 17:26 |
| pabelanger | http://mirror.regionone.infracloud-vanilla.openstack.org/ | 17:26 |
| clarkb | pabelanger: I missed what happened in infracloud, was it moving the mirror? | 17:26 |
| pabelanger | clarkb: we lost compute015 this morning, which the mirror was running on | 17:27 |
| pabelanger | clarkb: ilo access is down too | 17:27 |
| clarkb | fun fun, I agree that that url is accessbile so should be safe to enable that region again | 17:27 |
| *** prateek has quit IRC | 17:28 | |
| pabelanger | clarkb: I'm manually editing nodepool.o.o since it is still in emergency. We have already landed the revert to project-config | 17:29 |
| clarkb | kk | 17:29 |
| pabelanger | #status log infracloud-vanilla back online | 17:29 |
| openstackstatus | pabelanger: finished logging | 17:29 |
| *** sambetts is now known as sambetts|afk | 17:30 | |
| *** rbrndt has quit IRC | 17:32 | |
| *** esikachev has joined #openstack-infra | 17:34 | |
| *** tosky has quit IRC | 17:36 | |
| *** esikachev has quit IRC | 17:38 | |
| *** baoli has quit IRC | 17:39 | |
| *** baoli has joined #openstack-infra | 17:40 | |
| lin_yang | Dear team, After the patch https://review.openstack.org/#/c/487601/ got merged, we got new repo and new core list "rsd-core", could anyone help to add me "lin.a.yang@intel.com" into that list as the first core, then I can add others? Thanks in advance. | 17:42 |
| clarkb | lin_yang: yes I can get that sorted out | 17:43 |
| lin_yang | clarkb: awesome. Thanks a lot :) | 17:44 |
| clarkb | lin_yang: ok all done. If you find you don't have the expected +2 voting ability try a hard refresh of the gerrit page | 17:44 |
| lin_yang | clarkb: got it. Appreciated. | 17:45 |
| *** esikachev has joined #openstack-infra | 17:47 | |
| *** jamesden_ has joined #openstack-infra | 17:47 | |
| *** jamesdenton has quit IRC | 17:47 | |
| *** dizquierdo has joined #openstack-infra | 17:47 | |
| *** _ari_ is now known as alivigni | 17:48 | |
| *** alivigni is now known as _ari_ | 17:49 | |
| mtreinish | clarkb: I'm fine with giving it a try | 17:50 |
| clarkb | mtreinish: thanks I think having someone ack it from qa (whcih you did yay) will make others more willign to give it a go | 17:52 |
| *** krtaylor has quit IRC | 17:53 | |
| *** jamesmcarthur has quit IRC | 17:53 | |
| mtreinish | clarkb: I'll trade you for: https://review.openstack.org/#/q/topic:more-firehose+status:open | 17:54 |
| clarkb | mtreinish: sure | 17:55 |
| clarkb | infra-root mtreinish is willing to give https://review.openstack.org/#/c/483761/ a try, anyone willing to be a second infra reviewer on that? | 17:55 |
| *** slaweq has joined #openstack-infra | 17:56 | |
| *** yamahata has joined #openstack-infra | 17:57 | |
| clarkb | cmurphy: while I'm taking on stacks of changes to review where was your puppet4 change(s)? | 17:58 |
| cmurphy | clarkb: here https://review.openstack.org/#/c/481955/ | 18:00 |
| clarkb | mtreinish: ok I've approved that stack we'll need to restart the logstash workers once puppet puts everythign in place to pick up the mqtt event publishing | 18:03 |
| mtreinish | clarkb: ok sure, I should be around if any issues pop up | 18:04 |
| clarkb | cmurphy: I've approved yours as well but left a comment if you want to follow up on that (its mostly a consistency in code thing not a correctness issue) | 18:09 |
| *** jpena is now known as jpena|off | 18:10 | |
| openstackgerrit | Adam Harwell proposed openstack-infra/os-loganalyze master: Add Octavia logs to filter regex https://review.openstack.org/490121 | 18:11 |
| rm_work | fungi: ^^ I assume it's just that, copied from the commit Manila made to add theirs | 18:11 |
| rm_work | i did test just to verify it matches | 18:11 |
| clarkb | infra-root, if asked for 404 listings from docs.o.o I have been using these two commands on files.o.o. `grep ' 404 ' /var/log/apache2/docs.openstack.org_access.log | sed -n -e 's/.*"GET \(\/.*\) HTTP\/1\.." 404 .*/\1/p' > 404s` and `sort 404s | uniq -c | sort -rn | grep '\(html\|\/$\)' | grep $projectfilter | head -100` They can be stringed together into a single comamdn too | 18:12 |
| rm_work | is there a way to test it on a real gate run? can I do a depends-on and will it use that version? or is this one of the kinda hardcoded things | 18:12 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-openstack_health master: Switch api server to use system python instead of venv https://review.openstack.org/490123 | 18:12 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Replace references to deprecated controllerExtraConfig https://review.openstack.org/480395 | 18:12 |
| cmurphy | thanks clarkb ! | 18:12 |
| mtreinish | jeblair, clarkb, fungi: ^^^ I finally got around to doing that | 18:12 |
| mtreinish | although i probably screwed something up | 18:13 |
| clarkb | rm_work: its deployed to the runnign server and runs as a wsgi app so depends on won't help. There test framework for it should allow you to test it pretty well though | 18:13 |
| clarkb | rm_work: just using unittests | 18:13 |
| fungi | pabelanger: thanks for getting the vanilla mirror back online. i'm nearly done with the volume creation on the osic mirror replacement now and have it in dns, so we're close to being ready to turn that back up too once i test it | 18:13 |
| pabelanger | fungi: np! and \o/ | 18:13 |
| cmurphy | calebb: next step on puppet-4, if you're interested, is getting http://git.openstack.org/cgit/openstack-infra/system-config/tree/install_modules.sh to install modules to the right place | 18:13 |
| *** baoli has quit IRC | 18:13 | |
| rm_work | hmm yeah i did add one bit to a unit test but removed it before i pushed up, since others didn't seem to be adding stuff specific to them | 18:13 |
| fungi | just got sidelined on the tarball signing zuulv3 discussion for a bit, but finishing up the mirror replacement now | 18:14 |
| calebb | cmurphy: o7 Ill take a look in a bit :) | 18:15 |
| openstackgerrit | Merged openstack-infra/system-config master: Use https://docs.openstack.org everywhere https://review.openstack.org/457825 | 18:15 |
| openstackgerrit | Merged openstack-infra/system-config master: Sanitize ansible results in mqtt callback plugin https://review.openstack.org/461214 | 18:16 |
| openstackgerrit | Merged openstack-infra/system-config master: Add firehose documentation for notifications https://review.openstack.org/469802 | 18:16 |
| openstackgerrit | Merged openstack-infra/system-config master: Add a golang firehose example https://review.openstack.org/463619 | 18:16 |
| openstackgerrit | Merged openstack-infra/puppet-log_processor master: Add MQTT support to the gearman worker https://review.openstack.org/449294 | 18:16 |
| mordred | clarkb, mtreinish: d-g proxy patch +a'd | 18:16 |
| openstackgerrit | Merged openstack-infra/elastic-recheck master: Add query for troveclient AttributeError bug 1708190 https://review.openstack.org/490058 | 18:16 |
| openstack | bug 1708190 in OpenStack DBaaS (Trove) "AttributeError: 'module' object has no attribute '_logger'" [Undecided,In progress] https://launchpad.net/bugs/1708190 - Assigned to Matt Riedemann (mriedem) | 18:16 |
| openstackgerrit | Merged openstack-infra/project-config master: stop publishing docs when we tag releases https://review.openstack.org/489231 | 18:17 |
| *** kfox1111_ is now known as kfox1111 | 18:19 | |
| openstackgerrit | Merged openstack-infra/system-config master: Enable mqtt notifications from logstash gearman worker https://review.openstack.org/460730 | 18:19 |
| openstackgerrit | Merged openstack-infra/puppet-logstash master: Remove the mqtt options from logstash::indexer https://review.openstack.org/462279 | 18:20 |
| *** rbrndt has joined #openstack-infra | 18:20 | |
| *** baoli has joined #openstack-infra | 18:20 | |
| *** bobh has joined #openstack-infra | 18:20 | |
| openstackgerrit | Merged openstack-infra/system-config master: Parameterize puppet version in install script https://review.openstack.org/481955 | 18:22 |
| *** rbrndt has quit IRC | 18:24 | |
| *** Rockyg has quit IRC | 18:25 | |
| *** links has quit IRC | 18:31 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Create tox-publisher trusted job https://review.openstack.org/490132 | 18:31 |
| *** esikachev has quit IRC | 18:32 | |
| pabelanger | jeblair: mordred: ^First attempt at tox-publisher trusted playbook | 18:32 |
| fungi | infra-root: mirror.regionone.osic-cloud1.openstack.org is back online, and my lonely held node there in osic is able to install python packages again where it was hanging before. i'm going to abandon 489776 and take nodepool.o.o out of emergency disable again unless there are any objections | 18:32 |
| *** adisky__ has quit IRC | 18:32 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Create tox-publisher trusted job https://review.openstack.org/490132 | 18:33 |
| *** esikachev has joined #openstack-infra | 18:33 | |
| pabelanger | fungi: ++ | 18:34 |
| jeblair | pabelanger: cool; should be able to take a look in ~1hr | 18:34 |
| pabelanger | jeblair: np, off for some eats myself | 18:35 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Optionally limit github to protected branches https://review.openstack.org/490134 | 18:35 |
| fungi | dtantsur|afk: i'm going to start digging into your duplicate gerrit accounts situation now... apologies for the delay | 18:36 |
| *** slaweq has quit IRC | 18:36 | |
| *** slaweq has joined #openstack-infra | 18:37 | |
| *** jaypipesjuryduty is now known as jaypipes | 18:37 | |
| *** esikachev has quit IRC | 18:39 | |
| mnaser | https://review.openstack.org/#/c/458200/2 added a screenshot to make the review easier and finish moving the logos :) | 18:39 |
| *** kjackal_ has quit IRC | 18:40 | |
| *** slaweq has quit IRC | 18:41 | |
| *** kjackal_ has joined #openstack-infra | 18:42 | |
| *** ihrachys has quit IRC | 18:42 | |
| *** ihrachys has joined #openstack-infra | 18:42 | |
| dmsimard | fungi: so I've poked around sdist and I can't make it and PBR bend to my will so I will have to work with that extra .0 I guess. | 18:43 |
| clarkb | mnaser: tyty | 18:43 |
| dmsimard | fungi: Looking at other projects, say, nova which is also in the format "15.0.0" also carries an extra ".0" for pre-releases (i.e, 16.0.0.0b1) | 18:43 |
| dmsimard | My brain still can't make sense out of it but I'll have to comply :) | 18:44 |
| *** krtaylor has joined #openstack-infra | 18:44 | |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Optionally limit github to protected branches https://review.openstack.org/490134 | 18:45 |
| openstackgerrit | Tobias Henkel proposed openstack-infra/zuul feature/zuulv3: Optionally limit github to protected branches https://review.openstack.org/490134 | 18:47 |
| mnaser | clarkb are you going to propose a small patch for the big logo or you want me to do that? | 18:48 |
| *** jascott1 has joined #openstack-infra | 18:49 | |
| *** rbrndt has joined #openstack-infra | 18:51 | |
| fungi | dmsimard: to reiterate, semver covers the first three dot-separated components of the version number, so the 16.0.0 part, and has no provisions for pre-release version components so the b1 can't be in one of those first three components. setuptools interpretation of pep440 normalization adds a leading 0 to any version component which starts with a letter, hence the 0 in the 0b1 component | 18:53 |
| fungi | i agree it's not terribly aesthetically pleasing, but rather the result of mashing two standards (semver and pep440) together | 18:54 |
| dmsimard | :) | 18:54 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: tripleo/oooq/cleanup: remove unused jobs in gate https://review.openstack.org/490148 | 18:55 |
| mtreinish | pabelanger: the asterisk server doesn't record the audio from the call? | 18:56 |
| *** annegentle has quit IRC | 18:56 | |
| AJaeger | EmilienM: could you quickly check https://review.openstack.org/#/c/490086/ , please? | 18:57 |
| EmilienM | sure | 18:57 |
| *** annegentle has joined #openstack-infra | 18:58 | |
| EmilienM | AJaeger: good spot. I commented | 18:58 |
| AJaeger | thanks | 18:59 |
| clarkb | mnaser: I'm not quite sure where to start with that. Do we go back to png so the pixels are fixed? | 18:59 |
| clarkb | mnaser: feel free to push something up | 18:59 |
| AJaeger | clarkb: the multi-node/single-node changes in change https://review.openstack.org/#/c/482048/ look wrong to me - do we really want one job for both? Could you review that, please? | 18:59 |
| mnaser | clarkb np, svg just needs a fixed size somewhere in css, ill do it | 18:59 |
| *** jcoufal has quit IRC | 19:00 | |
| *** apuimedo has quit IRC | 19:00 | |
| *** jamesmcarthur has joined #openstack-infra | 19:00 | |
| *** coolsvap has quit IRC | 19:01 | |
| openstackgerrit | Merged openstack-infra/project-config master: Transit Heat convergence job to use quickstart https://review.openstack.org/483292 | 19:03 |
| mnaser | clarkb i need to edit this file.. but i cant find it in system-config, any ideas? https://review.openstack.org/gitweb-site.css | 19:05 |
| pabelanger | mtreinish: it can, but not enabled. I've thought about enabling it from time to time | 19:05 |
| clarkb | infra-root sdage what do you think about having mod_rewrite rewrite https://docs.openstack.org/404s.html to a simple listing of the top 1k 404s by count against the last day of docs.o.o apache logs? Then we can have cron update that file daily? | 19:05 |
| pabelanger | mtreinish: should be hard to record and publish some place | 19:05 |
| clarkb | infra-root sdague, that way this doesn't need afs write permissiosn and remains super simple | 19:05 |
| clarkb | mnaser: I think that just comes from the normal gitweb install. We can likely override it somehow let me check some docs | 19:06 |
| mtreinish | pabelanger: yeah I was just a bit surprised considering how we like things to be archived that it wasn't turned on | 19:06 |
| *** skelso has joined #openstack-infra | 19:06 | |
| mtreinish | its not a big deal either way, it's not like I have any interest in listening to it | 19:06 |
| mnaser | clarkb thats the "site" css file, so it contains all the openstack theme related stuff | 19:06 |
| mtreinish | I was just curious | 19:06 |
| mnaser | so it was configured somewhere.. but i cant find where | 19:06 |
| clarkb | mnaser: ah ok let me find it then | 19:06 |
| mnaser | not in system-config anyways | 19:06 |
| openstackgerrit | Merged openstack-infra/project-config master: func_py3 experimental check for openstack-ansible-plugins https://review.openstack.org/488516 | 19:07 |
| clarkb | mnaser: openstack-infra/system-config/modules/openstack_project/files/gerrit/GerritSite.css ? | 19:07 |
| pabelanger | mtreinish: I can write up some patches to support it. Good friday hacking project | 19:08 |
| clarkb | mnaser: looks to be sahred with gerrit too so you'll want to check it against gerrit's rendering of the svg too (though you likely know far more about all that than I do) | 19:08 |
| pabelanger | mtreinish: or russellb can do it to bump his stackalytics count | 19:08 |
| pabelanger | :) | 19:08 |
| mnaser | clarkb i *suspect* someone manually cp'd the GerritSite.css content to the gitweb-site.css file | 19:09 |
| clarkb | mnaser: I think gerrit may do it | 19:09 |
| clarkb | since gitweb runs as part of gerrit I think? | 19:09 |
| AJaeger | EmilienM: could you review this as well, please? https://review.openstack.org/#/c/480230 | 19:09 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: tripleo/quickstart: use zuul template for scenarios https://review.openstack.org/490163 | 19:09 |
| EmilienM | AJaeger: sure thing | 19:09 |
| *** rbrndt has quit IRC | 19:09 | |
| clarkb | aptitude says gitweb is installed though so maybe not | 19:10 |
| mnaser | https://review.openstack.org/gitweb-site.css vs https://raw.githubusercontent.com/openstack-infra/system-config/master/modules/openstack_project/files/gerrit/GerritSite.css | 19:10 |
| *** nicolasbock__ has quit IRC | 19:10 | |
| mnaser | fungi mentioned today that the style was automatically loaded in without restarting gerrit | 19:10 |
| mnaser | maybe the restart is what does the copy? | 19:10 |
| EmilienM | AJaeger: ouch, -1 | 19:11 |
| mnaser | i mean, the css will still need changes but i just want to make sure that editing the GerritSite.css is the right way (and there isn't some other file i have to edit) | 19:11 |
| clarkb | mnaser: ya that is a possibility, looking at the config management for it we just ensure the gitweb package is installed but don't configure it | 19:11 |
| mnaser | thats what i saw in puppet-gerrit too | 19:11 |
| clarkb | I think gerrit must be doing that, but am digging more | 19:11 |
| AJaeger | lbragstad, stevemar , could you review this keystone related project-config change, please? https://review.openstack.org/#/c/483576 | 19:12 |
| AJaeger | EmilienM: good that I waited ;) | 19:12 |
| lbragstad | AJaeger: sure | 19:12 |
| EmilienM | AJaeger: please ping me everytime CI changes are done for tripleo :D | 19:12 |
| *** jamesmcarthur has quit IRC | 19:12 | |
| *** EricGonczer_ has joined #openstack-infra | 19:13 | |
| AJaeger | EmilienM: I'll try to not +2 before the infra liaisons +1 - and you're listed... | 19:13 |
| AJaeger | those were older, so I decided to ping... | 19:13 |
| EmilienM | AJaeger: lgtm | 19:13 |
| EmilienM | yea | 19:13 |
| mnaser | clarkb https://review.openstack.org/Documentation/config-gitweb.html#_logo_and_css | 19:14 |
| fungi | clarkb: or we could add a 404s index (in a fixed path or under a new subdomain even) separate from a less noisy 404 page | 19:14 |
| AJaeger | EmilienM: https://review.openstack.org/#/c/481711/1 is duplicate of https://review.openstack.org/#/c/480230/ | 19:14 |
| clarkb | mnaser: ya /usr/share/gitweb/static has stuff in it | 19:15 |
| clarkb | mnaser: but I don't see where we generate that | 19:15 |
| mnaser | is it a symlink or is it a static file with the contents of that? | 19:15 |
| EmilienM | bnemec: please abandon https://review.openstack.org/#/c/481711 | 19:15 |
| *** rbrndt has joined #openstack-infra | 19:15 | |
| mnaser | maybe someone just cp'd it waaaay back and its not managed by puppet | 19:15 |
| sdague | clarkb: that would be fine for me | 19:16 |
| sdague | clarkb: I was assuming a cron publish to text file would be just as safe as well | 19:16 |
| mordred | clarkb: works for me | 19:16 |
| AJaeger | EmilienM: so, I would rather have your team ping you instead of me doing it - but will do for odd and old things occasionally | 19:17 |
| clarkb | sdague: mordred fungi oh ya I meant rewrite to a file on disk that isn't in afs | 19:17 |
| clarkb | sdague: mordred fungi that file would update once a day via cron. I just don't want to rely on afs for server specific content as it would get tricky if we added more than one server | 19:17 |
| clarkb | mnaser: maybe? we have an /etc/gitweb.conf too but I don't see what process runs the cgi | 19:18 |
| mordred | clarkb: yup. | 19:18 |
| mordred | mnaser, clarkb: I believe gerrit handles all gitweb related config itself and we don't directly configure it ourselves | 19:18 |
| fungi | clarkb: seems fine as long as we cleanly postprocess out everything except the url requested from the log, and make sure we do it in such a way as to not risk any sort of shellcode injection in whatever script is doing that | 19:18 |
| openstackgerrit | Hongbin Lu proposed openstack-infra/project-config master: [Zun] Disable Swift for dsvm py35 job https://review.openstack.org/490165 | 19:19 |
| EmilienM | AJaeger: ok | 19:19 |
| EmilienM | AJaeger: works for me | 19:19 |
| *** jamesmcarthur has joined #openstack-infra | 19:19 | |
| *** jtomasek has quit IRC | 19:19 | |
| EmilienM | AJaeger: https://review.openstack.org/#/c/490163/ is ready for review | 19:19 |
| *** panda|bbl is now known as panda | 19:19 | |
| openstackgerrit | Hongbin Lu proposed openstack-infra/project-config master: [Fuxi] Disable Swift for py35 fullstack job https://review.openstack.org/490166 | 19:20 |
| clarkb | mordred: mnaser ya the more I look the more I think gerrit must be injecting its own stuff in there. So yes best guess is restarting gerrit will cause it to update gitweb's configs | 19:20 |
| mnaser | clarkb do you want me to submit a change with the fixes to GerritSite.css and then we can start and hopefully it overrides | 19:21 |
| mnaser | and if it doesnt override it we can figure it out then | 19:21 |
| clarkb | mnaser: ya I think that may be simplest | 19:21 |
| fungi | yay! osic nodes building leave quite a spike in the test nodes graph on the status page ;) | 19:21 |
| clarkb | fungi: well it will run a fixed script from cron so there is no input to sanitize to avoid shellcodei njection | 19:21 |
| clarkb | fungi: and it would run my commands above basically just without a project filter. I'll work on pushuing up a patch with more concrete infos | 19:22 |
| fungi | clarkb: yeah, that command looks safe enough. no actual strings from the file ending up in parameters to anything invoked from the shell | 19:23 |
| fungi | it's all via pipes, so we should be good | 19:23 |
| *** dizquierdo has quit IRC | 19:25 | |
| AJaeger | team, here's a change to adjust our nodepool ssh setup for project-config that nobody likes to +2: https://review.openstack.org/#/c/467919/2 could some expert please review and tell us whether this is wanted or not? | 19:26 |
| AJaeger | pabelanger: do we still need https://review.openstack.org/#/c/461518/ ? | 19:26 |
| pabelanger | AJaeger: no, I think ianw fixed it | 19:27 |
| pabelanger | will abandon | 19:27 |
| *** e0ne has joined #openstack-infra | 19:27 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-openstack_health master: Switch api server to use system python instead of venv https://review.openstack.org/490123 | 19:29 |
| AJaeger | fungi, clarkb: ages ago you commented on https://review.openstack.org/#/c/346619 - I think we don't want it, correct? | 19:30 |
| *** slaweq has joined #openstack-infra | 19:31 | |
| fungi | dtantsur|afk: it looks like your extra 12773 account from 2015-12-16 wasn't used by anything, so i have deactivated it, leaving only your older 10239 account. please let me know if you experience any issues with gerrit, but i expect you won't notice any difference | 19:32 |
| AJaeger | Oops, we maxed out our capacity ;/ I have jobs waiting for an hour for a node to be ready... | 19:32 |
| openstackgerrit | Hongbin Lu proposed openstack-infra/project-config master: [Kuryr] Setup a non-voting py35 job for libnetwork https://review.openstack.org/490169 | 19:33 |
| AJaeger | and logstash queue is at 20k events since a good hour - from 0. That looks strange... | 19:33 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/system-config master: Fix OpenStack header logo in Gitweb https://review.openstack.org/490171 | 19:33 |
| mnaser | clarkb ^ there with screenshots | 19:33 |
| clarkb | AJaeger: it often does that if some jobs inject massive log files | 19:33 |
| AJaeger | interesting | 19:34 |
| pabelanger | large amount of nodes building in nodepool too | 19:34 |
| pabelanger | must be osic coming back online | 19:34 |
| fungi | yeah, i commented on it in channel earlier | 19:34 |
| fungi | http://grafana.openstack.org/dashboard/db/nodepool-osic | 19:34 |
| *** e0ne has quit IRC | 19:34 | |
| fungi | time to ready there spiked to 15 minutes and we got quite a few error nodes, so likely slammed it hard | 19:35 |
| AJaeger | ok, let's give it time to settle ;) | 19:35 |
| *** skelso has quit IRC | 19:37 | |
| *** EricGonczer_ has quit IRC | 19:39 | |
| *** wolverineav has quit IRC | 19:39 | |
| *** skelso has joined #openstack-infra | 19:39 | |
| *** wolverineav has joined #openstack-infra | 19:40 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Add zuul.executor.work_root to inventory https://review.openstack.org/490173 | 19:40 |
| *** EricGonczer_ has joined #openstack-infra | 19:40 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test https://review.openstack.org/475747 | 19:41 |
| *** EricGonczer_ has quit IRC | 19:42 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove gate-tempest-dsvm-full from grafana https://review.openstack.org/490174 | 19:42 |
| *** EricGonczer_ has joined #openstack-infra | 19:42 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Generate list of top 100 404s for docs.o.o https://review.openstack.org/490175 | 19:43 |
| clarkb | fungi: mordred sdague ^ something like that (note script is untested) | 19:44 |
| *** wolverineav has quit IRC | 19:44 | |
| clarkb | fungi: unless grep, sort, uniq, or sed have a vulnerability and can be influenced on their inputs I think we are ok. I don't think there are shell injections concerns | 19:44 |
| clarkb | AJaeger: I'm reviewing the mulitnode change now then mnasers css fix | 19:45 |
| AJaeger | thanks, clarkb | 19:46 |
| clarkb | AJaeger: what is the concern on the multinode change? | 19:46 |
| sdague | clarkb: would it be possible to timestamp the files `date +%Y-%m-%d`-docs-404s.txt ? | 19:46 |
| AJaeger | clarkb: normally we have different jobs for multinode and single node - I wonder whether it's ok to use this template this way for both | 19:46 |
| sdague | I think at least to get started, it would be valuable to keep the old files around and see how they change as we fix things | 19:47 |
| clarkb | sdague: not if we want the apache alias to work and i don't want ot implement rotations. I can add a timestamp prefix to them though | 19:47 |
| fungi | clarkb: i'm trying to work out your requests session params encoding suggestion now. doing a separate change to refactor the duplicate requests logic out of the gerrit query function first and will generalize it enough so it can be used for the new member directory lookups in the later change | 19:47 |
| clarkb | sdague: we could keep the old files around but then I'd have to use proper rewrite | 19:47 |
| sdague | clarkb: or just dirindex a directory | 19:47 |
| clarkb | sdague: and alias to the dir? | 19:48 |
| sdague | sure | 19:48 |
| clarkb | sdague: the problem I'm trying to work around is that the docroot is in afs and intended to be shared by many apaches potentionally. So I can't put a single apache's log data in there | 19:49 |
| clarkb | want to keep that local | 19:49 |
| clarkb | I can use the hacky elasticsearch method of file rotation and use find to delete anyhting more than a month old | 19:49 |
| sdague | clarkb: these are small files, do we really need to worry about delete at this point? | 19:50 |
| sdague | like, the entire runtime of this for the history of the project is smaller than the output of a pep8 job | 19:50 |
| clarkb | sdague: yes, beacuse every time we tell ourselves that we inevitably run into problems later | 19:50 |
| jeblair | clarkb: let's stick it in afs. it's not that hard. | 19:50 |
| sdague | clarkb: ok, the order of magnitude is so small here though | 19:50 |
| clarkb | jeblair: the problem isn't sticking it in afs. The problem is the data isn't shared data intended to be served by many servers | 19:50 |
| jeblair | clarkb: servername as path component | 19:51 |
| sdague | clarkb: also, is 100 lines sufficient if it's more than 1 project? I thought the nova 404 dump was 40+ | 19:51 |
| sdague | at least to start we know there are going to be a lot of 404s because of the move | 19:52 |
| clarkb | sdague: http://paste.openstack.org/show/617316/ thats 100 from right now | 19:52 |
| clarkb | jeblair: or we can just serve off local server and avoid afs because it is unnecessary | 19:53 |
| clarkb | jeblair: this also allows our servers to continue to be anonymous | 19:53 |
| sdague | clarkb: yeh, so the problem with that is that if a project, like nova, gets on top of it and fixes those top 5, we can't see the rest of our 404s to fix | 19:53 |
| clarkb | there are ~3k 404'd paths in total | 19:54 |
| clarkb | so we likely could do the whole set | 19:54 |
| clarkb | 3107 from today | 19:54 |
| jeblair | clarkb: i don't have a rebuttal to that other than we have a pretty good content publishing system and it seems unecessary to invent another one. literally no apache configuration is required if you publish it into afs. | 19:54 |
| *** annegentle has quit IRC | 19:55 | |
| clarkb | jeblair: but it does require server configuration changes that change the servers from anonymous read only to needing authenticated write perms | 19:55 |
| jeblair | clarkb: (as for anonymity, the rewrite system requires you know the actual hostnames of every web serving host, so that seems the same either way) | 19:55 |
| *** sileht has quit IRC | 19:55 | |
| *** annegentle has joined #openstack-infra | 19:56 | |
| jeblair | clarkb: yeah, we can make a keytab for the cronjob (we have like 20 already, so that's pretty well established) | 19:56 |
| jeblair | clarkb: (we could also ip acl the server, but keytabs are better) | 19:56 |
| jeblair | but we don't need to change how apache runs or anything | 19:57 |
| jeblair | i'm happy to pitch in on the afs stuff if you want | 19:57 |
| kmARC | hi channel, the upstream-institute project decided to bring the virtual environment source code over to openstack.org from GitHub and set up Gerrit reviewing workflow, etc. Currently, it's hosted here: https://github.com/kmARC/openstack-training-virtual-environment | 19:57 |
| kmARC | Since it's not really related to the training slide deck sources, we would create a new repository for it, `upstream-institute-virtual-environment`. Can you guys help me with that? | 19:57 |
| clarkb | jeblair: and is that preferable to maintaining ro access? I quite like read only access as being a thing that is possible for functioning servers | 19:58 |
| clarkb | makes them far simpler to deploy and reduces the security concern of having lots of things with lots of access | 19:58 |
| pabelanger | kmARC: sure, we have the process documented at https://docs.openstack.org/infra/manual/creators.html | 19:58 |
| pabelanger | kmARC: be sure to note the section about 'upstream' field | 19:58 |
| jeblair | clarkb: the keytab would grant write access to only one subdirectory, and would only be used by the cronjob. | 19:59 |
| jeblair | clarkb: a server deployed without the keytab for whatever reason would still function as a web head, but its cronjob to build 404 lists would fail | 19:59 |
| kmARC | @pabelanger thanks you, will read it now /sorry for the disruption/ ;-) | 19:59 |
| clarkb | jeblair: which we weigh against 4 linse of apache config change? | 20:00 |
| fungi | #status log OSIC environment is active in Nodepool and running jobs normally once more | 20:00 |
| openstackstatus | fungi: finished logging | 20:00 |
| pabelanger | kmARC: np, were here to help | 20:00 |
| clarkb | I dunno to me updating apache is far simpler here | 20:00 |
| pabelanger | fungi: thanks! | 20:00 |
| pabelanger | fungi: will be a sad day when we lose it :( | 20:00 |
| *** e0ne has joined #openstack-infra | 20:01 | |
| *** lrossetti_ has quit IRC | 20:01 | |
| fungi | pabelanger: we'll get by, but it's been nice while we have it! | 20:01 |
| *** lrossetti has joined #openstack-infra | 20:01 | |
| pabelanger | ++ | 20:01 |
| clarkb | kmARC: pabelanger also you'll want to "curate" the repo branches and tags before hand as we import them all as is | 20:02 |
| *** skelso has quit IRC | 20:03 | |
| *** skelso has joined #openstack-infra | 20:03 | |
| sdague | clarkb: are those really all today? because some of those 404s we already have rules for | 20:03 |
| clarkb | sdague: its the last ~day. I'm actually looking at last two log files | 20:03 |
| clarkb | and they get rotated daily | 20:04 |
| sdague | ok | 20:04 |
| sdague | so fixes we landed yesterday will not be reflected here yet | 20:04 |
| sdague | that's fine | 20:04 |
| jeblair | clarkb: 4 lines of apache doesn't include creating and managing the directories, etc. honestly either solution is about the same complexity. 4 lines of apache vs 4 lines of puppet. i'm just suggesting that using afs the way we're already using it seems very straightforward. | 20:04 |
| clarkb | sdague: correct | 20:04 |
| *** skelso has quit IRC | 20:08 | |
| *** skelso has joined #openstack-infra | 20:08 | |
| jeblair | clarkb: to be clear, i'm happy to have either. | 20:08 |
| *** krtaylor has quit IRC | 20:08 | |
| clarkb | jeblair: I'm just concerned that we are taking a compeltely local resource and by hosting it remotely can only make it more complicated | 20:08 |
| jeblair | clarkb: yep. i'm just trying to convince you that having our afs-based webservers serve data from afs isn't complicated. if i've failed, it's okay, i'll move on. | 20:10 |
| jeblair | it's not that important and it's easy to change | 20:10 |
| jeblair | it's more important for sdague to get the list somehow. :) | 20:11 |
| sdague | yes, I do not care about implementation at all. Anything that gets me a list is more information than I have now. :) | 20:11 |
| sdague | I would prefer to have a historical record, so we can track progress, as that's been important in getting people to contribute | 20:12 |
| AJaeger | clarkb: thanks for the list, I addressed the top 4 of the 404s - and a few more further down... | 20:13 |
| AJaeger | clarkb: for progress, it would be nice to give the total list of pages in the list as summary. | 20:14 |
| *** e0ne has quit IRC | 20:15 | |
| *** slaweq has quit IRC | 20:18 | |
| *** slaweq has joined #openstack-infra | 20:19 | |
| openstackgerrit | Hongbin Lu proposed openstack-infra/project-config master: [Kuryr] Setup a non-voting py35 job for libnetwork https://review.openstack.org/490169 | 20:20 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Generate list of 404s for docs.o.o https://review.openstack.org/490175 | 20:20 |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test https://review.openstack.org/475747 | 20:20 |
| clarkb | sdague: jeblair ^ that addresses most of the comments I've seen on irc. And should be largely applicable whether we host locally or in afs | 20:21 |
| *** Julien-zte has quit IRC | 20:21 | |
| clarkb | gives you a month worth of data, doesn't trim it to top 100, and serves it out of a dir with file per day | 20:21 |
| jeblair | clarkb: is the require ssl/private intentional? | 20:22 |
| clarkb | no thats like copy pasta fail | 20:23 |
| *** krtaylor has joined #openstack-infra | 20:23 | |
| jeblair | kk | 20:23 |
| *** Julien-zte has joined #openstack-infra | 20:23 | |
| clarkb | ya copy pasta fail | 20:23 |
| *** slaweq has quit IRC | 20:23 | |
| *** Douhet has quit IRC | 20:23 | |
| jeblair | clarkb: otherwise lgtm | 20:24 |
| *** slaweq has joined #openstack-infra | 20:25 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Generate list of 404s for docs.o.o https://review.openstack.org/490175 | 20:25 |
| clarkb | that removes the unnecessary require. Now to properly test that script | 20:25 |
| *** https_GK1wmSU has joined #openstack-infra | 20:26 | |
| *** bobh has quit IRC | 20:27 | |
| *** e0ne has joined #openstack-infra | 20:27 | |
| *** https_GK1wmSU has left #openstack-infra | 20:27 | |
| *** Douhet has joined #openstack-infra | 20:29 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Add README for tox role https://review.openstack.org/489750 | 20:31 |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Update tox-tarball to use tox role https://review.openstack.org/489705 | 20:32 |
| clarkb | oh there is slight bug | 20:33 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Use undefined values instead of 40 zeroes https://review.openstack.org/486225 | 20:35 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Generate list of 404s for docs.o.o https://review.openstack.org/490175 | 20:36 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-openstack_health master: Switch api server to use system python instead of venv https://review.openstack.org/490123 | 20:36 |
| clarkb | I've run that script against test input and output looks good now. But there was bug in last patchset which this one fixes | 20:37 |
| *** Douhet has quit IRC | 20:39 | |
| *** Douhet has joined #openstack-infra | 20:40 | |
| *** hashar has joined #openstack-infra | 20:40 | |
| *** lrossetti has quit IRC | 20:42 | |
| *** jascott1 has quit IRC | 20:42 | |
| *** efoley has joined #openstack-infra | 20:42 | |
| *** lrossetti has joined #openstack-infra | 20:42 | |
| rm_work | fungi / clarkb: so would https://review.openstack.org/#/c/490121/ require testing to merge it? It seems like the testing is only really in place for core projects | 20:44 |
| clarkb | sdague: ok responded to the list with details and an example output of running that script against my test inputs | 20:44 |
| sdague | clarkb: thank you | 20:45 |
| *** gyee has quit IRC | 20:48 | |
| *** krtaylor has quit IRC | 20:50 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Bindmount /etc/lsb-release into bubblewrap https://review.openstack.org/490200 | 20:50 |
| *** e0ne has quit IRC | 20:52 | |
| *** jrist has quit IRC | 20:54 | |
| *** jamesmcarthur has quit IRC | 20:54 | |
| openstackgerrit | Mark Korondi proposed openstack-infra/project-config master: Bringing upstream training virtual environment over here https://review.openstack.org/490202 | 20:55 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Create publish-tox-openstack-tarballs trusted job https://review.openstack.org/490132 | 20:55 |
| *** Goneri has quit IRC | 20:57 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul-jobs master: Log file stats for tox tarball / wheels https://review.openstack.org/490045 | 20:58 |
| *** vhosakot has quit IRC | 20:58 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Don't request empty nodesets https://review.openstack.org/487243 | 20:58 |
| *** vhosakot has joined #openstack-infra | 21:00 | |
| openstackgerrit | Gabriele Cerami proposed openstack-infra/tripleo-ci master: WIP: containers periodic test https://review.openstack.org/475747 | 21:01 |
| EmilienM | pabelanger: https://review.openstack.org/490206 | 21:04 |
| pabelanger | EmilienM: it is likely the variable isn't on disk just yet | 21:05 |
| fungi | clarkb: i have some annoying news... apparently gerrit's rest api is befuddled by %-encoded query strings... before i dig too deeply into requests documentation, do you happen to know of simple ways to figure out which encoded things the remote system chokes on and disable encoding just those? | 21:05 |
| EmilienM | pabelanger: I know but I'm ready :) | 21:06 |
| *** thorst_afk has quit IRC | 21:06 | |
| EmilienM | pabelanger: doing the same for tripleo now | 21:06 |
| *** jamesden_ has quit IRC | 21:06 | |
| fungi | clarkb: aha, after much trial and error, it's that gerrit dislikes + being rewritten to %2B | 21:07 |
| pabelanger | EmilienM: I can kick off a image build for centos-7 now to pick it up | 21:08 |
| *** esberglu has quit IRC | 21:08 | |
| *** ijw has joined #openstack-infra | 21:08 | |
| pabelanger | EmilienM: or wait until tonight | 21:08 |
| *** yamamoto_ has joined #openstack-infra | 21:08 | |
| jeblair | fungi: https://docs.python.org/3/library/urllib.parse.html#urllib.parse.quote_plus is what gertty uses in everything it passes to gerrit as a url | 21:09 |
| *** eharney has quit IRC | 21:10 | |
| fungi | jeblair: i'll give that a shot. other solutions i'm seeing suggested involve subclassing to override requests.Session.send() with a custom urllib.quote list | 21:10 |
| jeblair | that could have been more clear: that's what gertty uses when constructing all of the gerrit rest api urls it uses | 21:10 |
| *** thorst_afk has joined #openstack-infra | 21:11 | |
| *** slaweq has quit IRC | 21:13 | |
| *** annegentle has quit IRC | 21:13 | |
| EmilienM | pabelanger: I don't mind waiting | 21:14 |
| *** annegentle has joined #openstack-infra | 21:14 | |
| *** thorst_afk has quit IRC | 21:15 | |
| *** yamamoto_ has quit IRC | 21:15 | |
| *** hashar has quit IRC | 21:15 | |
| *** ldnunes has quit IRC | 21:15 | |
| clarkb | fungi: sorry was getting csa box, I'm not sure of a way to easily determine that but sounds like you got it sorted | 21:16 |
| jesusaur | the gerrit change-ids in commit messages become links in a git hook, right? where does that live in config? | 21:16 |
| *** yamamoto_ has joined #openstack-infra | 21:17 | |
| clarkb | jesusaur: they become links via the comment links config can be found at system-config/modules/openstack_projects/manifests/review.pp iirc | 21:17 |
| fungi | jeblair: clarkb: actually, i just found requests.utils.requote_uri(), but turns out that requests seems to use urllib.parse.quote_plus() by default so just having " " in place of "+" in my parameters seems to do the trick! | 21:17 |
| openstackgerrit | Mark Korondi proposed openstack-infra/project-config master: Bringing upstream training virtual environment over here https://review.openstack.org/490202 | 21:18 |
| clarkb | fungi: ah its the whitespace that needs to become + instead of + becoming escaped | 21:18 |
| fungi | jeblair: clarkb: i added some debugging to splat out the url object from the request and indeed " " was transformed to "+" right before my very eyes | 21:18 |
| openstackgerrit | Mark Korondi proposed openstack-infra/project-config master: Bringing upstream training virtual environment over here https://review.openstack.org/490202 | 21:19 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Make timer tests less racy https://review.openstack.org/490208 | 21:19 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: External Calendar Sync https://review.openstack.org/487683 | 21:19 |
| jesusaur | clarkb: ah, thanks | 21:20 |
| jeblair | fungi: magic! | 21:21 |
| *** jascott1 has joined #openstack-infra | 21:21 | |
| *** priteau has quit IRC | 21:21 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Refactor requests usage in owners script https://review.openstack.org/490209 | 21:22 |
| fungi | clarkb: jeblair: i'll wip that ^ until i get a comparison run before and after to confirm they produce the same data | 21:22 |
| fungi | and then i'll rebuild the rest of the other series on top of that reusing the new functions | 21:23 |
| *** dprince has quit IRC | 21:23 | |
| fungi | clarkb: hopefully that will address your parameter encoding concerns with the member directory api calls in 483974? | 21:23 |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Add /etc/lsb-release to zuul bwrap environment https://review.openstack.org/490211 | 21:24 |
| *** jrist has joined #openstack-infra | 21:26 | |
| openstackgerrit | Merged openstack-infra/system-config master: Use trusted_ro_paths https://review.openstack.org/488862 | 21:27 |
| *** baoli has quit IRC | 21:28 | |
| *** Douhet has quit IRC | 21:33 | |
| *** Douhet has joined #openstack-infra | 21:33 | |
| openstackgerrit | Mark Korondi proposed openstack-infra/project-config master: Bringing upstream training virtual environment over here https://review.openstack.org/490202 | 21:34 |
| *** jamesmcarthur has joined #openstack-infra | 21:35 | |
| *** esberglu has joined #openstack-infra | 21:37 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Allow use of the uri module from localhost https://review.openstack.org/490215 | 21:37 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Prevent execution of locally overridden core modules https://review.openstack.org/490216 | 21:37 |
| *** jamesmca_ has joined #openstack-infra | 21:39 | |
| *** jamesmcarthur has quit IRC | 21:39 | |
| *** slaweq has joined #openstack-infra | 21:43 | |
| *** thorst_afk has joined #openstack-infra | 21:46 | |
| *** vhosakot has quit IRC | 21:47 | |
| openstackgerrit | Mark Korondi proposed openstack-infra/project-config master: Bringing upstream training virtual environment over here https://review.openstack.org/490202 | 21:48 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Add zuul.executor.work_root to inventory https://review.openstack.org/490173 | 21:49 |
| *** thorst_afk has quit IRC | 21:50 | |
| *** slaweq has quit IRC | 21:52 | |
| *** slaweq has joined #openstack-infra | 21:52 | |
| *** kgiusti has left #openstack-infra | 21:53 | |
| *** slaweq has quit IRC | 21:56 | |
| *** slaweq has joined #openstack-infra | 21:56 | |
| *** ijw has quit IRC | 21:57 | |
| *** jamesmca_ has quit IRC | 21:57 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Create publish-openstack-python-tarball trusted job https://review.openstack.org/490132 | 21:58 |
| *** jamesmcarthur has joined #openstack-infra | 21:58 | |
| clarkb | fungi: yup +2 lgtm | 21:59 |
| *** efoley has quit IRC | 21:59 | |
| *** jamesmcarthur has quit IRC | 21:59 | |
| *** jamesmcarthur has joined #openstack-infra | 21:59 | |
| *** xyang1 has quit IRC | 21:59 | |
| *** annegentle has quit IRC | 22:00 | |
| *** jkilpatr has quit IRC | 22:01 | |
| *** zer0c00l_ has joined #openstack-infra | 22:01 | |
| zer0c00l_ | Any one running into issues with devstack stable/ocata branch on RHEL 7? | 22:01 |
| *** armax has quit IRC | 22:01 | |
| zer0c00l_ | i am having issues with libvirt-python package | 22:01 |
| zer0c00l_ | https://bugs.launchpad.net/devstack/+bug/1636567 | 22:01 |
| openstack | Launchpad bug 1636567 in devstack "devstack mitaka installation fails with error "Running setup.py bdist_wheel for libvirt-python: finished with status 'error'" in Ubuntu 16.10" [Undecided,Opinion] | 22:01 |
| zer0c00l_ | Not sure if this is an appropriate channel for devstack questions | 22:02 |
| pabelanger | zer0c00l_: we only have access to centos-7 upstream | 22:02 |
| *** panda has quit IRC | 22:02 | |
| pabelanger | zer0c00l_: ianw does a good job keeping devstack running on centos-7 | 22:02 |
| clarkb | zer0c00l_: pabelanger lookng at it though looks like new rhel release has libvirt 3.2 but we use libvirt-python 2.5 | 22:03 |
| *** marst has quit IRC | 22:03 | |
| clarkb | you may need to update to libvirt-python 3.something? | 22:03 |
| zer0c00l_ | clarkb: yes | 22:03 |
| clarkb | zer0c00l_: that is what I would try. Update constraints to use whatever is latest libvirt-python and see if that fixes | 22:03 |
| zer0c00l_ | clarkb: so, how do i change the upper constraints in the middle of devstack installation? | 22:04 |
| zer0c00l_ | stable/ocata seems to be tied to libvirt-python 2.5.0 | 22:04 |
| zer0c00l_ | master has libvirt-python 3.5.x i think | 22:04 |
| *** skelso has quit IRC | 22:05 | |
| clarkb | zer0c00l_: one way is to make the commit in the requirements repo then set the error on clone flag in devstack to prevent it from updatign the repo | 22:05 |
| clarkb | I think you might also be able to set the ref you want | 22:05 |
| zer0c00l_ | Ah i see | 22:05 |
| zer0c00l_ | You are suggesting i summit a review to stable/ocata branch and see the REPO BRANCH to use the review? | 22:06 |
| zer0c00l_ | Am i understanding it right? | 22:06 |
| clarkb | you don't have to submit a review to test it locally but that is one way to do it | 22:06 |
| clarkb | you can just edit it locally | 22:07 |
| zer0c00l_ | I see, i am using chef to do my devstack deployments. | 22:07 |
| zer0c00l_ | There is no manualy editing there :( | 22:07 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add some lookup plugin tests https://review.openstack.org/454396 | 22:08 |
| clarkb | fungi: before I forget and before you disappear for the evening. Is september 18 still a go for upgrading gerrit? | 22:09 |
| clarkb | I haven't heard any one say its a bad day, but you had the known conflict | 22:09 |
| *** ijw has joined #openstack-infra | 22:09 | |
| clarkb | (want to make sure I get the email out soon) | 22:09 |
| *** https_GK1wmSU has joined #openstack-infra | 22:10 | |
| *** felipemonteiro__ has quit IRC | 22:10 | |
| *** https_GK1wmSU has left #openstack-infra | 22:11 | |
| fungi | clarkb: yeah, christine has given me a list of ways to make it up to her ;) | 22:12 |
| fungi | announce at will | 22:12 |
| *** slaweq has quit IRC | 22:13 | |
| *** slaweq has joined #openstack-infra | 22:13 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Docs: use zuul:attr in project definition https://review.openstack.org/489774 | 22:13 |
| clarkb | mtreinish: mqtt things are bropken in logstash woprker | 22:15 |
| clarkb | mtreinish: File "/usr/local/bin/log-gearman-worker.py", line 366 IndentationError: unexpected indent | 22:15 |
| *** gouthamr has quit IRC | 22:16 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Don't request empty nodesets https://review.openstack.org/487243 | 22:16 |
| openstackgerrit | Merged openstack-infra/project-config master: Create publish-openstack-python-tarball trusted job https://review.openstack.org/490132 | 22:17 |
| *** slaweq has quit IRC | 22:17 | |
| clarkb | mtreinish: and then AttributeError: 'Server' object has no attribute 'configget' | 22:18 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-log_processor master: Fix syntax errors https://review.openstack.org/490233 | 22:19 |
| clarkb | mtreinish: fungi ^ review on that much appreciated | 22:19 |
| clarkb | we are mostly not running logstash workers right now because of that | 22:19 |
| *** thingee has quit IRC | 22:19 | |
| fungi | clarkb: when did that break? wondering what bug we landed | 22:22 |
| mtreinish | clarkb: sigh, I was really off by 1 space on that indent | 22:23 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Docs: use zuul:attr in secrets https://review.openstack.org/490236 | 22:23 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-log_processor master: Fix syntax errors https://review.openstack.org/490233 | 22:23 |
| mtreinish | fungi: clarkb approved the rest of the more-firehose patches earlier today, which included the support for the mqtt worker stuff | 22:24 |
| clarkb | ya so it broke earlier today when puppet applied it. | 22:24 |
| clarkb | new patchset fixes more things | 22:24 |
| clarkb | mtreinish: also tls is unworking | 22:24 |
| clarkb | mtreinish: ValueError: Python 2.7.9 and 3.2 are the minimum supported versions for TLS. | 22:24 |
| fungi | cool, so at least we weren't broken terribly long :/ | 22:24 |
| *** ijw has quit IRC | 22:24 | |
| mtreinish | clarkb: oh that's the latest version of paho thing | 22:25 |
| mtreinish | we hit that on another trusty server | 22:25 |
| clarkb | mtreinish: I think it may be breaking publishing to elasticsaerch too because it bails out the entire run loop | 22:25 |
| mtreinish | we can downgrade to <1.3.0 iirc | 22:25 |
| clarkb | but maybe not because there is a time delta between pushing logs to logsatsh and getting that message | 22:26 |
| clarkb | may just force us to redo run loop for every file | 22:26 |
| fungi | yeah, on that note review of 485275 would be appreciated too | 22:26 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Use undefined values instead of 40 zeroes https://review.openstack.org/486225 | 22:27 |
| *** gordc has quit IRC | 22:28 | |
| clarkb | mtreinish: can you push a port of https://review.openstack.org/#/c/485275/1/modules/openstack_project/manifests/puppetmaster.pp for the logstash workers? or should I go ahead and do that? | 22:28 |
| *** yamamoto_ has quit IRC | 22:29 | |
| *** EricGonczer_ has quit IRC | 22:29 | |
| mtreinish | clarkb: I can, unless you want to just add it onto 490233 | 22:30 |
| clarkb | mtreinish: nah 490233 is already approved and neatly fixes the python bugs | 22:30 |
| mtreinish | it gets installed in init.pp of puppet-log_processor | 22:30 |
| clarkb | mtreinish: so second change would be good | 22:30 |
| mtreinish | ah ok, I'll push that up right now, one sec | 22:30 |
| *** thingee has joined #openstack-infra | 22:32 | |
| *** yamamoto has joined #openstack-infra | 22:32 | |
| *** jamesmcarthur has quit IRC | 22:33 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-log_processor master: Ensure we use paho 1.2.3 https://review.openstack.org/490237 | 22:34 |
| mtreinish | clarkb: ^^^ | 22:34 |
| fungi | clarkb: catching up on e-mail, should we exclude the 404s page in robots.txt? might cause some mighty strange search results on things appearing in those url strings | 22:35 |
| fungi | happy to propose the patch for that | 22:35 |
| clarkb | fungi: oh probably a good idea, you could tack it onto my change or a followup | 22:36 |
| *** yamamoto has quit IRC | 22:36 | |
| openstackgerrit | Merged openstack-infra/system-config master: Pin paho-mqtt to 1.2.3 on the puppetmaster https://review.openstack.org/485275 | 22:37 |
| *** jkilpatr has joined #openstack-infra | 22:38 | |
| *** Sukhdev has quit IRC | 22:40 | |
| *** yamamoto has joined #openstack-infra | 22:41 | |
| clarkb | infra-root https://etherpad.openstack.org/p/gerrit-2.13-upgrade-notice how does that look for gerrit upgrade notice | 22:42 |
| mordred | clarkb: reading | 22:42 |
| *** yamamoto has quit IRC | 22:44 | |
| openstackgerrit | Merged openstack-infra/puppet-log_processor master: Fix syntax errors https://review.openstack.org/490233 | 22:44 |
| fungi | clarkb: i just reread the change and realized you're serving that as files.o.o/404s not docs.o.o/404s, so no need to adjust robots.txt assuming that's the url you meant for it to have? | 22:46 |
| clarkb | fungi: oh I meant docs.o.o. Did I not have the right vhost? | 22:46 |
| fungi | seems like you did not | 22:47 |
| clarkb | indeed | 22:47 |
| fungi | you put it in modules/openstack_project/templates/files.vhost.erb rather than modules/openstack_project/templates/docs.vhost.erb | 22:47 |
| clarkb | looking at it though files.o.o is where we put the unrendered stuff so maybe that is a better home for this data? | 22:48 |
| clarkb | then I can rename it to docs-404s | 22:48 |
| fungi | i was initially thrown because of your pointing 404s to docs-404s and assumed it was on the docs site until i tried to figure out why the robots.txt entry was entirely missing from the docs vhost definition | 22:48 |
| fungi | so yes, just keep the url the same as the filename | 22:49 |
| fungi | and then we're all good | 22:49 |
| clarkb | ya let me push that patch up | 22:49 |
| clarkb | I do think files.o.o is a better home for this looking at what it is serving | 22:49 |
| mordred | clarkb: etherpad looks great | 22:49 |
| clarkb | and then I don't even need my alias | 22:49 |
| fungi | only downside is that'll shadow any top-level docs-404s file/directory from our afs tree, but not really that critical | 22:49 |
| mordred | clarkb: do you think it's worth mentioning that we expect this to be the last time we have to do an offline secondary index update? | 22:49 |
| clarkb | oh is that all afs? then ya still need alias | 22:50 |
| clarkb | mordred: I think if we get better at upgrading release to release yes | 22:50 |
| *** jkilpatr has quit IRC | 22:50 | |
| clarkb | since they seem to be trying to make it so that n-1 to n can do an online reindex | 22:50 |
| *** jkilpatr has joined #openstack-infra | 22:50 | |
| *** sdague has quit IRC | 22:50 | |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Docs: use zuul:attr in secrets https://review.openstack.org/490236 | 22:52 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Generate list of 404s for docs.o.o https://review.openstack.org/490175 | 22:52 |
| clarkb | fungi: ^ how is that? | 22:52 |
| openstackgerrit | Merged openstack-infra/puppet-log_processor master: Ensure we use paho 1.2.3 https://review.openstack.org/490237 | 22:54 |
| clarkb | mordred: I'm hoping that with 2.13 behind us going to 2.14 while complicated will be easier simply because we've got a recent one under our belts | 22:54 |
| clarkb | mordred: then as long as we upgrade release to release we should be good. | 22:54 |
| fungi | clarkb: that seems saner, though i haven't finished reading through the script itself yet | 22:55 |
| *** pbourke has quit IRC | 22:55 | |
| *** jrist has quit IRC | 22:55 | |
| fungi | clarkb: also, maintenance announcement lgtm--thanks! | 22:55 |
| clarkb | ok sending out maintenance announcement now | 22:55 |
| *** yamamoto has joined #openstack-infra | 22:56 | |
| *** pbourke has joined #openstack-infra | 22:57 | |
| mordred | clarkb: ++ | 22:57 |
| mordred | clarkb: I'd love to get on a more frequent cadence with that - especially if we can do it with minimal downtime and whatnot | 22:58 |
| fungi | part of the delay for this one is that gerrit released a new version while we were evaluating the old one and then we mostly started over again | 22:58 |
| mordred | yup | 22:58 |
| clarkb | gerrit upgrades have honestly also gotten a lot more scary | 22:59 |
| fungi | and when they released a new version the second time, we decided to stick with what we'd already tested rather than keep spinning our wheels forever | 22:59 |
| clarkb | the track record of not having world breaking bugs is not great recently | 22:59 |
| clarkb | (I think 2.13 is in good shape though as its at the end of its life and has had 9 point releases and wmf runs it) | 22:59 |
| clarkb | but ya | 22:59 |
| fungi | i loved the forum reply where they asked you why we were upgrading to 2.13 and not 2.14 | 23:00 |
| *** jaypipes has quit IRC | 23:00 | |
| *** xarses has quit IRC | 23:00 | |
| clarkb | I do think that step 0 of 2.14 is replacing the database | 23:01 |
| clarkb | we should get on mysql 5.6 or 5.7, have 4 byte utf8. | 23:02 |
| fungi | i wouldn't mind seeing us do that soon after the 2.13 upgrade even | 23:02 |
| clarkb | ++ | 23:02 |
| fungi | just to have it out of the way | 23:02 |
| fungi | that would be a relatively fast maintenance | 23:02 |
| *** rbrndt has quit IRC | 23:03 | |
| clarkb | as a heads up I have removed logstash-worker01 from the ansible emergency file. I think it was put in there the first time we did mqtt things | 23:03 |
| *** gtrxcb has joined #openstack-infra | 23:06 | |
| fungi | oh, hah. fim | 23:07 |
| fungi | s/fim/fun/ | 23:07 |
| *** jascott1 has quit IRC | 23:07 | |
| *** jascott1 has joined #openstack-infra | 23:08 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Prevent execution of locally overridden core modules https://review.openstack.org/490216 | 23:09 |
| clarkb | as soon as paho version updates I am going to start all the daemons | 23:09 |
| clarkb | *logstash daemons | 23:09 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Prevent execution of locally overridden core modules https://review.openstack.org/490216 | 23:09 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: Allow use of the uri module from localhost https://review.openstack.org/490215 | 23:09 |
| *** jascott1 has quit IRC | 23:10 | |
| *** yamamoto has quit IRC | 23:11 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Add zuul:var directive and role https://review.openstack.org/490246 | 23:13 |
| clarkb | the puppet run starting now should update paho-mqtt for us | 23:14 |
| *** yamamoto has joined #openstack-infra | 23:14 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Ensure ref-updated jobs run with their ref https://review.openstack.org/488216 | 23:16 |
| *** hongbin has quit IRC | 23:21 | |
| *** dingyichen has joined #openstack-infra | 23:22 | |
| *** aeng has joined #openstack-infra | 23:23 | |
| *** http_GK1wmSU has joined #openstack-infra | 23:23 | |
| *** http_GK1wmSU has left #openstack-infra | 23:24 | |
| *** gildub has joined #openstack-infra | 23:28 | |
| *** gouthamr has joined #openstack-infra | 23:32 | |
| *** aviau has quit IRC | 23:36 | |
| *** aviau has joined #openstack-infra | 23:36 | |
| clarkb | mtreinish: fungi http://paste.openstack.org/show/617350/ fyi | 23:36 |
| clarkb | I think the user there wants to be logproccessor | 23:37 |
| * clarkb writes a patch | 23:38 | |
| *** mtanino_ is now known as mtanino | 23:39 | |
| ianw | new logo day huh | 23:39 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Don't use subunit user on logstash workers https://review.openstack.org/490249 | 23:40 |
| clarkb | ianw: indeed! | 23:40 |
| clarkb | mtreinish: fungi ianw https://review.openstack.org/490249 that should fix the subunit user error in the paste up there | 23:40 |
| *** jascott1 has joined #openstack-infra | 23:42 | |
| *** krtaylor has joined #openstack-infra | 23:43 | |
| mtreinish | clarkb: :/ lol, I really made a bunch of mistakes on this roll out | 23:44 |
| clarkb | ansible looks like it may have gone out to lunch at 23:22 too | 23:44 |
| clarkb | pabelanger: what does it mean when all of the ansible playbook processes are defunct? | 23:45 |
| pabelanger | clarkb: usually problems SSHing to remote node | 23:45 |
| pabelanger | clarkb: if on puppetmaster, they will get killed by timeout fuction | 23:46 |
| pabelanger | OH | 23:46 |
| pabelanger | I wonder if it is related to new mirrors | 23:46 |
| pabelanger | I didn't ssh into it from puppetmaster this morning | 23:46 |
| pabelanger | or delete old infracloud mirror in openstack client | 23:46 |
| clarkb | ah ok, I'm impatietn so I went ahead and downgarded paho and restarted things | 23:50 |
| clarkb | but when puppet runs it should be happy | 23:51 |
| fungi | pabelanger: also if you replace a server you need to blow away the inventory cache | 23:51 |
| fungi | otherwise it gets mightily confused from what i've seen | 23:52 |
| clarkb | mtreinish: fungi can you check that mqtt is happening as expected now? | 23:54 |
| clarkb | all but logstash-worker01 should be configured to mqtt | 23:55 |
| *** thorst_afk has joined #openstack-infra | 23:55 | |
| fungi | the example from https://docs.openstack.org/infra/system-config/firehose.html#cli-example is at least showing gerrit events... | 23:56 |
| openstackgerrit | Paul Belanger proposed openstack-infra/zuul feature/zuulv3: Use py35 tox envlist for documentation https://review.openstack.org/489707 | 23:56 |
| mtreinish | clarkb: I haven't seen any logstash event come by yet | 23:57 |
| *** http_GK1wmSU has joined #openstack-infra | 23:57 | |
| fungi | nor ansible | 23:57 |
| mtreinish | just launchpad, gerrit, and gearman-subunit so far | 23:58 |
| mtreinish | fungi: it might be in an ansible lull | 23:58 |
| mtreinish | the storm is coming :) | 23:58 |
| fungi | {"status": "success", "build_uuid": "a672db40bc664541aab995f926724aeb"} | 23:58 |
| fungi | that's gearman-subunit? | 23:58 |
| *** http_GK1wmSU has left #openstack-infra | 23:58 | |
| mtreinish | probably | 23:58 |
| mtreinish | I'm also printing the topics in mine and I haven't seen a logstash one yet | 23:59 |
| fungi | if the ansible storm comes, it'll start in about 45 seconds | 23:59 |
| mtreinish | I also have another window subscribed to gearman-logstash/# | 23:59 |
| mtreinish | which hasn't printed anything yet | 23:59 |
| fungi | how do you print topics? | 23:59 |
| fungi | with mosquitto_sub anyway? | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
