| fungi | i think that's planned for the public api, but i don't think it's implemented yet | 00:00 |
|---|---|---|
| fungi | i can manually query the administrative rpc api and get you a list | 00:00 |
| *** jesusaur has quit IRC | 00:01 | |
| *** diablo_rojo has quit IRC | 00:02 | |
| *** eernst has joined #openstack-infra | 00:03 | |
| studarus | yes, please. | 00:03 |
| studarus | Nodepool thinks there are 30 in a deleting state. | 00:04 |
| studarus | but my building + ready + in use = what openstack says is running | 00:05 |
| studarus | so those 30 don't exist... | 00:05 |
| *** eernst has quit IRC | 00:07 | |
| *** yamamoto has joined #openstack-infra | 00:08 | |
| *** eernst_ has joined #openstack-infra | 00:09 | |
| fungi | studarus: http://paste.openstack.org/show/731307 | 00:10 |
| studarus | Thank you! Looks like they are there in a mix of build and active states. Bizarre. | 00:12 |
| studarus | Does nodepool retry deletes? | 00:12 |
| fungi | yes | 00:12 |
| fungi | something like every 5 seconds i think? | 00:12 |
| *** eernst_ has quit IRC | 00:14 | |
| *** markvoelker has quit IRC | 00:17 | |
| *** markvoelker has joined #openstack-infra | 00:17 | |
| *** jesusaur has joined #openstack-infra | 00:20 | |
| *** agopi|off has joined #openstack-infra | 00:21 | |
| *** markvoelker has quit IRC | 00:21 | |
| studarus | looks like it was a state issue | 00:27 |
| studarus | I had to run nova reset-state before I could delete it | 00:27 |
| studarus | is this a worthwhile bug to file? | 00:27 |
| studarus | nodepool had likely been trying for days to delete the instance... | 00:28 |
| *** gyee has quit IRC | 00:29 | |
| fungi | can't hurt to find out if it's been reported to nova yet | 00:32 |
| fungi | though probably need to know what caused it to get out of sync | 00:32 |
| studarus | I was going to file it with nodepool... | 00:33 |
| studarus | that if it tries for a long time to delete an instance to reset the state | 00:33 |
| fungi | oh... how would nodepool reset the state? | 00:33 |
| fungi | doesn't that require administrative access to nova? | 00:34 |
| studarus | hmmm | 00:34 |
| fungi | nodepool doesn't expect to have any admin access to a cloud, the vast majority of our quota is in public cloud providers who won't be giving us admin access to their infrastructure for obvious reasons | 00:35 |
| studarus | gotcha - well, at least I know the workaround in case I see it again. Plus it is nice to see the Grafana graphs pretty again (delete count went from 30 to 2) | 00:39 |
| studarus | And in use count is back to 99% :) | 00:39 |
| fungi | nice! | 00:40 |
| fungi | thanks for looking into it too | 00:40 |
| *** longkb has joined #openstack-infra | 00:55 | |
| *** studarus has quit IRC | 01:03 | |
| *** Qiming has quit IRC | 01:06 | |
| *** Qiming has joined #openstack-infra | 01:09 | |
| *** hamzy has joined #openstack-infra | 01:12 | |
| *** ianychoi has quit IRC | 01:17 | |
| *** mrsoul has joined #openstack-infra | 01:19 | |
| mtreinish | hmm, it looks like I can't connect to the subunit2sql db directly anymore: http://paste.openstack.org/show/731310/ | 01:23 |
| mtreinish | maybe something got messed up with the forwarding rule? It looks like openstack-health is still wokring | 01:23 |
| mtreinish | infra-root: ^^^ | 01:24 |
| *** ianychoi has joined #openstack-infra | 01:28 | |
| fungi | it's likely something was missed when iptables configuration got moved from puppet to ansible | 01:28 |
| *** ianychoi has quit IRC | 01:42 | |
| mtreinish | fungi: that would explain it | 01:44 |
| fungi | kinda late here and we're short a lot of the team this week with ansiblefest going on, but i'll try to remember to look into it | 01:46 |
| mtreinish | there's no rush, as I'm probably the only person who ever used the sql interface | 01:47 |
| mtreinish | and it's not super critical for me, I was just trying to pull the row count for one of the tables to answer a question I got | 01:48 |
| mtreinish | although I guess it does block anyone using subunit2sql-graph or any external subunit2sql db api usage too (which is a bit more common) | 01:49 |
| mtreinish | (but not super common either) | 01:49 |
| fungi | yeah, i mean, we still should figure it out and fix it ;) | 01:50 |
| *** armax has quit IRC | 01:50 | |
| *** dave-mccowan has joined #openstack-infra | 01:56 | |
| *** gfidente has joined #openstack-infra | 01:56 | |
| *** jbadiapa has quit IRC | 01:56 | |
| *** jbadiapa has joined #openstack-infra | 01:58 | |
| *** jamesmcarthur has joined #openstack-infra | 01:58 | |
| *** tinwood has quit IRC | 02:10 | |
| *** tinwood has joined #openstack-infra | 02:11 | |
| *** annp has joined #openstack-infra | 02:16 | |
| *** markvoelker has joined #openstack-infra | 02:18 | |
| *** ianychoi has joined #openstack-infra | 02:19 | |
| *** markvoelker has quit IRC | 02:51 | |
| *** roman_g has quit IRC | 02:51 | |
| *** aidin has joined #openstack-infra | 02:53 | |
| *** jamesmcarthur has quit IRC | 02:59 | |
| *** dave-mccowan has quit IRC | 03:08 | |
| *** aidin has quit IRC | 03:14 | |
| *** aidin has joined #openstack-infra | 03:25 | |
| *** hongbin has joined #openstack-infra | 03:41 | |
| *** ykarel has joined #openstack-infra | 03:47 | |
| *** markvoelker has joined #openstack-infra | 03:48 | |
| *** hongbin has quit IRC | 03:59 | |
| *** Douhet has joined #openstack-infra | 04:01 | |
| *** njohnston has joined #openstack-infra | 04:09 | |
| *** longkb has quit IRC | 04:10 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove release-openstack-server template https://review.openstack.org/531830 | 04:19 |
| *** markvoelker has quit IRC | 04:21 | |
| *** ramishra has joined #openstack-infra | 04:33 | |
| *** rossella_s has quit IRC | 04:34 | |
| *** rossella_s has joined #openstack-infra | 04:35 | |
| *** aidin has quit IRC | 04:36 | |
| *** rossella_s has quit IRC | 04:40 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: fetch-javascript-content-tarball: add create_tarball variable https://review.openstack.org/606740 | 04:44 |
| *** ykarel_ has joined #openstack-infra | 04:45 | |
| *** rossella_s has joined #openstack-infra | 04:46 | |
| *** chandankumar has quit IRC | 04:46 | |
| *** chandankumar has joined #openstack-infra | 04:47 | |
| *** ykarel has quit IRC | 04:48 | |
| *** jamesmcarthur has joined #openstack-infra | 04:59 | |
| *** Bhujay has joined #openstack-infra | 05:01 | |
| *** Bhujay has quit IRC | 05:01 | |
| *** Bhujay has joined #openstack-infra | 05:02 | |
| *** jamesmcarthur has quit IRC | 05:04 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: remove job settings for heat repositories https://review.openstack.org/597289 | 05:13 |
| *** ykarel_ is now known as ykarel | 05:13 | |
| *** markvoelker has joined #openstack-infra | 05:18 | |
| *** rossella_s has quit IRC | 05:30 | |
| *** rossella_s has joined #openstack-infra | 05:30 | |
| *** rossella_s has quit IRC | 05:35 | |
| *** rossella_s has joined #openstack-infra | 05:38 | |
| *** kjackal has joined #openstack-infra | 05:38 | |
| *** udesale has joined #openstack-infra | 05:41 | |
| *** markvoelker has quit IRC | 05:52 | |
| *** quiquell|off is now known as quiquell | 05:53 | |
| *** ykarel is now known as ykarel|afk | 05:57 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack-infra/project-config master: Normalize projects.yaml https://review.openstack.org/607470 | 06:02 |
| *** longkb has joined #openstack-infra | 06:06 | |
| *** vivsoni has joined #openstack-infra | 06:10 | |
| *** dims has quit IRC | 06:24 | |
| *** e0ne has joined #openstack-infra | 06:24 | |
| *** dims has joined #openstack-infra | 06:26 | |
| *** ykarel_ has joined #openstack-infra | 06:27 | |
| *** ykarel|afk has quit IRC | 06:29 | |
| *** kjackal has quit IRC | 06:32 | |
| *** dims has quit IRC | 06:34 | |
| *** vivsoni has quit IRC | 06:35 | |
| *** dims has joined #openstack-infra | 06:35 | |
| ajo | fungi: thanks for the wiki cleanups for the OsLo & Os-log-merger pages, I didn't find the delete button. | 06:38 |
| *** dpawlik has joined #openstack-infra | 06:39 | |
| openstackgerrit | Merged openstack-infra/project-config master: Normalize projects.yaml https://review.openstack.org/607470 | 06:42 |
| *** aojea has joined #openstack-infra | 06:43 | |
| *** yamamoto has quit IRC | 06:51 | |
| *** yamamoto has joined #openstack-infra | 06:51 | |
| *** pcaruana has joined #openstack-infra | 06:51 | |
| *** vivsoni has joined #openstack-infra | 06:53 | |
| *** kjackal has joined #openstack-infra | 07:01 | |
| *** tosky has joined #openstack-infra | 07:03 | |
| *** ginopc has joined #openstack-infra | 07:04 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: Revert "Revert "web: rewrite interface in react"" https://review.openstack.org/607479 | 07:05 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: Revert "Revert "web: rewrite interface in react"" https://review.openstack.org/607479 | 07:10 |
| *** ykarel__ has joined #openstack-infra | 07:13 | |
| *** ykarel_ has quit IRC | 07:15 | |
| *** jroll has quit IRC | 07:19 | |
| *** Tengu has quit IRC | 07:19 | |
| *** logan- has quit IRC | 07:19 | |
| *** quite has quit IRC | 07:19 | |
| *** dmellado has quit IRC | 07:19 | |
| *** karimsye has quit IRC | 07:19 | |
| *** colonwq has quit IRC | 07:19 | |
| *** ajo has quit IRC | 07:19 | |
| *** odyssey4me has quit IRC | 07:19 | |
| *** vdrok has quit IRC | 07:19 | |
| *** bgmccollum has quit IRC | 07:19 | |
| *** ykarel__ is now known as ykarel | 07:20 | |
| *** olivierb has joined #openstack-infra | 07:24 | |
| *** quite has joined #openstack-infra | 07:24 | |
| *** Tengu has joined #openstack-infra | 07:32 | |
| *** jroll has joined #openstack-infra | 07:32 | |
| *** logan- has joined #openstack-infra | 07:32 | |
| *** colonwq has joined #openstack-infra | 07:32 | |
| *** dmellado has joined #openstack-infra | 07:32 | |
| *** karimsye has joined #openstack-infra | 07:32 | |
| *** ajo has joined #openstack-infra | 07:32 | |
| *** vdrok has joined #openstack-infra | 07:32 | |
| *** odyssey4me has joined #openstack-infra | 07:32 | |
| *** bgmccollum has joined #openstack-infra | 07:32 | |
| *** jpich has joined #openstack-infra | 07:35 | |
| *** aidin has joined #openstack-infra | 07:36 | |
| *** alexchadin has joined #openstack-infra | 07:41 | |
| *** aidin has quit IRC | 07:41 | |
| *** ykarel has quit IRC | 07:43 | |
| *** ykarel has joined #openstack-infra | 07:45 | |
| *** jpena|off is now known as jpena | 07:51 | |
| *** rcernin has quit IRC | 07:55 | |
| hwoarang | fungi: good day. Is the node still held for https://review.openstack.org/#/c/599521/ (bifrost-integration-dhcp-opensuse-150) ? | 07:57 |
| *** gwhaley has joined #openstack-infra | 08:00 | |
| ttx | etherpad.openstack.org is 503 | 08:02 |
| *** gwhaley has quit IRC | 08:03 | |
| *** ykarel_ has joined #openstack-infra | 08:07 | |
| *** vivsoni has quit IRC | 08:07 | |
| *** ykarel_ has quit IRC | 08:07 | |
| *** ykarel_ has joined #openstack-infra | 08:09 | |
| *** shardy has joined #openstack-infra | 08:11 | |
| *** ykarel has quit IRC | 08:13 | |
| *** jroll has quit IRC | 08:13 | |
| *** Tengu has quit IRC | 08:13 | |
| *** logan- has quit IRC | 08:13 | |
| *** dmellado has quit IRC | 08:13 | |
| *** karimsye has quit IRC | 08:13 | |
| *** colonwq has quit IRC | 08:13 | |
| *** ajo has quit IRC | 08:13 | |
| *** odyssey4me has quit IRC | 08:13 | |
| *** vdrok has quit IRC | 08:13 | |
| *** bgmccollum has quit IRC | 08:13 | |
| *** jlibosva has joined #openstack-infra | 08:16 | |
| openstackgerrit | Sorin Sbarnea proposed openstack-infra/openstack-zuul-jobs master: Silence syncronize on post to avoid console spam https://review.openstack.org/598599 | 08:17 |
| jlibosva | hi, can somebody point me at some jenkins job that builds the tarballs once project gets a new release tag, pretty please? I couldn't find it anywhere | 08:17 |
| *** markvoelker has joined #openstack-infra | 08:18 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Cancelling nova-notification subteam meeting https://review.openstack.org/607314 | 08:18 |
| *** vivsoni has joined #openstack-infra | 08:18 | |
| chandankumar | jlibosva: http://zuul.openstack.org/builds.html?job_name=publish-openstack-python-branch-tarball and http://zuul.openstack.org/builds.html?job_name=python-upload-pypi these two are related to tarballs | 08:21 |
| *** jtomasek_ has joined #openstack-infra | 08:21 | |
| jlibosva | chandankumar++ thanks a lot! | 08:21 |
| *** alexchadin has quit IRC | 08:21 | |
| *** jtomasek has quit IRC | 08:23 | |
| *** Tengu has joined #openstack-infra | 08:27 | |
| *** jroll has joined #openstack-infra | 08:27 | |
| *** logan- has joined #openstack-infra | 08:27 | |
| *** colonwq has joined #openstack-infra | 08:27 | |
| *** dmellado has joined #openstack-infra | 08:27 | |
| *** karimsye has joined #openstack-infra | 08:27 | |
| *** ajo has joined #openstack-infra | 08:27 | |
| *** vdrok has joined #openstack-infra | 08:27 | |
| *** odyssey4me has joined #openstack-infra | 08:27 | |
| *** bgmccollum has joined #openstack-infra | 08:27 | |
| jlibosva | chandankumar: as networking-ansible is not an official OS project I manually tagged our stable branch but I see it was not picked by the job. Do you have any ideas why could that happen or what did I do wrong? | 08:28 |
| *** roman_g has joined #openstack-infra | 08:30 | |
| *** derekh has joined #openstack-infra | 08:33 | |
| *** electrofelix has joined #openstack-infra | 08:34 | |
| *** jtomasek has joined #openstack-infra | 08:34 | |
| chandankumar | jlibosva: may be you need to do something like this http://git.openstack.org/cgit/openstack/tripleo-ansible/tree/zuul.d/layout.yaml#n10 | 08:35 |
| chandankumar | jlibosva: for creating a new release or tag we generally send a change to openstack/releases | 08:36 |
| *** jtomasek_ has quit IRC | 08:36 | |
| chandankumar | jlibosva: I think from there it is automatically done | 08:36 |
| jlibosva | chandankumar: it worked before with 1.0.1 tag but now we don't have last two tags | 08:36 |
| *** gfidente has quit IRC | 08:36 | |
| jlibosva | chandankumar: yeah, we cannot use openstack/release as we're not official | 08:36 |
| jlibosva | chandankumar: so I was told to do all the steps manually but before, we got automatically 1.0.1 release but now we dodn't | 08:37 |
| chandankumar | jlibosva: may be AJaeger ^^ can help on that | 08:37 |
| AJaeger | jlibosva: which repo? | 08:38 |
| jlibosva | AJaeger: it's networking-ansible | 08:38 |
| jlibosva | chandankumar: thanks! | 08:38 |
| AJaeger | jlibosva: you have publish-to-pypi-neutron configured for the repo, so that should run. Which tag did you use? | 08:39 |
| jlibosva | AJaeger: it's 1.0.3 | 08:40 |
| jlibosva | AJaeger: also 1.0.2 is missing so I assume I did something wrong | 08:40 |
| AJaeger | http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/projects.yaml#n5856 is the line | 08:40 |
| jlibosva | AJaeger: I pushed the tag to gerrit and then pushed the tarball to PyPI | 08:40 |
| AJaeger | jlibosva: check http://lists.openstack.org/pipermail/release-job-failures/ | 08:41 |
| *** Emine has joined #openstack-infra | 08:41 | |
| jlibosva | AJaeger: it works for master and stable branches but not for tags | 08:41 |
| AJaeger | jlibosva: I see http://lists.openstack.org/pipermail/release-job-failures/2018-October/000961.html | 08:41 |
| jlibosva | oh, there we go | 08:41 |
| jlibosva | there are failures | 08:41 |
| AJaeger | jlibosva: http://logs.openstack.org/c1/c15e907e95ab5404ed8799cdb60783bcf5947709/release/release-openstack-python/d23159a/job-output.txt.gz#_2018-10-02_08_18_54_556274 - not sure where | 08:41 |
| jlibosva | 400 Client Error: File already exists. | 08:42 |
| AJaeger | jlibosva: https://pypi.org/project/networking-ansible/ has 1.0.3 - did you upload manually? | 08:42 |
| jlibosva | AJaeger: so the job uploads it by itself? | 08:42 |
| jlibosva | AJaeger: yes, I did that manually | 08:42 |
| AJaeger | jlibosva: yes, sure. That's the main purpose of using the job ;) | 08:43 |
| jlibosva | AJaeger: that's cool. so I did an extra manual step. yeah, I thought when I'm not using the openstack/release, I need to do all the stuff manually :-/ | 08:43 |
| AJaeger | jlibosva: please read https://docs.openstack.org/infra/manual/drivers.html#tagging-a-release and if anything is unclear, send an update... | 08:43 |
| jlibosva | AJaeger: thank you very much for your help! | 08:44 |
| jlibosva | now it makes sense | 08:44 |
| AJaeger | jlibosva: you only need to tag yourself - the uploading etc. is the same process whether you use openstack/release or not. | 08:44 |
| jlibosva | "After a tag is created the release build will generate a source code tarball and may publish it to a repository such as PyPI." - I will get my eyes checked | 08:45 |
| AJaeger | ;) | 08:45 |
| *** roman_g_ has joined #openstack-infra | 08:47 | |
| tinwood | fungi, excellent, thanks very much for doing that. Much appreciated! | 08:47 |
| tinwood | AJaeger, thanks also with your help, too. | 08:48 |
| *** roman_g has quit IRC | 08:51 | |
| *** jroll has quit IRC | 08:51 | |
| *** Tengu has quit IRC | 08:51 | |
| *** logan- has quit IRC | 08:51 | |
| *** dmellado has quit IRC | 08:51 | |
| *** karimsye has quit IRC | 08:51 | |
| *** colonwq has quit IRC | 08:51 | |
| *** ajo has quit IRC | 08:51 | |
| *** odyssey4me has quit IRC | 08:51 | |
| *** vdrok has quit IRC | 08:51 | |
| *** bgmccollum has quit IRC | 08:51 | |
| *** markvoelker has quit IRC | 08:51 | |
| *** jesusaur has quit IRC | 08:53 | |
| *** ykarel_ is now known as ykarel|lunch | 08:55 | |
| *** derekh has quit IRC | 08:58 | |
| *** jtomasek has quit IRC | 08:59 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Update release jobs for driverlog and tempest https://review.openstack.org/607501 | 09:01 |
| AJaeger | tinwood: you're welcome. | 09:05 |
| *** Tengu has joined #openstack-infra | 09:05 | |
| *** jroll has joined #openstack-infra | 09:05 | |
| *** logan- has joined #openstack-infra | 09:05 | |
| *** colonwq has joined #openstack-infra | 09:05 | |
| *** dmellado has joined #openstack-infra | 09:05 | |
| *** karimsye has joined #openstack-infra | 09:05 | |
| *** ajo has joined #openstack-infra | 09:05 | |
| *** vdrok has joined #openstack-infra | 09:05 | |
| *** odyssey4me has joined #openstack-infra | 09:05 | |
| *** bgmccollum has joined #openstack-infra | 09:05 | |
| *** aidin has joined #openstack-infra | 09:08 | |
| *** Emine has quit IRC | 09:12 | |
| *** alexchadin has joined #openstack-infra | 09:13 | |
| *** priteau has joined #openstack-infra | 09:13 | |
| *** jtomasek has joined #openstack-infra | 09:15 | |
| tobberydberg | etherpad.openstack.org down? | 09:16 |
| ssbarnea | AJaeger fungi : thanks for the help with coats, it got in and apparently working well. One more nit: https://review.openstack.org/#/admin/groups/1952,members and https://review.openstack.org/#/admin/groups/1953,members need ssbarnea and mpeterson added to them, also a good idea to add openstack-infra and tripleo-quickstart-core groups for fallback for the first one. | 09:19 |
| AJaeger | ssbarnea: best wait until fungi is around - and we only add you to it and you can do the rest. I don't have those permissions. | 09:21 |
| AJaeger | infra-root, etherpad.openstack.org is down ;( | 09:21 |
| ssbarnea | AJaeger: sure, nopb. | 09:21 |
| *** Emine has joined #openstack-infra | 09:25 | |
| mpeterson | /away lunch | 09:40 |
| *** alexchadin has quit IRC | 09:46 | |
| aojea | one question, how can I do to recheck the gate pipeline? | 09:59 |
| aojea | per example, doing recheck https://review.openstack.org/#/c/604731/ seems to kick only the check pipeline | 09:59 |
| *** ykarel|lunch is now known as ykarel | 10:05 | |
| *** yamamoto has quit IRC | 10:05 | |
| *** yamamoto has joined #openstack-infra | 10:06 | |
| *** Emine has quit IRC | 10:07 | |
| AJaeger | aojea: you cannot - a "recheck" goes first to check, then through gate. We had broken patches that blocked the pipeline and just passed by accident first time - or might break with current repo state | 10:10 |
| *** yamamoto has quit IRC | 10:10 | |
| *** kukacz has quit IRC | 10:12 | |
| *** kukacz has joined #openstack-infra | 10:13 | |
| *** yamamoto has joined #openstack-infra | 10:14 | |
| *** aidin has quit IRC | 10:19 | |
| *** derekh has joined #openstack-infra | 10:22 | |
| *** aidin has joined #openstack-infra | 10:24 | |
| *** udesale has quit IRC | 10:26 | |
| *** jtomasek has quit IRC | 10:28 | |
| aojea | AJaeger: thx | 10:33 |
| *** aidin has quit IRC | 10:34 | |
| *** gfidente has joined #openstack-infra | 10:35 | |
| openstackgerrit | Natal Ngétal proposed openstack/gertty master: [Documentation] Change the fedora install. https://review.openstack.org/607527 | 10:42 |
| *** markvoelker has joined #openstack-infra | 10:49 | |
| *** kjackal has quit IRC | 10:58 | |
| openstackgerrit | Merged openstack/diskimage-builder master: allow building non-gentoo images on gentoo hosts https://review.openstack.org/561002 | 11:08 |
| *** jtomasek has joined #openstack-infra | 11:19 | |
| *** yamamoto has quit IRC | 11:19 | |
| *** markvoelker has quit IRC | 11:22 | |
| *** yamamoto has joined #openstack-infra | 11:24 | |
| *** kjackal has joined #openstack-infra | 11:25 | |
| *** gary_perkins has joined #openstack-infra | 11:31 | |
| *** quiquell is now known as quiquell|lunch | 11:37 | |
| *** yamamoto has quit IRC | 11:38 | |
| *** jpena is now known as jpena|lunch | 11:38 | |
| *** yamamoto has joined #openstack-infra | 11:39 | |
| *** yamamoto has quit IRC | 11:44 | |
| *** rh-jelabarre has joined #openstack-infra | 11:44 | |
| *** agopi|off is now known as agopi | 11:50 | |
| *** jtomasek has quit IRC | 11:57 | |
| *** ansmith has quit IRC | 12:02 | |
| *** trown|outtypewww is now known as trown | 12:03 | |
| *** udesale has joined #openstack-infra | 12:03 | |
| *** e0ne has quit IRC | 12:04 | |
| *** jcoufal has joined #openstack-infra | 12:05 | |
| frickler | infra-root: etherpad.o.o had disk full, I cleaned up a bit of the backups as temporary measure. maybe a larger disk or less retention for the backups would be good? http://paste.openstack.org/show/731355/ | 12:08 |
| ttx | etherpad is back | 12:13 |
| ttx | thanks frickler | 12:13 |
| *** Emine has joined #openstack-infra | 12:14 | |
| *** e0ne has joined #openstack-infra | 12:16 | |
| *** dave-mccowan has joined #openstack-infra | 12:19 | |
| *** longkb has quit IRC | 12:20 | |
| *** yamamoto has joined #openstack-infra | 12:20 | |
| *** ssbarnea|bkp2 has quit IRC | 12:20 | |
| *** longkb has joined #openstack-infra | 12:21 | |
| *** yamamoto has quit IRC | 12:22 | |
| *** yamamoto has joined #openstack-infra | 12:22 | |
| *** quiquell|lunch is now known as quiquell | 12:24 | |
| *** longkb has quit IRC | 12:27 | |
| *** rlandy has joined #openstack-infra | 12:32 | |
| *** ykarel has quit IRC | 12:34 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Move prepare-zanata-client to o-z-j https://review.openstack.org/525760 | 12:36 |
| mordred | morning AJaeger - anything I can help with before getting consumed by ansiblefest again? | 12:37 |
| *** ykarel has joined #openstack-infra | 12:37 | |
| *** jtomasek has joined #openstack-infra | 12:39 | |
| *** jpena|lunch is now known as jpena | 12:41 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Move prepare-zanata-client to o-z-j https://review.openstack.org/525760 | 12:42 |
| AJaeger | mordred: nothing burning that I'm aware off - thanks for asking. | 12:42 |
| AJaeger | mordred: if you or a config-core have time for reviews - https://review.openstack.org/605644 https://review.openstack.org/583870 https://review.openstack.org/607059 and https://review.openstack.org/599890 are ready - but nothing urgent | 12:44 |
| AJaeger | as well as https://review.openstack.org/607348 | 12:44 |
| AJaeger | dhellmann: are you around already? | 12:45 |
| dhellmann | AJaeger : hi, yes | 12:45 |
| dhellmann | just settling in | 12:45 |
| AJaeger | dhellmann: good morning - I propose to go forward with heat. So, abandon https://review.openstack.org/607348 and merge https://review.openstack.org/607348 - what do you think? | 12:46 |
| * dhellmann looks | 12:46 | |
| AJaeger | dhellmann: no urgency, just wanted to put this on your list for consideration... | 12:46 |
| dhellmann | AJaeger : those 2 links are to the same patch | 12:46 |
| AJaeger | https://review.openstack.org/597289 is the other one - the project-config | 12:47 |
| dhellmann | so which one do you want to abandon? | 12:47 |
| AJaeger | dhellmann: and other topic is monasca, a single repo is missing that had nothing merged for 18 months. We could split the project-config change in two and give team 2 weeks to sort that repo out (that what witek told them) | 12:48 |
| AJaeger | abandon the stable/ocata one - https://review.openstack.org/597272 | 12:48 |
| AJaeger | dhellmann: sorry, I screwed up pasting completely before ;( | 12:48 |
| dhellmann | no worries, I've got it now | 12:48 |
| dhellmann | yes, if they're going to close the ocata branch we should abandon that one | 12:49 |
| dhellmann | however, I think it would be good to get them to confirm that | 12:49 |
| *** kaiokmo has quit IRC | 12:49 | |
| AJaeger | dhellmann: fine with me - will you try to catch ricolin? | 12:49 |
| mordred | AJaeger: cool, reviewing | 12:50 |
| dhellmann | yeah, I will do that today | 12:50 |
| dhellmann | AJaeger : so about monasca? | 12:50 |
| dhellmann | splitting the monasca project-config change seems like the right approach. let me look at what's open | 12:50 |
| clarkb | infra-root we can reduce the number of locally held etherpad backups | 12:50 |
| clarkb | they are there for convenience since bup should dedup thrm remotely | 12:51 |
| dhellmann | AJaeger : I agree with splitting that up so we can keep moving. Do you want to do that, or should I put it on my list? | 12:51 |
| AJaeger | dhellmann: yeah, talking with witek he said that he gave the openstack/monasca-analytics cores two weeks to sort it out - but I would rather get those out of project-config now. | 12:51 |
| clarkb | but not necessary if causing disk issues | 12:51 |
| AJaeger | dhellmann: if you have time today, please do it. I will step away for a bit after our discussion. | 12:51 |
| dhellmann | AJaeger : ok, I'll put it on my list | 12:52 |
| AJaeger | thanks, dhellmann. Hope the remaining teams move forward - only a handful and some are very slow or have failing jobs that they ignore. Some also might need help from stable team in reviewing the rest... ;( | 12:53 |
| dhellmann | I'll start an email thread for the teams that are lagging | 12:54 |
| AJaeger | thanks | 12:54 |
| AJaeger | dhellmann: and a quick review request for openstack-manuals - we have right now a broken tree since we removed some content: https://review.openstack.org/607461 , please | 12:55 |
| * AJaeger will be back later... | 12:55 | |
| dhellmann | AJaeger : +2a | 12:56 |
| *** kgiusti has joined #openstack-infra | 12:57 | |
| *** boden has joined #openstack-infra | 12:58 | |
| *** kaiokmo has joined #openstack-infra | 12:58 | |
| *** sthussey has joined #openstack-infra | 13:00 | |
| *** udesale has quit IRC | 13:05 | |
| openstackgerrit | Merged openstack-infra/project-config master: Enable storyboard for os-log-merger https://review.openstack.org/605644 | 13:07 |
| openstackgerrit | Merged openstack-infra/project-config master: Add tempest-slow to Neutron failure rate dashboard https://review.openstack.org/583870 | 13:07 |
| openstackgerrit | Merged openstack-infra/project-config master: Add release job for os-ken https://review.openstack.org/607059 | 13:07 |
| openstackgerrit | Merged openstack-infra/project-config master: Add api-ref job to ironic-inspector https://review.openstack.org/599890 | 13:07 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: remove job settings for monasca repositories https://review.openstack.org/595744 | 13:11 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: remove job settings for openstack/monasca-analytics repo https://review.openstack.org/607560 | 13:11 |
| *** ansmith has joined #openstack-infra | 13:12 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul-jobs master: WIP: Add role to install kubernetes https://review.openstack.org/605823 | 13:14 |
| fungi | ajo: your wiki account wasn't verified at the time (i verified it after patrolling your edits) and only verified users can delete pages. it was no trouble | 13:14 |
| fungi | hwoarang: yes, there is a node held just waiting for you to tell us what ssh key you want granted access to it | 13:14 |
| ajo | fungi: ah, thanks for the patrolling then, I know how exhausting spammers can be | 13:15 |
| hwoarang | fungi: i will pm it to you | 13:15 |
| fungi | ssbarnea: i've added you as the initial member of coats-core and coats-release | 13:17 |
| *** ssbarnea|bkp2 has joined #openstack-infra | 13:17 | |
| fungi | frickler: thanks for cleaning up etherpad.o.o disk space; clarkb: i agree reducing database dump retention on the filesystem is a reasonable choice | 13:18 |
| openstackgerrit | Miguel Angel Ajo proposed openstack-infra/project-config master: Remove os-log-merger noop job. https://review.openstack.org/607568 | 13:21 |
| fungi | hwoarang: okay, i've added your key. you should now be able to ssh root@162.209.77.227 | 13:21 |
| fungi | let me know if it doesn't work, or if you need anything else | 13:21 |
| hwoarang | fungi: works fine. thank you very much | 13:22 |
| fungi | it's my pleasure | 13:22 |
| *** eharney has joined #openstack-infra | 13:23 | |
| *** abregman has joined #openstack-infra | 13:24 | |
| abregman | hey everyone. question - how can I set up a job that runs tempest twice? | 13:24 |
| clarkb | abregman: you'll probably need to override the run playbook for the base tempest job to run tempest multiple times. | 13:29 |
| clarkb | abregman: is there something specific you are trying to achieve by running tempest twice? | 13:29 |
| tosky | clarkb: retesting after changing the setup | 13:30 |
| hwoarang | fungi: i am done with the node. thank you again | 13:32 |
| fungi | hwoarang: so okay for me to delete it now? | 13:33 |
| hwoarang | yep | 13:34 |
| fungi | done, thanks for the quick heads-up! | 13:34 |
| *** aidin has joined #openstack-infra | 13:34 | |
| abregman | clarkb: yes. I would like to set up resources before OpenStack upgrade and verify them after the upgrade | 13:36 |
| abregman | clarkb: so basically I need to execute tempest pre-upgrade tests and post-upgrade tests. Is there a documentation I can follow that will help me to achieve that? | 13:37 |
| fungi | abregman: and the initial run of tempest is to make sure it was working before the upgrade? | 13:37 |
| clarkb | abregman: for that you may want to look at grenade. It already understands that process of spin up "old cloud", test it and set up resources, upgrade to "new cloud" then run tests again | 13:37 |
| fungi | this is sounding a _lot_ like grenade | 13:37 |
| *** mriedem has joined #openstack-infra | 13:37 | |
| abregman | fungi: for making sure it was working + set up some resources for post-testing | 13:38 |
| abregman | clarkb: thanks, I'll have a look :) | 13:38 |
| *** aidin has quit IRC | 13:38 | |
| fungi | using tempest to set up those resources? grenade has its own utility to do the resource setup prior to upgrade but i forget what it's called | 13:39 |
| *** ykarel is now known as ykarel|away | 13:39 | |
| abregman | thanks. I'll take a deep look at grenade | 13:42 |
| *** agopi has quit IRC | 13:42 | |
| *** ykarel_ has joined #openstack-infra | 13:43 | |
| fungi | grenade's main purpose is to set up a devstack environment, build some test resources in it, make changes to the environment (e.g. upgrade things) and then you run tempest against it to verify it still works afterward | 13:44 |
| fungi | this is how most of the integrated openstack projects are managing their upgrade testing | 13:45 |
| *** yamamoto has quit IRC | 13:45 | |
| *** ykarel|away has quit IRC | 13:45 | |
| *** yamamoto has joined #openstack-infra | 13:45 | |
| abregman | fungi: can I run also tests before upgrade? for example I would like to see that a ping test works from before upgrade until upgrade is done | 13:45 |
| fungi | i believe it includes a stage which verifies the initial environment to catch bad setup | 13:46 |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Retain fewer etherpad database backups https://review.openstack.org/607575 | 13:46 |
| clarkb | fungi: frickler ^ | 13:46 |
| abregman | fungi: also, where can I find example how set up grenade job? documentation is..minimal. unless I'm looking at the wrong page | 13:46 |
| fungi | abregman: but #openstack-qa is probably a better place to get answers about grenade, devstack and tempest | 13:46 |
| abregman | fungi: thank you. I'll try to ask there | 13:46 |
| fungi | since those projects aren't developed by the infra team, but rather the qa team | 13:47 |
| fungi | thanks clarkb! | 13:47 |
| *** kaiokmo has quit IRC | 13:47 | |
| *** ssbarnea|bkp2 has quit IRC | 13:49 | |
| *** yamamoto has quit IRC | 13:50 | |
| *** tosky__ has joined #openstack-infra | 13:51 | |
| *** tosky has quit IRC | 13:51 | |
| *** tosky__ is now known as tosky | 13:52 | |
| *** kaiokmo has joined #openstack-infra | 13:52 | |
| *** jamesmcarthur has joined #openstack-infra | 13:57 | |
| *** aojeagarcia has joined #openstack-infra | 14:02 | |
| *** olivierb_ has joined #openstack-infra | 14:02 | |
| *** rpittau_ has joined #openstack-infra | 14:02 | |
| *** rkukura_ has joined #openstack-infra | 14:02 | |
| clarkb | hrm new stackviz failures. I wonder if the image hasn't updated everywhere or if the install process is different than we expected | 14:02 |
| *** aojea has quit IRC | 14:03 | |
| *** olivierb has quit IRC | 14:03 | |
| *** rpittau has quit IRC | 14:03 | |
| *** abregman has quit IRC | 14:03 | |
| *** adriant has quit IRC | 14:03 | |
| dtroyer | I need some help debugging a post pipeline job (stx publish to afs specifically) failure. I managed to grab this log from the console output, so question 1 is how do I find that after-the-fact? http://paste.openstack.org/show/731371/ The obvious failure is at 2018-10-03 02:43:04.099496 | 14:04 |
| *** adriant has joined #openstack-infra | 14:04 | |
| *** niedbalski has quit IRC | 14:04 | |
| *** SergeyLukjanov has quit IRC | 14:04 | |
| *** agopi has joined #openstack-infra | 14:04 | |
| *** bobh has joined #openstack-infra | 14:04 | |
| clarkb | dtroyer: https://zuul.openstack.org/builds.html is the gateway to all the post job logs | 14:05 |
| dtroyer | cool, thanks | 14:05 |
| dtroyer | the failing job is publish-stx-docs | 14:05 |
| dtroyer | in https://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/starlingx-jobs.yam | 14:05 |
| clarkb | dtroyer: putting that job name in the job filter on the link above gets me http://logs.openstack.org/b0/b003280698868b43cc0b6fc31db8278d1c4c1c57/post/publish-stx-docs/6a07deb/ as the url | 14:06 |
| *** Bhujay has quit IRC | 14:06 | |
| *** rkukura has quit IRC | 14:06 | |
| *** rkukura_ is now known as rkukura | 14:06 | |
| *** SergeyLukjanov has joined #openstack-infra | 14:06 | |
| clarkb | dtroyer: fungi http://logs.openstack.org/b0/b003280698868b43cc0b6fc31db8278d1c4c1c57/post/publish-stx-docs/6a07deb/ara-report/result/265b7ff0-7c55-42b9-bf09-51ea7f7fa310/ is what ara says it he error. permission denied | 14:06 |
| *** annp has quit IRC | 14:07 | |
| *** Bhujay has joined #openstack-infra | 14:07 | |
| clarkb | we may need to set the zuul afs user as write perms on that new volume path? | 14:07 |
| fungi | does the www subdir need to be precreated? | 14:07 |
| fungi | though, yes, i haven't modified anything about that volume so no idea what all was done to prep it | 14:08 |
| *** yamamoto has joined #openstack-infra | 14:08 | |
| clarkb | fungi: I did things to make the volume and put it under the tree but didn't assign any special perms that I can remember so guessing that is the issue | 14:08 |
| fungi | no worries, i'll figure out how to confirm/change that | 14:09 |
| clarkb | tyty, we all owe you a beverage in berlin! | 14:10 |
| dtroyer | many beverage | 14:10 |
| fungi | heck, i owe all of you for going to ansiblefest so i wouldn't need to ;) | 14:10 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Move docs/cover job for monasca-{transform,ui} in-repo https://review.openstack.org/607582 | 14:13 |
| AJaeger | dhellmann: want to WIP https://review.openstack.org/607560 for now? Or make it Depends-On the import change? | 14:15 |
| *** Bhujay has quit IRC | 14:16 | |
| *** quiquell is now known as quiquell|off | 14:17 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove noop jobs for ara-{plugin,client} https://review.openstack.org/607584 | 14:20 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove noop jobs for ara-{plugins,clients} https://review.openstack.org/607584 | 14:20 |
| AJaeger | dmsimard: ^ | 14:21 |
| dmsimard | AJaeger: thanks, I was going to be doing that soon | 14:21 |
| *** chandankumar is now known as chkumar|pyconind | 14:21 | |
| dmsimard | just started ramping up the projects .. | 14:21 |
| AJaeger | dmsimard: ah, just noticed it looking at zuul.o.o status page... | 14:21 |
| *** armax has joined #openstack-infra | 14:21 | |
| dmsimard | AJaeger: added a comment | 14:22 |
| *** munimeha1 has joined #openstack-infra | 14:23 | |
| AJaeger | dmsimard: indeed - I just check for .zuul.yaml, not zuul.d/... | 14:23 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove noop jobs for ara-{plugins,clients,server},os-log-merger https://review.openstack.org/607584 | 14:24 |
| AJaeger | dmsimard: updated and added one more ^ | 14:24 |
| dmsimard | AJaeger: thanks you rock | 14:24 |
| openstackgerrit | Nate Johnston proposed openstack-infra/project-config master: Align neutron grafana with actual jobs being run https://review.openstack.org/607586 | 14:25 |
| openstackgerrit | Nate Johnston proposed openstack-infra/project-config master: Align neutron grafana with actual jobs being run https://review.openstack.org/607586 | 14:27 |
| *** dpawlik has quit IRC | 14:27 | |
| *** yamamoto has quit IRC | 14:40 | |
| *** yamamoto has joined #openstack-infra | 14:41 | |
| *** yamamoto has quit IRC | 14:42 | |
| *** yamamoto has joined #openstack-infra | 14:43 | |
| *** Shrews has quit IRC | 14:45 | |
| *** yamamoto has quit IRC | 14:48 | |
| fungi | clarkb: okay, so i guess this implies that i need to grant write access to the docs principal and read access to system:anyuser http://paste.openstack.org/show/731380/ | 14:48 |
| *** slaweq has quit IRC | 14:48 | |
| fungi | there also seems to be a zuul-website principal which isn't used because it's not listed on the www subdir acl which is the only content of that volume | 14:49 |
| fungi | do we need a starlingx-website principal added for completeness? | 14:49 |
| *** ssbarnea|bkp2 has joined #openstack-infra | 14:51 | |
| dtroyer | fungi: if you need that to continue, use me for now. I can find out from the docs team who they would want for that long term if needed | 14:51 |
| *** ssbarnea|bkp2 has quit IRC | 14:51 | |
| *** ssbarnea|bkp2 has joined #openstack-infra | 14:52 | |
| fungi | dtroyer: nah, these are names of what amount to role accounts for our automation in kerberos | 14:52 |
| dtroyer | kk | 14:53 |
| *** Shrews has joined #openstack-infra | 14:53 | |
| fungi | i'm just battling with connection timeouts on the fs setacl command at the moment getting the minimum permission added for zuul to be able to push content and apache to be able to serve it | 14:53 |
| *** ykarel_ has quit IRC | 14:54 | |
| *** bobh has quit IRC | 14:54 | |
| fungi | here's the updated acls for that volume now: http://paste.openstack.org/show/731383/ | 14:54 |
| fungi | i'll try reenqueuing that commit into post next and see if it still breaks | 14:54 |
| dtroyer | thanks | 14:55 |
| *** slaweq has joined #openstack-infra | 14:56 | |
| fungi | it's awaiting a node assignment for the publish-stx-docs job now | 14:56 |
| *** Emine has quit IRC | 14:58 | |
| *** ykarel has joined #openstack-infra | 15:07 | |
| *** eharney has quit IRC | 15:09 | |
| *** jtomasek has quit IRC | 15:10 | |
| *** jtomasek has joined #openstack-infra | 15:10 | |
| *** dpawlik has joined #openstack-infra | 15:19 | |
| fungi | looks like it published successfully, but still getting a 403 forbidden from apache so i need to troubleshoot the vhost next | 15:19 |
| fungi | might still be the public read perms | 15:19 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: remove job settings for openstack/monasca-analytics repo https://review.openstack.org/607560 | 15:21 |
| *** dpawlik has quit IRC | 15:24 | |
| fungi | oh, i bet the vos release cron just hasn't fired yet | 15:24 |
| *** eharney has joined #openstack-infra | 15:24 | |
| *** fuentess has joined #openstack-infra | 15:26 | |
| *** yamamoto has joined #openstack-infra | 15:28 | |
| *** rf0lc0 is now known as rfolco_doc | 15:28 | |
| *** ramishra has quit IRC | 15:30 | |
| *** bobh has joined #openstack-infra | 15:31 | |
| *** pcaruana has quit IRC | 15:33 | |
| rpioso | I'm giving gertty a test drive. Can in-line comments be displayed in a review's main screen as they are in Gerrit's web interface? | 15:39 |
| *** hamzy has quit IRC | 15:40 | |
| fungi | like how they're embedded in the review comment blocks? i don't think it supports that yet | 15:42 |
| fungi | it's more like the old gerrit change screen (circa 2.11 and before) rather than the new change screen | 15:42 |
| *** hamzy has joined #openstack-infra | 15:43 | |
| fungi | owing in part to the fact that the gertty author really doesn't like the new change screen | 15:43 |
| fungi | nor do a lot of us | 15:43 |
| rpioso | fungi: Yes, like how they're embedded in the review comment blocks. I'm not familiar with the old gerrit change screen. | 15:46 |
| rpioso | fungi: I can pull the in-line comments up on a patch set by patch set basis by electing to diff each one. Is there any other way? | 15:48 |
| fungi | rpioso: yes, in the patch sets list you can choose the patch-specific diff | 15:54 |
| fungi | though i usually just hit d to look at the latest diff and then p to choose start/end patch set | 15:54 |
| fungi | rather than picking the patch set out of the change view | 15:55 |
| rpioso | fungi: Thank you! | 15:57 |
| *** gfidente is now known as gfidente|afk | 15:57 | |
| *** _erlon_ has joined #openstack-infra | 15:59 | |
| rpioso | fungi: Is it possible to reply to an in-line comment? | 16:01 |
| dtroyer | fungi: /me in meeting overlap this morning… I don't see anything in http://files.openstack.org/project/, should I? | 16:01 |
| *** bobh has quit IRC | 16:02 | |
| fungi | rpioso: i don't think gertty makes use of the api options to perform threading of inline comments, though that and the in-review-comment display of inline comments and context are probably both improvements worth discussing with corvus when he's not in the middle of ansiblefest | 16:05 |
| *** gyee has joined #openstack-infra | 16:06 | |
| fungi | dtroyer: it looks like the content is in /afs/.openstack.org/project/starlingx.io/www/ but has not been replicated to the read-only volumes yet. i'm looking into why (probably either the replicas weren't set up or we need to add that volume to the vos release cron) | 16:06 |
| rpioso | fungi: Gotcha. I found this -- https://openstack.nimeyo.com/61975/openstack-dev-gertty-replying-to-inline-comments | 16:06 |
| clarkb | fungi: there are no replicas like zuul-ci.org iirc | 16:07 |
| rpioso | Sounds like it's not threaded, though. | 16:07 |
| clarkb | fungi: should use the RW volume like zuul I think | 16:07 |
| *** tpsilva has joined #openstack-infra | 16:08 | |
| fungi | clarkb: the zuul-ci.org also has DocumentRoot /afs/openstack.org//project/zuul-ci.org/www | 16:09 |
| fungi | not .openstack.org | 16:09 |
| *** slaweq has quit IRC | 16:09 | |
| *** ykarel is now known as ykarel|away | 16:09 | |
| fungi | so something is causing it to show up in the read-only path | 16:10 |
| *** slaweq has joined #openstack-infra | 16:10 | |
| fungi | but i agree the /usr/local/bin/release-volumes script we call from cron seems to only be for the docs and docs.dev volumes | 16:12 |
| fungi | so it's not that | 16:12 |
| *** trown is now known as trown|lunch | 16:14 | |
| *** studarus has joined #openstack-infra | 16:16 | |
| *** panda is now known as panda|off | 16:24 | |
| *** dims has quit IRC | 16:28 | |
| *** olivierb_ has quit IRC | 16:29 | |
| weshay | folks sorry for the resets last week.. that suuuucked.. fyi http://lists.openstack.org/pipermail/openstack-dev/2018-October/135396.html | 16:32 |
| weshay | ping me or jaosorior w/ any questions | 16:33 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove sphinx publishing from openstack-ansible-os_monasca https://review.openstack.org/607636 | 16:33 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Add release-zuul-python job that deals with javascript https://review.openstack.org/607637 | 16:35 |
| *** dims_ has joined #openstack-infra | 16:35 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Run release-zuul-python on release https://review.openstack.org/607638 | 16:36 |
| fungi | weshay: i saw your announcement, thanks for working on such drastic reductions in usage! | 16:39 |
| AJaeger | fungi, mordred, could either of you +2A https://review.openstack.org/#/c/597289 - python3-first heat change, please? | 16:39 |
| weshay | fungi, aye lots of good reasons to change things up a bit including reducing the amount of resources we're using :) | 16:39 |
| AJaeger | weshay: thanks! | 16:40 |
| njohnston | fungi: apropos to this discussion, I was wondering if there was a way to break down total volume of test executions by project over time? The Neutron CI subteam has tried to be sensitive to our impact on the CI system, but I'd like to bring some facts to the discussion so that if we are using a large share we can address it. | 16:42 |
| AJaeger | fungi, clarkb, mordred, please also review https://review.openstack.org/#/c/607636/ - so that we can continue in openstack-ansible with python3-first | 16:42 |
| fungi | njohnston: not really completely yet. we have a lot of data in http://graphite.openstack.org/ but we're missing a mapping of job/project to node in use duration | 16:44 |
| fungi | and really what you want to know is how many nodes a job uses x how long it uses them | 16:44 |
| njohnston | fungi: true, true. OK, thanks! | 16:45 |
| fungi | there's some work in progress to expose some of that at least in the zuul debug logs initially, but i've not been following closely enough | 16:46 |
| fungi | also we're wary of setting up a scoreboard and having projects point fingers at one another | 16:46 |
| *** e0ne has quit IRC | 16:47 | |
| AJaeger | thanks, fungi and clarkb | 16:48 |
| openstackgerrit | Monty Taylor proposed openstack-infra/nodepool master: Run release-zuul-python on release https://review.openstack.org/607649 | 16:49 |
| njohnston | fungi: that is very wise. I may just try to find an aggregate graph of all usage so that I can perhaps try to make statements like "When we added these 3 jobs to the check queue, usage went up 5% across the board" or similar, with correlation != causation caveats. | 16:49 |
| AJaeger | config-core, we can also move forward with monasca and python3-first: https://review.openstack.org/595744 - and here's a small cleanup https://review.openstack.org/607348 - reviews welcome | 16:50 |
| *** chkumar|pyconind has quit IRC | 16:51 | |
| openstackgerrit | Merged openstack-infra/project-config master: remove job settings for heat repositories https://review.openstack.org/597289 | 16:53 |
| kmalloc | is there any interest in applying LetsEncrypt (or similar) to zuul.o.o (or wherever it eventually will live) | 16:53 |
| kmalloc | oh wait | 16:54 |
| kmalloc | we should just add a redirect for http->https. that does it. | 16:54 |
| kmalloc | *facepalm* i can't belive i didn't just try https... | 16:54 |
| kmalloc | knikolla: auth flaskification is ready for eyes,. | 16:56 |
| kmalloc | whoopse, wrong channel | 16:56 |
| *** chandankumar has joined #openstack-infra | 16:56 | |
| *** derekh has quit IRC | 17:00 | |
| *** ykarel|away has quit IRC | 17:02 | |
| fungi | kmalloc: but separately, yes there is a proposed spec. we're mostly now arguing over whether centralized or decentralized acme will work better | 17:05 |
| fungi | there are pros and cons to both models | 17:05 |
| kmalloc | right. i like decentralized where possible for obvious reasons, i know the negatives | 17:06 |
| kmalloc | but more important that all of that, we should do a simple http->https redirect for zuul.o.o in the short term. | 17:06 |
| kmalloc | since we have TLS on zuul etc. | 17:06 |
| * kmalloc should go poke at that | 17:07 | |
| kmalloc | but before I can do that, i owe yall some OIDC test infrastructure | 17:07 |
| *** chandankumar has quit IRC | 17:07 | |
| kmalloc | now that i have my account with vexxhost setup, i'll get that going (last week was crazy personal life wise) | 17:07 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/zuul-jobs master: WIP: Add role to install kubernetes https://review.openstack.org/605823 | 17:10 |
| *** jpena is now known as jpena|off | 17:11 | |
| *** trown|lunch is now known as trown | 17:11 | |
| *** bobh has joined #openstack-infra | 17:14 | |
| *** gfidente|afk has quit IRC | 17:16 | |
| *** rfolco_doc has quit IRC | 17:23 | |
| AJaeger | fungi, could either you review //review.openstack.org/595744 for monasca python3-first as well, please? | 17:28 |
| openstackgerrit | Merged openstack-infra/system-config master: Retain fewer etherpad database backups https://review.openstack.org/607575 | 17:29 |
| *** jpich has quit IRC | 17:32 | |
| *** aojeagarcia has quit IRC | 17:37 | |
| *** olivierb has joined #openstack-infra | 17:41 | |
| *** jamesmcarthur has quit IRC | 17:41 | |
| fungi | clarkb: do you think i need to do something like `fs mkmount /afs/openstack.org/project/starlingx.io project.starlingx` to get it appearing there? | 17:46 |
| *** shardy has quit IRC | 17:47 | |
| clarkb | fungi: I dont think so afs should mount it automatocally? | 17:47 |
| fungi | `fs lsmount -dir /afs/openstack.org/project/zuul-ci.org` claims: | 17:47 |
| clarkb | oh maybe I need to create a RO volume on the same fileserver? I modeled it after zuul | 17:47 |
| fungi | '/afs/openstack.org/project/zuul-ci.org' is a mount point for volume '#project.zuul' | 17:47 |
| clarkb | and I dont think zuul had that | 17:47 |
| fungi | zuul only has a rw site | 17:48 |
| clarkb | ya | 17:48 |
| clarkb | maybe there is a way to tell afs that RO is RW? | 17:48 |
| fungi | i likely need the second mkmount | 17:48 |
| *** amoralej has joined #openstack-infra | 17:48 | |
| amoralej | hi | 17:48 |
| clarkb | fungi possibly | 17:48 |
| openstackgerrit | Merged openstack-infra/project-config master: remove job settings for monasca repositories https://review.openstack.org/595744 | 17:49 |
| fungi | i suppose i can just fs rmmount it if not | 17:49 |
| amoralej | how can i check what are the nodepool node types i can use in upstream jobs? | 17:49 |
| amoralej | i'm looking the fedora one | 17:49 |
| amoralej | it's fedora-28 ? | 17:49 |
| fungi | clarkb: no dice. `fs mkmount /afs/openstack.org/project/starlingx.io project.starlingx` errors with "fs: You can not change a backup or readonly volume" | 17:49 |
| clarkb | fungi: I can see if mordred or corvys can help after lunch | 17:51 |
| fungi | still pouring through documentation but it's not immediately obvious what i should be looking for | 17:51 |
| fungi | `vos examine` gives basically the same info for both -id project.zuul and -id project.starlingx | 17:55 |
| *** bhavikdbavishi has joined #openstack-infra | 17:57 | |
| *** jamesmcarthur has joined #openstack-infra | 17:57 | |
| *** njohnston has quit IRC | 17:57 | |
| *** njohnston has joined #openstack-infra | 17:58 | |
| *** e0ne has joined #openstack-infra | 18:02 | |
| *** jamesmcarthur has quit IRC | 18:08 | |
| *** bhavikdbavishi1 has joined #openstack-infra | 18:14 | |
| *** bhavikdbavishi has quit IRC | 18:15 | |
| *** bhavikdbavishi1 is now known as bhavikdbavishi | 18:15 | |
| *** studarus has quit IRC | 18:17 | |
| *** imacdonn has quit IRC | 18:21 | |
| *** imacdonn has joined #openstack-infra | 18:21 | |
| fungi | TheJulia: regarding your post on the tenks thread, we _do_ have arm64 nodes available to run jobs today. you probably knew that, but just in case you didn't... | 18:22 |
| *** pcaruana has joined #openstack-infra | 18:24 | |
| AJaeger | config-core, some smaller cleanups for review, please: https://review.openstack.org/#/c/607584/ https://review.openstack.org/#/c/607348/ https://review.openstack.org/#/c/607586/ | 18:27 |
| mnaser | clarkb: is it possible for us to add a flavor with min-servers: 0 and max-servers: 2 in infra (machines with gpus) | 18:29 |
| *** felipemonteiro has joined #openstack-infra | 18:29 | |
| mnaser | would be useful for experimental testing of gpus in nova CI | 18:30 |
| mnaser | for perhaps things like vgpus etc | 18:30 |
| pabelanger | mnaser: should be possible, we'd just create a new pool for it | 18:31 |
| mnaser | the flavors are currently available.. so just wondering who would be available to help add those (i'm really stretched thin on time honestly) | 18:32 |
| clarkb | amoralej: https://git.openstack.org/cgit/openstack-infra/project-config/tree/nodepool/nodepool.yaml#n115 is the list of images built | 18:32 |
| mnaser | and it's a bit out of my normal nodepool knowledge | 18:32 |
| clarkb | amoralej: the instance labels should be 1:1 with those image names too | 18:32 |
| amoralej | ok, thanks clarkb | 18:33 |
| *** ssbarnea|bkp2 has quit IRC | 18:33 | |
| clarkb | mnaser: ya should be possible, you can copy the kata setup for that and tweak the number values | 18:33 |
| mnaser | oh kata would be an example, right | 18:33 |
| mnaser | let me try | 18:33 |
| fungi | clarkb: reading the openafs admin guide, is it possible the starlingx.io mount point needed to be created with the -rw option? | 18:34 |
| fungi | http://docs.openafs.org/AdminGuide/HDRWQ208.html#HDRWQ212 | 18:34 |
| clarkb | fungi: maybe, I think I followed our docs on it and copy pasted stuff | 18:34 |
| clarkb | fungi: I've asked mordred and corvus to help too | 18:35 |
| fungi | yeah, our docs seem to be specific to creating volumes where we're going to have replicas | 18:35 |
| fungi | i have a feeling we needed something slightly different for a volume which wasn't going to have separate ro replicas | 18:35 |
| fungi | or else something is broken somewhere | 18:35 |
| *** bhavikdbavishi has quit IRC | 18:37 | |
| corvus | fungi: `fs mkmount /afs/.openstack.org/project/starlingx.io project.starlingx` | 18:41 |
| *** diablo_rojo has joined #openstack-infra | 18:41 | |
| fungi | corvus: okay, well that looks like what clarkb probably did | 18:42 |
| fungi | what would cause it to not appear under /afs/openstack.org/project/ and only under /afs/.openstack.org/project/ ? | 18:42 |
| corvus | fungi: oh, then 'vos release cell.root' | 18:43 |
| fungi | are we simply missing a required step? | 18:43 |
| fungi | VLDB: no such entry | 18:43 |
| corvus | er maybe use its actual name | 18:43 |
| fungi | should that be a literal command or was i supposed to sub something in? | 18:43 |
| corvus | fungi: root.cell | 18:44 |
| *** jamesmcarthur has joined #openstack-infra | 18:44 | |
| fungi | aha! that seems to be doing something | 18:44 |
| fungi | my cache may be stale | 18:44 |
| corvus | fungi: the read-only copy of the volume named root.cell mounted at /afs/openstack.org. the read-write copy is mounted at /afs/.openstack.org | 18:45 |
| fungi | Released volume root.cell successfully | 18:45 |
| corvus | we've created the mountpoint on the read-write volume. vos release will update the read-only volume with the new data | 18:45 |
| corvus | oh, project is a volume too | 18:45 |
| *** e0ne has quit IRC | 18:45 | |
| corvus | fungi: vos release project | 18:45 |
| fungi | Released volume project successfully | 18:46 |
| corvus | so, er, all of what i just said, but actually we're writing to the volume named 'project' mounted at /afs/openstack.org/project | 18:46 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/project-config master: Add GPU instances to CI infrastructure https://review.openstack.org/607686 | 18:46 |
| corvus | fungi: http://files.openstack.org/project/ lgtm now | 18:46 |
| mnaser | clarkb: pabelanger ^ | 18:46 |
| fungi | corvus: great! i probably still have a stale local afs cache... "ls: cannot access '/afs/openstack.org/project/starlingx.io': No such file or directory" | 18:47 |
| fungi | thanks corvus!!! | 18:47 |
| fungi | dtroyer: https://docs.starlingx.io/ is working now | 18:47 |
| clarkb | corvus: thank you! | 18:48 |
| *** e0ne has joined #openstack-infra | 18:48 | |
| fungi | corvus: so just to clarify, if we create a mount point under another volume, we need to vos release that volume? | 18:48 |
| clarkb | fungi: ya that was how I understood it because the mount point is part of that other volume | 18:48 |
| clarkb | and that volume needs a release to see the new mountpoint | 18:49 |
| fungi | excellent | 18:49 |
| fungi | i didn't even consider trying that | 18:49 |
| corvus | fungi: yeah, a mountpoint is just a special kind of file | 18:49 |
| corvus | (like a symlink in posix) | 18:49 |
| fungi | got it | 18:49 |
| fungi | i'll see if i can figure out what hint we should add to our runbook article | 18:50 |
| *** gyee has quit IRC | 18:50 | |
| *** pcaruana has quit IRC | 18:50 | |
| pabelanger | mnaser: looks correct, we maybe should discuss label names, but should be valid nodepool.yaml syntax | 18:51 |
| mnaser | i'm open, as long as we have min-ready: 0 because these are pricey | 18:51 |
| mnaser | :) | 18:51 |
| *** jlibosva has quit IRC | 18:55 | |
| *** rfolco has joined #openstack-infra | 18:56 | |
| *** efried has quit IRC | 18:58 | |
| *** efried has joined #openstack-infra | 19:00 | |
| openstackgerrit | Clint 'SpamapS' Byrum proposed openstack-infra/zuul-jobs master: Add a markdownlint job https://review.openstack.org/607691 | 19:03 |
| *** amoralej is now known as amoralej|off | 19:03 | |
| *** e0ne has quit IRC | 19:10 | |
| *** jlibosva has joined #openstack-infra | 19:12 | |
| *** jlibosva has quit IRC | 19:16 | |
| *** ssbarnea|bkp2 has joined #openstack-infra | 19:18 | |
| *** jamesmcarthur has quit IRC | 19:19 | |
| *** jamesmcarthur has joined #openstack-infra | 19:19 | |
| *** spartakos has joined #openstack-infra | 19:25 | |
| dtroyer | fungi, clarkb, corvus: thank you gentlemen! | 19:29 |
| fungi | dtroyer: i think you must have me confused with an actual gentleman, but will accept your thanks nonetheless | 19:30 |
| *** jlibosva has joined #openstack-infra | 19:33 | |
| *** rockyg has joined #openstack-infra | 19:37 | |
| *** jlibosva has quit IRC | 19:38 | |
| *** anteaya has joined #openstack-infra | 19:40 | |
| *** rfolco has quit IRC | 19:47 | |
| *** jlibosva has joined #openstack-infra | 19:54 | |
| *** jlibosva has quit IRC | 19:58 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Hyperlink task footers on review-dev https://review.openstack.org/607698 | 20:01 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Hyperlink task footers https://review.openstack.org/607699 | 20:01 |
| *** slaweq has quit IRC | 20:10 | |
| *** slaweq has joined #openstack-infra | 20:11 | |
| *** jlibosva has joined #openstack-infra | 20:15 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-pip master: [DNM] debugging https://review.openstack.org/607048 | 20:17 |
| *** spartakos has quit IRC | 20:17 | |
| *** rossella_s has quit IRC | 20:19 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-pip master: [DNM] debugging https://review.openstack.org/607048 | 20:19 |
| *** jlibosva has quit IRC | 20:20 | |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-pip master: [DNM] debugging https://review.openstack.org/607048 | 20:20 |
| openstackgerrit | Colleen Murphy proposed openstack-infra/puppet-pip master: [debug] Fix openstack_pip provider for pip 18 https://review.openstack.org/606021 | 20:21 |
| *** gyee has joined #openstack-infra | 20:24 | |
| *** ansmith has quit IRC | 20:30 | |
| *** rossella_s has joined #openstack-infra | 20:36 | |
| *** jlibosva has joined #openstack-infra | 20:36 | |
| *** felipemonteiro has quit IRC | 20:40 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove sphinx publishing from openstack-ansible-os_monasca https://review.openstack.org/607636 | 20:41 |
| *** jlibosva has quit IRC | 20:42 | |
| AJaeger | fungi, if you have some review cycles later: some smaller cleanups for review, please: https://review.openstack.org/#/c/607584/ https://review.openstack.org/#/c/607348/ https://review.openstack.org/#/c/607586/ | 20:43 |
| *** ijw has joined #openstack-infra | 20:46 | |
| *** olivierb has quit IRC | 20:47 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Remove docs from os_monasca-agent https://review.openstack.org/607706 | 20:48 |
| AJaeger | argh, fixed the wrong repo - we have too many ;( | 20:48 |
| AJaeger | mnaser, clarkb , fungi, could you review https://review.openstack.org/607706 , please? | 20:48 |
| * AJaeger waves good night... | 20:48 | |
| *** _erlon_ has quit IRC | 20:48 | |
| mnaser | AJaeger: night +2 | 20:48 |
| *** bobh has quit IRC | 20:51 | |
| *** rockyg has quit IRC | 20:52 | |
| *** spartakos has joined #openstack-infra | 20:54 | |
| *** bobh has joined #openstack-infra | 20:56 | |
| *** eharney has quit IRC | 20:57 | |
| *** jlibosva has joined #openstack-infra | 20:58 | |
| *** priteau has quit IRC | 21:01 | |
| *** jlibosva has quit IRC | 21:02 | |
| *** trown is now known as trown|outtypewww | 21:05 | |
| *** spartakos has quit IRC | 21:08 | |
| manjeets | is there any documentation up there for zuul v3 adoption in third party ci's for openstack | 21:08 |
| manjeets | we have 3 rd party CI which is still using jenkins and old guidelines. but we'd like to do zuul v3 adoption like other projects in community are doing | 21:09 |
| fungi | manjeets: there's lots of documentation for running and configuring zuul v3 at https://zuul-ci.org/docs/ but none that's a walkthrough specific to operating an openstack project third-party ci system no | 21:09 |
| fungi | there is a spec under review https://review.openstack.org/563849 | 21:10 |
| *** felipemonteiro has joined #openstack-infra | 21:11 | |
| *** kgiusti has left #openstack-infra | 21:12 | |
| *** jlibosva has joined #openstack-infra | 21:19 | |
| *** jlibosva has quit IRC | 21:24 | |
| *** spartakos has joined #openstack-infra | 21:26 | |
| *** jamesmcarthur has quit IRC | 21:26 | |
| *** agopi is now known as agopi|brb | 21:30 | |
| *** slaweq has quit IRC | 21:30 | |
| *** rossella_s has quit IRC | 21:34 | |
| *** jamesmcarthur has joined #openstack-infra | 21:36 | |
| *** agopi|brb has quit IRC | 21:37 | |
| *** jlibosva has joined #openstack-infra | 21:40 | |
| *** rh-jelabarre has quit IRC | 21:43 | |
| *** slagle has quit IRC | 21:44 | |
| *** jlibosva has quit IRC | 21:45 | |
| *** boden has quit IRC | 21:53 | |
| *** jtomasek has quit IRC | 21:54 | |
| *** munimeha1 has quit IRC | 21:56 | |
| *** jlibosva has joined #openstack-infra | 22:01 | |
| *** jlibosva has quit IRC | 22:06 | |
| *** bobh has quit IRC | 22:07 | |
| *** agopi|brb has joined #openstack-infra | 22:13 | |
| *** spartakos has quit IRC | 22:17 | |
| *** jlibosva has joined #openstack-infra | 22:23 | |
| *** tosky has quit IRC | 22:24 | |
| *** rcernin has joined #openstack-infra | 22:25 | |
| *** jlibosva has quit IRC | 22:28 | |
| *** flaper87 has quit IRC | 22:32 | |
| *** flaper87 has joined #openstack-infra | 22:32 | |
| *** agopi_ has joined #openstack-infra | 22:36 | |
| *** agopi|brb has quit IRC | 22:38 | |
| *** agopi__ has joined #openstack-infra | 22:38 | |
| *** agopi_ has quit IRC | 22:41 | |
| *** jlibosva has joined #openstack-infra | 22:44 | |
| *** agopi__ has quit IRC | 22:47 | |
| *** agopi__ has joined #openstack-infra | 22:47 | |
| *** jlibosva has quit IRC | 22:49 | |
| *** agopi__ has quit IRC | 22:58 | |
| *** agopi__ has joined #openstack-infra | 22:58 | |
| *** bkero has joined #openstack-infra | 23:01 | |
| *** spartakos has joined #openstack-infra | 23:01 | |
| *** pbourke has quit IRC | 23:03 | |
| *** pbourke has joined #openstack-infra | 23:03 | |
| *** jlibosva has joined #openstack-infra | 23:05 | |
| *** agopi__ has quit IRC | 23:06 | |
| *** agopi__ has joined #openstack-infra | 23:06 | |
| *** jlibosva has quit IRC | 23:10 | |
| *** jamesmcarthur has quit IRC | 23:11 | |
| *** rlandy is now known as rlandy|afk | 23:21 | |
| *** jlibosva has joined #openstack-infra | 23:26 | |
| manjeets | fungi, thanks, can I use my own gerrit account for experiments over sandbox project ? | 23:29 |
| manjeets | i'll setup official account later once I learn the whole pipeline | 23:29 |
| fungi | manjeets: yes, you can use whatever account you like for testing with the sandbox repo | 23:30 |
| *** jlibosva has quit IRC | 23:31 | |
| mnaser | clarkb: https://review.openstack.org/#/c/607686/ let me know what you think from a ptl pov :) | 23:35 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/project-config master: Disable vexxhost mtl1 https://review.openstack.org/607737 | 23:42 |
| openstackgerrit | Mohammed Naser proposed openstack-infra/project-config master: Disable vexxhost mtl1 https://review.openstack.org/607737 | 23:44 |
| fungi | mnaser: what's the connection between mtl1 and ca-ymq-1? are those just names for the same region? | 23:46 |
| mnaser | fungi: we've realized that naming our regions with the prefix of the country and airport code very redundant | 23:46 |
| fungi | heh, indeed | 23:46 |
| mnaser | so sjc1 was the move towards that, ca-ymq-1 will eventually become mtl1 or ymq1 or something along those lines to make region names cleaner | 23:47 |
| mnaser | we just haven't gotten around finalizing that | 23:47 |
| fungi | makes sense! | 23:47 |
| mnaser | i'm guessing it going to be a whole process to do that and make sure we don't break $world | 23:47 |
| *** jlibosva has joined #openstack-infra | 23:48 | |
| *** slagle has joined #openstack-infra | 23:51 | |
| *** mriedem has quit IRC | 23:51 | |
| *** jlibosva has quit IRC | 23:53 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!