| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient master: Update fontawesome to version 5 https://review.openstack.org/545676 | 00:05 |
|---|---|---|
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient master: Make board and worklist icons unique https://review.openstack.org/545677 | 00:05 |
| *** mriedem has joined #openstack-infra | 00:11 | |
| persia | SotK: http://logs.openstack.org/77/545677/7/check/nodejs-npm-run-lint/dd262a0/job-output.txt.gz still :( | 00:13 |
| persia | (collected from zuul.openstack.org before gerrit updates) | 00:13 |
| *** _jbadiapa has quit IRC | 00:13 | |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient master: Update fontawesome to version 5 https://review.openstack.org/545676 | 00:14 |
| openstackgerrit | Adam Coldrick proposed openstack-infra/storyboard-webclient master: Make board and worklist icons unique https://review.openstack.org/545677 | 00:14 |
| SotK | persia: that is me making stupid mistakes rather than a problem with the job at least :) | 00:15 |
| persia | Despite that, I wonder if it makes sense to modify the job so npm-debug.log is captured. | 00:16 |
| * persia discovers http://zuul.openstack.org/stream.html?uuid=036728fec4c947f18987b3249ddbe27b&logfile=console.log and is entranced | 00:17 | |
| *** lbragstad has joined #openstack-infra | 00:20 | |
| *** tosky has quit IRC | 00:26 | |
| *** panda has joined #openstack-infra | 00:26 | |
| *** lbragstad has quit IRC | 00:27 | |
| *** aviau has quit IRC | 00:36 | |
| *** aviau has joined #openstack-infra | 00:36 | |
| *** panda is now known as panda|off | 00:38 | |
| pabelanger | ianw: replied to 545314 | 00:41 |
| *** abelur_ has joined #openstack-infra | 00:46 | |
| *** claudiub has quit IRC | 00:55 | |
| *** bobh has joined #openstack-infra | 01:07 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Allow multiple declarations of ARM64 of HWE kernel package https://review.openstack.org/545733 | 01:12 |
| *** slaweq has joined #openstack-infra | 01:15 | |
| *** dhill__ has quit IRC | 01:18 | |
| *** slaweq has quit IRC | 01:20 | |
| pabelanger | ianw: thanks for glean +2, if you do get a change to +3 https://review.openstack.org/545611/ today, a new release of DIB would be helpful too! | 01:26 |
| pabelanger | I'm out for the night | 01:26 |
| *** SumitNaiksatam has quit IRC | 01:35 | |
| *** hongbin has joined #openstack-infra | 01:36 | |
| *** SumitNaiksatam has joined #openstack-infra | 01:37 | |
| *** yamahata has joined #openstack-infra | 02:03 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Reorgnize the output bundle split https://review.openstack.org/545706 | 02:04 |
| *** SumitNaiksatam has quit IRC | 02:11 | |
| *** links has joined #openstack-infra | 02:11 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/system-config master: Allow multiple declarations of ARM64 of HWE kernel package https://review.openstack.org/545733 | 02:12 |
| *** lbragstad has joined #openstack-infra | 02:15 | |
| *** r-daneel has joined #openstack-infra | 02:24 | |
| *** bobh has quit IRC | 02:29 | |
| *** eroux has quit IRC | 02:31 | |
| *** r-daneel has quit IRC | 02:32 | |
| *** bobh has joined #openstack-infra | 02:33 | |
| *** agopi has quit IRC | 02:36 | |
| *** agopi has joined #openstack-infra | 02:36 | |
| *** salv-orl_ has joined #openstack-infra | 02:45 | |
| openstackgerrit | Merged openstack-infra/system-config master: Allow multiple declarations of ARM64 of HWE kernel package https://review.openstack.org/545733 | 02:48 |
| *** salv-orlando has quit IRC | 02:48 | |
| *** VW has joined #openstack-infra | 02:50 | |
| *** rlandy has quit IRC | 02:52 | |
| *** VW has quit IRC | 02:55 | |
| *** yamamoto_ has joined #openstack-infra | 03:08 | |
| *** xarses has joined #openstack-infra | 03:10 | |
| *** andreww has joined #openstack-infra | 03:11 | |
| *** yamamoto has quit IRC | 03:12 | |
| *** xarses has quit IRC | 03:14 | |
| *** VW has joined #openstack-infra | 03:26 | |
| *** lbragstad has quit IRC | 03:29 | |
| *** sree has joined #openstack-infra | 03:32 | |
| *** cshastri has joined #openstack-infra | 03:32 | |
| *** VW has quit IRC | 03:33 | |
| *** VW has joined #openstack-infra | 03:34 | |
| *** r-daneel has joined #openstack-infra | 03:40 | |
| ianw | gema: been trying today to make sure our puppet "just works" deploying arm64 servers by rebuilding mirror01. going ok; i have noticed that it's pretty hit or miss when the host comes up if it gets network or not. when it does, and we can ssh into it after a minute or so, seems to work fine | 03:46 |
| ianw | but there seems to be a second case where it just never responds on the floating ip. can probably see the various vm's that have come and gone in the os-control account | 03:46 |
| *** andreww has quit IRC | 03:55 | |
| *** andreww has joined #openstack-infra | 03:59 | |
| *** xarses has joined #openstack-infra | 04:01 | |
| *** andreww has quit IRC | 04:04 | |
| *** kiennt26 has joined #openstack-infra | 04:06 | |
| *** ykarel has joined #openstack-infra | 04:14 | |
| *** dhajare|mtg has joined #openstack-infra | 04:15 | |
| *** bobh has quit IRC | 04:18 | |
| *** dhajare|mtg has quit IRC | 04:24 | |
| *** rosmaita has quit IRC | 04:25 | |
| openstackgerrit | Merged openstack/diskimage-builder master: Install systemd earlier for Ubuntu Bionic https://review.openstack.org/545611 | 04:31 |
| *** bhujay has joined #openstack-infra | 04:40 | |
| *** psachin has joined #openstack-infra | 04:46 | |
| *** dave-mccowan has quit IRC | 04:53 | |
| *** kiennt26 has quit IRC | 05:01 | |
| *** coolsvap has joined #openstack-infra | 05:03 | |
| *** stakeda has quit IRC | 05:04 | |
| *** olaph has joined #openstack-infra | 05:13 | |
| *** stakeda has joined #openstack-infra | 05:15 | |
| *** olaph1 has quit IRC | 05:16 | |
| *** slaweq has joined #openstack-infra | 05:17 | |
| *** dhajare has joined #openstack-infra | 05:21 | |
| *** slaweq has quit IRC | 05:22 | |
| *** claudiub has joined #openstack-infra | 05:27 | |
| *** yamamoto has joined #openstack-infra | 05:34 | |
| *** janki has joined #openstack-infra | 05:38 | |
| *** yamamoto_ has quit IRC | 05:38 | |
| *** ramishra has joined #openstack-infra | 05:50 | |
| *** bhavik1 has joined #openstack-infra | 05:52 | |
| *** hongbin has quit IRC | 05:53 | |
| openstackgerrit | Merged openstack-infra/project-config master: Sync sphinx post jobs https://review.openstack.org/545609 | 05:54 |
| *** caphrim007_ has quit IRC | 05:56 | |
| *** VW has quit IRC | 05:56 | |
| *** caphrim007 has joined #openstack-infra | 05:56 | |
| *** bhavik1 has quit IRC | 06:00 | |
| *** caphrim007 has quit IRC | 06:01 | |
| *** olaph1 has joined #openstack-infra | 06:03 | |
| *** olaph has quit IRC | 06:03 | |
| robcresswell | AJaeger: New version of python-twitter is out on PyPI as 3.4. Once that filters through requirements, (and I guess that bot is restarted?) then the Twitter updates shouldn't split prematurely, and updates on there will be a little neater. | 06:05 |
| *** threestrands has quit IRC | 06:10 | |
| *** kjackal has joined #openstack-infra | 06:12 | |
| *** auristor1 has joined #openstack-infra | 06:23 | |
| *** auristor has quit IRC | 06:23 | |
| *** threestrands has joined #openstack-infra | 06:23 | |
| *** jbadiapa has joined #openstack-infra | 06:24 | |
| *** auristor1 has quit IRC | 06:25 | |
| *** auristor has joined #openstack-infra | 06:27 | |
| *** alexchadin has joined #openstack-infra | 06:42 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: web: angular controller for status and stream page https://review.openstack.org/545755 | 06:48 |
| *** olaph has joined #openstack-infra | 06:54 | |
| *** olaph1 has quit IRC | 06:56 | |
| AJaeger | robcresswell: no idea how it works - better ask some other infra-root ^. Thanks for the heads-up! | 06:57 |
| *** threestrands has quit IRC | 07:00 | |
| *** AJaeger has quit IRC | 07:00 | |
| *** pgadiya has joined #openstack-infra | 07:02 | |
| *** AJaeger has joined #openstack-infra | 07:04 | |
| *** rcernin has quit IRC | 07:11 | |
| *** e0ne has joined #openstack-infra | 07:11 | |
| *** armaan has joined #openstack-infra | 07:15 | |
| *** andreas_s has joined #openstack-infra | 07:27 | |
| *** alexchadin has quit IRC | 07:27 | |
| *** e0ne has quit IRC | 07:29 | |
| *** yamamoto has quit IRC | 07:33 | |
| *** e0ne has joined #openstack-infra | 07:35 | |
| *** alexchadin has joined #openstack-infra | 07:39 | |
| *** yamamoto has joined #openstack-infra | 07:41 | |
| *** slaweq has joined #openstack-infra | 07:43 | |
| *** dsariel has joined #openstack-infra | 07:43 | |
| *** slaweq_ has joined #openstack-infra | 07:45 | |
| *** yamamoto has quit IRC | 07:46 | |
| *** pcaruana has joined #openstack-infra | 07:47 | |
| *** pcichy has joined #openstack-infra | 07:47 | |
| *** e0ne has quit IRC | 07:47 | |
| *** slaweq has quit IRC | 07:48 | |
| *** links has quit IRC | 07:55 | |
| *** slaweq has joined #openstack-infra | 08:03 | |
| ianw | gema: ok, i've left one behind if you get a chance to look ... b78f9d24-8881-425f-8648-0a62cfa82f82 . afaict from console it's happy, but no network (211.148.24.206) | 08:04 |
| *** e0ne has joined #openstack-infra | 08:06 | |
| *** links has joined #openstack-infra | 08:07 | |
| *** e0ne has quit IRC | 08:08 | |
| *** slaweq has quit IRC | 08:10 | |
| *** pcichy has quit IRC | 08:12 | |
| *** florianf has joined #openstack-infra | 08:14 | |
| *** tesseract has joined #openstack-infra | 08:15 | |
| *** aviau has quit IRC | 08:19 | |
| *** aviau has joined #openstack-infra | 08:19 | |
| *** kopecmartin has joined #openstack-infra | 08:21 | |
| *** kopecmartin has quit IRC | 08:27 | |
| *** sree_ has joined #openstack-infra | 08:29 | |
| *** sree_ is now known as Guest79222 | 08:29 | |
| *** ralonsoh has joined #openstack-infra | 08:30 | |
| *** olaph1 has joined #openstack-infra | 08:32 | |
| *** sree has quit IRC | 08:32 | |
| *** olaph has quit IRC | 08:33 | |
| gnuoy | Hi, please could I get myself ( liam.young@canonical.com ) and the team PTL James Page ( james.page@canonical.com) added to the new charms-vault-core gerrit group please? | 08:33 |
| *** oidgar has joined #openstack-infra | 08:35 | |
| *** sshnaidm|off has quit IRC | 08:35 | |
| *** electrofelix has joined #openstack-infra | 08:35 | |
| *** krtaylor has quit IRC | 08:38 | |
| *** yamamoto has joined #openstack-infra | 08:39 | |
| *** amoralej|off is now known as amoralej | 08:41 | |
| *** jpena|off is now known as jpena | 08:43 | |
| *** sshnaidm has joined #openstack-infra | 08:44 | |
| *** bauwser is now known as bauazs | 08:53 | |
| *** bauazs is now known as bauzas | 08:53 | |
| ianw | gnuoy: done | 08:56 |
| gnuoy | fantastic, thanks ianw | 08:56 |
| *** caphrim007 has joined #openstack-infra | 08:59 | |
| *** dbecker has joined #openstack-infra | 09:00 | |
| *** gfidente has joined #openstack-infra | 09:00 | |
| *** jtomasek has joined #openstack-infra | 09:03 | |
| *** caphrim007 has quit IRC | 09:04 | |
| openstackgerrit | Antoine Musso proposed openstack-infra/zuul master: WIP: ensure that Change.number is a string https://review.openstack.org/545768 | 09:04 |
| *** jpich has joined #openstack-infra | 09:05 | |
| *** lpetrut has joined #openstack-infra | 09:06 | |
| *** HeOS has joined #openstack-infra | 09:06 | |
| *** sshnaidm has quit IRC | 09:07 | |
| *** sshnaidm has joined #openstack-infra | 09:08 | |
| *** ameeda has joined #openstack-infra | 09:17 | |
| ameeda | hello | 09:17 |
| ameeda | when I remove pycrypto as this bug needed https://bugs.launchpad.net/openstack-requirements/+bug/1749574, I got this error https://pasteboard.co/H8l6IC4.png | 09:17 |
| openstack | Launchpad bug 1749574 in OpenStack DBaaS (Trove) "[tracking] removal and migration of pycrypto" [Undecided,In progress] - Assigned to Ameed Ashour (ameeda) | 09:17 |
| *** adrianreza has joined #openstack-infra | 09:18 | |
| *** derekh has joined #openstack-infra | 09:21 | |
| *** xarses has quit IRC | 09:22 | |
| *** hashar_ has joined #openstack-infra | 09:24 | |
| *** rpittau has joined #openstack-infra | 09:24 | |
| *** r-daneel has quit IRC | 09:28 | |
| *** oidgar has quit IRC | 09:28 | |
| *** links has quit IRC | 09:32 | |
| *** alexchadin has quit IRC | 09:36 | |
| *** florianf has quit IRC | 09:41 | |
| *** oidgar has joined #openstack-infra | 09:43 | |
| *** links has joined #openstack-infra | 09:45 | |
| openstackgerrit | Antoine Musso proposed openstack-infra/zuul master: WIP: ensure that Change.number is a string https://review.openstack.org/545768 | 09:46 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul master: Tenant config dynamic loading of project from external script https://review.openstack.org/535878 | 09:46 |
| *** r-daneel has joined #openstack-infra | 09:46 | |
| *** oidgar has quit IRC | 09:47 | |
| AJaeger | ameeda: you need to chagne the code to use another lib for that, see https://bugs.launchpad.net/openstack-requirements/+bug/1749574/comments/1 | 09:51 |
| openstack | Launchpad bug 1749574 in OpenStack DBaaS (Trove) "[tracking] removal and migration of pycrypto" [Undecided,In progress] - Assigned to Ameed Ashour (ameeda) | 09:51 |
| *** snapiri has quit IRC | 09:53 | |
| *** e0ne has joined #openstack-infra | 09:54 | |
| *** florianf has joined #openstack-infra | 09:54 | |
| *** pcichy has joined #openstack-infra | 09:54 | |
| *** salv-orl_ has quit IRC | 10:02 | |
| *** salv-orlando has joined #openstack-infra | 10:03 | |
| *** r-daneel has quit IRC | 10:06 | |
| *** salv-orlando has quit IRC | 10:07 | |
| ameeda | AJaeger: thanks for reply, the requirements already has cryptography lib | 10:07 |
| *** snapiri has joined #openstack-infra | 10:09 | |
| *** Guest79222 has quit IRC | 10:09 | |
| openstackgerrit | Spyros Trigazis (strigazi) proposed openstack-infra/project-config master: [magnum] Add irc notifications for queens https://review.openstack.org/545786 | 10:10 |
| *** sree_ has joined #openstack-infra | 10:12 | |
| *** sree_ is now known as Guest85075 | 10:12 | |
| AJaeger | ameeda: you need to update the *code* as well to use the new library | 10:13 |
| *** kmalloc has joined #openstack-infra | 10:14 | |
| *** shardy has joined #openstack-infra | 10:15 | |
| *** Guest85075 has quit IRC | 10:16 | |
| ameeda | AJaeger: so I should change this line "from Crypto.PublicKey import RSA # nosec" from /opt/stack/barbican/barbican/tests/plugin/test_dogtag.py ? | 10:18 |
| AJaeger | ameeda: yes. Btw. better ask on #openstack-dev since that's the development channel... | 10:19 |
| openstackgerrit | Matthieu Huin proposed openstack-infra/zuul master: zuul web: add admin endpoint, enqueue & autohold commands https://review.openstack.org/539004 | 10:19 |
| ameeda | AJaeger: thank you very much!! | 10:19 |
| *** r-daneel has joined #openstack-infra | 10:22 | |
| *** snapiri has quit IRC | 10:24 | |
| chandankumar | AJaeger: Hello | 10:25 |
| chandankumar | AJaeger: please have a look at this review https://review.openstack.org/#/c/545745/ | 10:25 |
| *** tosky has joined #openstack-infra | 10:29 | |
| AJaeger | chandankumar: How can I help with that one? | 10:32 |
| AJaeger | chandankumar: better ask on #openstack-release channel... | 10:33 |
| chandankumar | AJaeger: sure | 10:34 |
| *** krenczewski has joined #openstack-infra | 10:35 | |
| *** stakeda has quit IRC | 10:40 | |
| *** panda|off is now known as panda | 10:45 | |
| ameeda | AJaeger: please check this out https://review.openstack.org/#/c/544873/ , is that correct ? | 10:46 |
| AJaeger | ameeda: I'm really not an expert on cryptography and thus cannot help further. If the repo has a good testsuite, I expect that running locally the py27 tests will gave a good indication of whether that's all that's needed. | 10:53 |
| ameeda | I see, I ran py27 locally and working . | 10:53 |
| *** dsariel has quit IRC | 10:54 | |
| AJaeger | ameeda: is that really only usage of Crypo? Check "git grep 'from Crypto'" and similar | 10:57 |
| ameeda | ok, I will check it now | 10:57 |
| *** alexchadin has joined #openstack-infra | 10:58 | |
| ameeda | I found those https://pasteboard.co/H8lN6V3.png | 10:59 |
| ameeda | but when I ran it locally it passes without any errors | 10:59 |
| AJaeger | ameeda: dogtag.py might not be run in the testsuite ;( You should update it as well. | 11:06 |
| *** lucas-afk is now known as lucasagomes | 11:18 | |
| *** priteau has joined #openstack-infra | 11:19 | |
| *** krtaylor has joined #openstack-infra | 11:22 | |
| *** zoli is now known as zoli|lunch | 11:22 | |
| *** zoli|lunch is now known as zoli | 11:22 | |
| *** dtantsur|afk is now known as dtantsur | 11:23 | |
| *** olaph1 has quit IRC | 11:23 | |
| *** salv-orlando has joined #openstack-infra | 11:25 | |
| *** olaph has joined #openstack-infra | 11:26 | |
| *** sree has joined #openstack-infra | 11:27 | |
| *** sree has quit IRC | 11:32 | |
| *** ldnunes has joined #openstack-infra | 11:36 | |
| *** pbourke has quit IRC | 11:37 | |
| *** pbourke has joined #openstack-infra | 11:38 | |
| *** bnemec is now known as bnemec-pto | 11:41 | |
| *** ldnunes has quit IRC | 11:42 | |
| *** agopi has quit IRC | 11:42 | |
| *** ldnunes has joined #openstack-infra | 11:42 | |
| *** katkapilatova has joined #openstack-infra | 11:50 | |
| *** masber has joined #openstack-infra | 11:58 | |
| *** alexchadin has quit IRC | 11:59 | |
| *** sree has joined #openstack-infra | 12:03 | |
| *** slaweq has joined #openstack-infra | 12:07 | |
| *** snapiri has joined #openstack-infra | 12:08 | |
| *** slaweq has quit IRC | 12:12 | |
| krenczewski | Hi, I wanted to write a custom CI job for my project. Is there any tutorial describing where to start withour reading whole Zuul reference? | 12:12 |
| *** dave-mccowan has joined #openstack-infra | 12:16 | |
| AJaeger | krenczewski: best to build on top of an existing job ;) What kind of testing do you want to do for your repo? | 12:17 |
| krenczewski | AJaeger: I need working instance of opencontrail to test if my plugin do the job correctly | 12:17 |
| AJaeger | krenczewski: Sorry, that's more involved then giving you a pointer - best ask again in two or three hours when more of the team is around | 12:19 |
| *** slagle has joined #openstack-infra | 12:20 | |
| *** olaph1 has joined #openstack-infra | 12:21 | |
| *** yamamoto has quit IRC | 12:21 | |
| krenczewski | AJaeger: Will try, I just wanted to know where to start. I am sure I will be able to make ansible part for spawning opencontrail instance, I just need to know how to setup a VM, and write the checks | 12:21 |
| *** olaph has quit IRC | 12:22 | |
| *** abelur_ has quit IRC | 12:23 | |
| *** olaph1 is now known as olaph | 12:23 | |
| *** yamamoto has joined #openstack-infra | 12:25 | |
| *** stevebaker has quit IRC | 12:25 | |
| *** dmellado has quit IRC | 12:26 | |
| AJaeger | krenczewski: we set up the VM automatically with a minimal set of packages installed. You would need to add your stuff to it. The openstack-infra/zuul-jobs , project-config and openstack-zuul-jobs include the common job definitions. The base - see http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul.d/jobs.yaml#n26 is the base for every one | 12:26 |
| *** dsariel has joined #openstack-infra | 12:26 | |
| AJaeger | So, that one sets gives you a virtual machine directly - and then you go from there. | 12:26 |
| andreykurilin | hi folks! Can someone guide me about adding a new repo under existing openstack programm? the new repo is exist on github, so it requires transfer... | 12:28 |
| AJaeger | krenczewski: a very simple unit test is in https://review.openstack.org/#/c/540666/ - but depending on what you do, you can do much more complex stuff. And since you test opencontrail with neutron, you should use the neutron tests and install your setup there | 12:29 |
| *** stevebaker has joined #openstack-infra | 12:29 | |
| *** stevebaker has quit IRC | 12:29 | |
| AJaeger | andreykurilin: all documented in the infra manual at https://docs.openstack.org/infra/manual/creators.html | 12:29 |
| *** stevebaker has joined #openstack-infra | 12:30 | |
| *** stevebaker has quit IRC | 12:36 | |
| *** links has quit IRC | 12:37 | |
| *** jpena is now known as jpena|lunch | 12:39 | |
| *** sshnaidm has quit IRC | 12:42 | |
| *** verdurin has joined #openstack-infra | 12:43 | |
| *** psachin has quit IRC | 12:51 | |
| *** ajo\pto is now known as ajo | 12:55 | |
| andreykurilin | AJaeger: thanks! | 12:56 |
| *** dprince has joined #openstack-infra | 12:57 | |
| krenczewski | AJaeger: Very much thanks! | 12:59 |
| *** caphrim007 has joined #openstack-infra | 13:00 | |
| *** rfolco|away is now known as rfolco | 13:02 | |
| *** jlabarre has joined #openstack-infra | 13:03 | |
| *** caphrim007 has quit IRC | 13:04 | |
| *** dmellado has joined #openstack-infra | 13:07 | |
| *** edmondsw has joined #openstack-infra | 13:12 | |
| *** jokke__ is now known as jokke_ | 13:14 | |
| *** sree_ has joined #openstack-infra | 13:14 | |
| *** sree_ is now known as Guest87972 | 13:15 | |
| *** sree has quit IRC | 13:17 | |
| *** ykarel has quit IRC | 13:20 | |
| *** olaph1 has joined #openstack-infra | 13:21 | |
| openstackgerrit | Andrey Kurilin proposed openstack-infra/project-config master: [rally] Add a separate repository for OpenStack plugins https://review.openstack.org/545824 | 13:22 |
| *** olaph has quit IRC | 13:22 | |
| *** sshnaidm has joined #openstack-infra | 13:25 | |
| openstackgerrit | Andrey Kurilin proposed openstack-infra/project-config master: [rally] Add a separate repository for OpenStack plugins https://review.openstack.org/545824 | 13:27 |
| openstackgerrit | Andrey Kurilin proposed openstack-infra/project-config master: [rally] Configure CI for a new repo https://review.openstack.org/545828 | 13:27 |
| *** oidgar has joined #openstack-infra | 13:28 | |
| *** rosmaita has joined #openstack-infra | 13:28 | |
| *** gnuoy has left #openstack-infra | 13:28 | |
| *** jpena|lunch is now known as jpena | 13:31 | |
| *** rlandy has joined #openstack-infra | 13:32 | |
| *** rlandy is now known as rlandy|rover | 13:33 | |
| *** bobh has joined #openstack-infra | 13:36 | |
| *** dmellado has quit IRC | 13:38 | |
| *** pgadiya has quit IRC | 13:38 | |
| *** jokke_ has quit IRC | 13:41 | |
| *** jokke_ has joined #openstack-infra | 13:41 | |
| *** trown|outtypewww is now known as trown | 13:42 | |
| openstackgerrit | Andrey Kurilin proposed openstack-infra/project-config master: [rally] Add a separate repository for OpenStack plugins https://review.openstack.org/545824 | 13:42 |
| *** jcoufal has joined #openstack-infra | 13:44 | |
| *** fultonj has joined #openstack-infra | 13:44 | |
| *** VW has joined #openstack-infra | 13:46 | |
| openstackgerrit | Andrey Kurilin proposed openstack-infra/project-config master: [rally] Add a separate repository for OpenStack plugins https://review.openstack.org/545824 | 13:49 |
| openstackgerrit | Andrey Kurilin proposed openstack-infra/project-config master: [rally] Configure CI for a new repo https://review.openstack.org/545828 | 13:49 |
| *** cshastri has quit IRC | 13:51 | |
| *** ykarel has joined #openstack-infra | 13:51 | |
| *** bobh has quit IRC | 13:54 | |
| *** masuberu has joined #openstack-infra | 13:54 | |
| *** dhill__ has joined #openstack-infra | 13:55 | |
| *** masber has quit IRC | 13:58 | |
| *** armaan has quit IRC | 14:01 | |
| *** armaan has joined #openstack-infra | 14:02 | |
| *** florianf has quit IRC | 14:05 | |
| *** amoralej is now known as amoralej|lunch | 14:06 | |
| *** armaan has quit IRC | 14:08 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Updated endpoint /api/v1/summits/{id}/event-types https://review.openstack.org/545838 | 14:10 |
| smcginnis | Seems like a lot of time outs and SSH connection errors over the weekend. | 14:15 |
| smcginnis | Any known issues in the gate right now? | 14:15 |
| smcginnis | Looks like I have a docs job that is hung and should eventually time out. | 14:15 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Updated endpoint /api/v1/summits/{id}/event-types https://review.openstack.org/545838 | 14:17 |
| *** florianf has joined #openstack-infra | 14:17 | |
| mnaser | I noticed a few over the weekend too smcginnis .. did you notice them being linked to a specific provider? | 14:17 |
| smcginnis | mnaser: Unfortunately I didn't take the time to look that closely. | 14:18 |
| cmurphy | I think it was a zuul issue, this one for example had failures on rax ovh and inap https://review.openstack.org/#/c/545680/ | 14:20 |
| *** esberglu has joined #openstack-infra | 14:20 | |
| cmurphy | it seems to be happier now | 14:20 |
| *** Goneri has joined #openstack-infra | 14:20 | |
| andreaf | corvus, clarkb: to help people migrate their jobs I plan on writing a document which a kind of map DEVSTACK_GATE flag -> zuul v3 variable - but I wonder where best to host it | 14:20 |
| andreaf | corvus, corvus: since the jobs are defined in devstack/tempest I was thinking perhaps to add it to tempest docs - an alternative could be in devstack-gate | 14:21 |
| andreaf | clarkb ^^^ | 14:21 |
| *** rosmaita has quit IRC | 14:22 | |
| *** lbragstad has joined #openstack-infra | 14:23 | |
| *** aviau has quit IRC | 14:27 | |
| *** aviau has joined #openstack-infra | 14:27 | |
| pabelanger | smcginnis: zuul lost access to zookeeper a few times over the weekend, when that happens, nodepool deletes nodes under running jobs and results in SSH errors. | 14:31 |
| *** bobh has joined #openstack-infra | 14:31 | |
| *** ramishra has quit IRC | 14:31 | |
| *** jistr is now known as jistr|mtg | 14:31 | |
| *** njohnston has quit IRC | 14:33 | |
| *** florianf has quit IRC | 14:34 | |
| *** njohnston has joined #openstack-infra | 14:34 | |
| *** florianf has joined #openstack-infra | 14:34 | |
| *** njohnston has quit IRC | 14:35 | |
| *** njohnston has joined #openstack-infra | 14:36 | |
| *** kgiusti has joined #openstack-infra | 14:40 | |
| *** links has joined #openstack-infra | 14:41 | |
| *** sree has joined #openstack-infra | 14:41 | |
| *** apetrich has quit IRC | 14:42 | |
| *** eharney has joined #openstack-infra | 14:42 | |
| *** apetrich has joined #openstack-infra | 14:43 | |
| smcginnis | pabelanger: That explains those. | 14:44 |
| *** dmellado has joined #openstack-infra | 14:44 | |
| smcginnis | Looks like zuul.o.o isn't responding right now though. | 14:44 |
| smcginnis | And I had a docs job that was sitting for at least 20 minutes. | 14:44 |
| *** njohnston has quit IRC | 14:45 | |
| *** Guest87972 has quit IRC | 14:45 | |
| *** njohnston has joined #openstack-infra | 14:45 | |
| pabelanger | yah, we are out of memory again and swapping | 14:46 |
| *** sshnaidm has quit IRC | 14:46 | |
| pabelanger | infra-root: ^ | 14:46 |
| *** armaan has joined #openstack-infra | 14:46 | |
| *** sshnaidm has joined #openstack-infra | 14:47 | |
| *** jroll has quit IRC | 14:47 | |
| pabelanger | the configloader changes from corvus merged over the weekend for zuul. So, this restart should pick them up | 14:47 |
| pabelanger | let me get some coffee before saving queues and starting the restart process | 14:47 |
| *** jroll has joined #openstack-infra | 14:48 | |
| *** armaan has quit IRC | 14:48 | |
| *** links has quit IRC | 14:49 | |
| *** stevebaker has joined #openstack-infra | 14:49 | |
| pabelanger | okay, saving queues | 14:51 |
| *** lbragstad has quit IRC | 14:53 | |
| andreaf | that's why http://zuul.openstack.org is not responding to me :) | 14:54 |
| *** lbragstad has joined #openstack-infra | 14:55 | |
| *** Miouge has joined #openstack-infra | 14:55 | |
| pabelanger | okay, zuul starting up | 14:55 |
| *** amoralej|lunch is now known as amoralej | 14:56 | |
| *** sree has quit IRC | 14:57 | |
| *** sree has joined #openstack-infra | 14:58 | |
| *** yamahata has quit IRC | 14:58 | |
| *** yamahata has joined #openstack-infra | 14:58 | |
| pabelanger | okay, zuul started | 15:00 |
| *** olaph has joined #openstack-infra | 15:01 | |
| *** olaph1 has quit IRC | 15:01 | |
| *** dtantsur is now known as dtantsur|brb | 15:01 | |
| *** links has joined #openstack-infra | 15:02 | |
| pabelanger | reenqueuing changes | 15:02 |
| *** dmellado has quit IRC | 15:02 | |
| *** jistr|mtg is now known as jistr | 15:02 | |
| *** tosky_ has joined #openstack-infra | 15:02 | |
| *** sree has quit IRC | 15:03 | |
| *** tosky has quit IRC | 15:03 | |
| *** bhujay has quit IRC | 15:06 | |
| *** bhujay has joined #openstack-infra | 15:07 | |
| *** stevebaker has quit IRC | 15:07 | |
| *** caphrim007 has joined #openstack-infra | 15:07 | |
| AJaeger | andreaf: we can also add a separate docs to infra-manual about it - just to give you options ;) | 15:11 |
| andreaf | AJaeger heh ok but I thought it would be more on the QA side - maybe we need a qa-manual :D | 15:11 |
| *** caphrim007 has quit IRC | 15:12 | |
| *** oidgar has quit IRC | 15:12 | |
| andreaf | AJaeger: but perhaps it's less confusing to have all zuulv3 related info in one place | 15:12 |
| pabelanger | #status notice Zuul has been restarted to pick up latest memory fixes. Queues were saved however patches uploaded after 14:40UTC may have been missed. Please recheck if needed. | 15:13 |
| openstackstatus | pabelanger: sending notice | 15:13 |
| mordred | andreaf, AJaeger: also - as part of doing that, could we document somewhere (if we haven't already) what services devstack runs if I don't specify anything? | 15:14 |
| mordred | andreaf, AJaeger: I've got some service disabling/enabling flags in shade/sdk jobs and I'm not 100% sure if the're necessary | 15:14 |
| AJaeger | mordred: I'm not going to write it ;) Happy to review if it's in infra-manual only... | 15:14 |
| andreaf | mordred that's based on the test matrix at the moment | 15:15 |
| mordred | andreaf: oh right. it is isn't it | 15:15 |
| -openstackstatus- NOTICE: Zuul has been restarted to pick up latest memory fixes. Queues were saved however patches uploaded after 14:40UTC may have been missed. Please recheck if needed. | 15:15 | |
| andreaf | mordred we were discussing with corvus about moving that to the job definition starting from master on | 15:15 |
| AJaeger | wow, Zuul memory is at 4 GB right now. Never have seen it that low. That sounds like even more than the 50 % improvements that corvus promised | 15:16 |
| *** sree has joined #openstack-infra | 15:16 | |
| andreaf | mordred but that will require checking that we are not losing anything from not using the test matrix | 15:16 |
| mordred | andreaf: yah - I support that idea, but also yeah, that will take some care | 15:17 |
| openstackstatus | pabelanger: finished sending notice | 15:17 |
| *** armaan has joined #openstack-infra | 15:19 | |
| ykarel | openstackstatus, My patch was posted around 14:00 UTC, it looks like job are lost. Status is mentioning jobs posted after 14:40 UTC would be lost | 15:19 |
| ykarel | should i recheck or it would be auto triggered | 15:19 |
| ykarel | Patch link https://review.openstack.org/#/c/545821 | 15:20 |
| AJaeger | ykarel: check zuul.openstack.org whether it is up | 15:20 |
| mordred | ykarel: they should be auto triggered - we captured the queue state as-of 14:40 before the restart | 15:20 |
| ykarel | AJaeger, i can't find it in zuul.openstack.org | 15:20 |
| ykarel | mordred, ok will wait | 15:20 |
| *** camunoz has joined #openstack-infra | 15:21 | |
| *** sree has quit IRC | 15:21 | |
| *** agopi has joined #openstack-infra | 15:21 | |
| AJaeger | ykarel: can't find it etiher - add a "recheck" comment, it will not hurt ;) | 15:21 |
| ykarel | AJaeger, Ok :) | 15:22 |
| *** slaweq has joined #openstack-infra | 15:26 | |
| pabelanger | changes to check are still being restored, I don't remember it taking this long before after a restart | 15:26 |
| *** sree has joined #openstack-infra | 15:27 | |
| openstackgerrit | Mohammed Naser proposed openstack-infra/statusbot master: Bump python-twitter to 3.4 https://review.openstack.org/545853 | 15:28 |
| mnaser | mordred: dmsimard | 15:28 |
| mnaser | ^ :) | 15:28 |
| dmsimard | mnaser: nice coincidence | 15:29 |
| *** slaweq has quit IRC | 15:31 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Remove .json suffix from web routes https://review.openstack.org/537010 | 15:31 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Port per-change status to zuul-web https://review.openstack.org/535903 | 15:31 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Re-enable test of returning 404 on unknown tenant https://review.openstack.org/545644 | 15:31 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Add /info and /{tenant}/info route to zuul-web https://review.openstack.org/537011 | 15:31 |
| AJaeger | pabelanger: ah, that explains it - was impatient ;( | 15:32 |
| mordred | mnaser: reviewed - small change - but otherwise YAY | 15:32 |
| AJaeger | robcresswell: see https://review.openstack.org/545853 | 15:33 |
| *** dsariel has quit IRC | 15:33 | |
| openstackgerrit | Mohammed Naser proposed openstack-infra/statusbot master: Bump python-twitter to 3.4 https://review.openstack.org/545853 | 15:33 |
| mnaser | mordred: voila | 15:33 |
| *** VW has quit IRC | 15:34 | |
| *** VW has joined #openstack-infra | 15:34 | |
| *** lucasagomes is now known as lucas-hungry | 15:34 | |
| *** sree has quit IRC | 15:37 | |
| mordred | mnaser: +2 - thanks! | 15:38 |
| pabelanger | okay, check jobs just now finished being enqueued after restart | 15:43 |
| pabelanger | took 40mins to do | 15:43 |
| pabelanger | corvus: ^FYI, unsure if that is something we need to look into | 15:44 |
| *** rosmaita has joined #openstack-infra | 15:44 | |
| *** mhayden has quit IRC | 15:48 | |
| *** dsariel has joined #openstack-infra | 15:48 | |
| *** mhayden has joined #openstack-infra | 15:49 | |
| *** sree_ has joined #openstack-infra | 15:50 | |
| *** sree_ is now known as Guest18191 | 15:51 | |
| *** pblaho1 has quit IRC | 15:53 | |
| *** slaweq_ has quit IRC | 15:53 | |
| *** mhayden has quit IRC | 15:53 | |
| *** dhajare has quit IRC | 15:53 | |
| *** mhayden has joined #openstack-infra | 15:53 | |
| *** dtantsur|brb is now known as dtantsur | 15:54 | |
| *** slaweq has joined #openstack-infra | 15:54 | |
| fungi | the patch to drop the extra cached config copies did mention it was at the expense of a "slight performance hit" | 15:54 |
| fungi | hopefully that's not the problem | 15:54 |
| *** slaweq has quit IRC | 15:58 | |
| *** salv-orlando has quit IRC | 15:59 | |
| *** Guest18191 has quit IRC | 16:00 | |
| pabelanger | so far, zuul seems to be hovering at 14GB of RAM | 16:00 |
| openstackgerrit | Mike Perez proposed openstack-infra/project-config master: Set sphinx-feature-classification own core group https://review.openstack.org/545861 | 16:01 |
| clarkb | andreaf: I think on the destination side, tempest, would be better | 16:01 |
| clarkb | andreaf: people are more likely to go looking at the new thing for new docs than the old thing I think | 16:01 |
| *** caphrim007 has joined #openstack-infra | 16:01 | |
| AJaeger | config-core, could you review https://review.openstack.org/545617 to simplify our docs publishing, please? | 16:06 |
| mordred | clarkb, andreaf: yes - although I'd say in devstack not tempest- the devstack/devstack-gate stuff gets used by many non-tempest things | 16:06 |
| mordred | and finding how to configure a job to enable/disalbe/configure things for those non-tempest things in tempest would be confusing | 16:07 |
| *** pcaruana has quit IRC | 16:07 | |
| clarkb | mordred: there is thread about importing oslo_db.test on the mailing list that you may want to weigh in on. IIRC you were big driver for making tests part of the python packages | 16:08 |
| *** katkapilatova has left #openstack-infra | 16:08 | |
| andreaf | mordred: who would want to use devstack without tempest :P | 16:08 |
| andreaf | mordred: it sounds like a good idea, I'll propose something in devstack | 16:09 |
| *** lpetrut has quit IRC | 16:09 | |
| mordred | andreaf: :) | 16:10 |
| mordred | clarkb: oh good. that conversation again | 16:10 |
| fungi | AJaeger: mordred: clarkb: not sure if you saw the docs publishing on tag problem with python-storyboardclient, but it's a new one on me: http://logs.openstack.org/ce/ceab113c77bec6b218cac28665684d20d7f8ef76/release/publish-openstack-sphinx-docs-infra/bede315/job-output.txt.gz#_2018-02-16_19_57_04_366966 | 16:11 |
| mordred | clarkb: fwiw, my original stance was "they're in different places in nova and swift, we should pick one of the approaches and stick with it" | 16:11 |
| fungi | the failing role looks consistent wit the corresponding one for publish-openstack-sphinx-docs | 16:11 |
| clarkb | fungi: I think we need to use sync instead of copy there | 16:11 |
| fungi | interestingly, publish-openstack-sphinx-docs also seems to use copy | 16:12 |
| fungi | i wonder why this isn't a broader issue | 16:12 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Simplify launcher cleanup worker https://review.openstack.org/545867 | 16:12 |
| fungi | and publish-openstack-sphinx-docs-infra is succeeding for other projects | 16:14 |
| *** pcichy has quit IRC | 16:14 | |
| fungi | all the successes are in post though, not release | 16:14 |
| *** yamamoto has quit IRC | 16:16 | |
| clarkb | fungi: do the others use remote_src: true? | 16:16 |
| clarkb | reading the error message I could see how ^ changed the behavior so that it fails here | 16:17 |
| fungi | oh, perhaps not | 16:17 |
| fungi | looks like it consistently fails in release for several projects though (git-review, bindep, python-storyboardclient) | 16:17 |
| clarkb | bindep was failed in the job earlier because it couldn't handle the lack of zuul.branch | 16:18 |
| clarkb | which I fixed recently which likely allows us to see these errors now | 16:18 |
| fungi | ahh, yeah i didn't look at the logs from earlier builds, just saw in the build tracker that they were all post_failure results | 16:18 |
| fungi | i'll see if it's remote_src doing it | 16:18 |
| *** wolverineav has joined #openstack-infra | 16:19 | |
| *** VW_ has joined #openstack-infra | 16:20 | |
| *** ilpianista_ has quit IRC | 16:21 | |
| *** ilpianista_ has joined #openstack-infra | 16:21 | |
| *** masuberu has quit IRC | 16:21 | |
| *** slaweq has joined #openstack-infra | 16:23 | |
| *** r-daneel has quit IRC | 16:23 | |
| *** VW has quit IRC | 16:24 | |
| *** pblaho1 has joined #openstack-infra | 16:24 | |
| *** VW_ has quit IRC | 16:26 | |
| *** VW has joined #openstack-infra | 16:26 | |
| *** slaweq has quit IRC | 16:27 | |
| *** slaweq has joined #openstack-infra | 16:28 | |
| *** slaweq has quit IRC | 16:29 | |
| *** ykarel is now known as ykarel|away | 16:29 | |
| *** lucas-hungry is now known as lucasagomes | 16:30 | |
| *** yamamoto has joined #openstack-infra | 16:32 | |
| *** pblaho1 has quit IRC | 16:32 | |
| *** mhayden has quit IRC | 16:35 | |
| *** yamamoto has quit IRC | 16:37 | |
| smcginnis | Could I get this failed job reenqueued? http://logs.openstack.org/2f/2f55317ebc22b38f425e3bdcf2e6fff25e710155/release-post/tag-releases/2021bf7/job-output.txt.gz#_2018-02-19_15_39_01_929945 | 16:41 |
| *** jpena is now known as jpena|brb | 16:42 | |
| *** slaweq has joined #openstack-infra | 16:42 | |
| *** armaan has quit IRC | 16:44 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Use a status code to detect unknown vs. missing tenant https://review.openstack.org/545879 | 16:45 |
| *** mhayden has joined #openstack-infra | 16:46 | |
| *** bhujay has quit IRC | 16:46 | |
| *** slaweq has quit IRC | 16:48 | |
| pabelanger | looking at http://grafana.openstack.org/dashboard/db/zuul-status?from=1519055755107&to=1519058114151 we had a window of 33 minutes were zuul didn't start any builds. | 16:49 |
| dmsimard | smcginnis: I can take care of it in a few if no one else has gotten the time to get to it by then, need to step away momentarily | 16:49 |
| *** e0ne has quit IRC | 16:52 | |
| smcginnis | dmsimard: Thanks! | 16:53 |
| *** dhajare has joined #openstack-infra | 16:57 | |
| *** slaweq has joined #openstack-infra | 17:00 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Use a status code to detect unknown vs. missing tenant https://review.openstack.org/545879 | 17:01 |
| *** slaweq_ has joined #openstack-infra | 17:01 | |
| *** yamamoto has joined #openstack-infra | 17:02 | |
| pabelanger | corvus: with recent changes to zuul ConfigLoader, is it possible zuul will use more CPU? Seem zuul-scheduler is spending a lot of time doing dynamic reloads now and not launching builds | 17:02 |
| *** VW has quit IRC | 17:02 | |
| *** armaan has joined #openstack-infra | 17:03 | |
| *** VW has joined #openstack-infra | 17:03 | |
| *** eernst has joined #openstack-infra | 17:04 | |
| *** slaweq_ has quit IRC | 17:06 | |
| *** yamamoto has quit IRC | 17:06 | |
| *** andreas_s has quit IRC | 17:08 | |
| *** andreas_s has joined #openstack-infra | 17:09 | |
| *** links has quit IRC | 17:09 | |
| *** zoli is now known as zoli|gone | 17:10 | |
| *** zoli|gone is now known as zoli | 17:10 | |
| *** dmellado has joined #openstack-infra | 17:10 | |
| *** myoung is now known as myoung|bbl | 17:11 | |
| corvus | pabelanger: no | 17:12 |
| corvus | pabelanger: well, perhaps a small amount. | 17:12 |
| corvus | pabelanger: it shouldn't make a big difference though; if it is, that's unexpected. | 17:13 |
| *** andreas_s has quit IRC | 17:14 | |
| pabelanger | corvus: okay, right now I cannot explain why we haven't started any builds since 16:36UTC: http://grafana.openstack.org/dashboard/db/zuul-status and see zuul-scheduler pinned at 100% cpu, and a lot of reloads happening | 17:14 |
| pabelanger | we just started launching nodes now | 17:16 |
| corvus | pabelanger: yeah, i think you're on to something. it looks like it's taking substantially longer to build dynamic layouts | 17:17 |
| corvus | more like 1 minute each rather than 10 seconds. | 17:17 |
| *** trown is now known as trown|lunch | 17:17 | |
| corvus | 2018-02-19 17:13:00,339 DEBUG zuul.Pipeline.openstack.gate: Loading dynamic layout (phase 2) | 17:18 |
| corvus | 2018-02-19 17:14:12,894 DEBUG zuul.Pipeline.openstack.gate: Loading dynamic layout complete | 17:18 |
| corvus | pabelanger: those are the interesting log lines^ | 17:18 |
| *** felipemonteiro has joined #openstack-infra | 17:18 | |
| pabelanger | corvus: okay, that would explain why zuul taking longer to launch builds | 17:18 |
| corvus | pabelanger: and similar lines from a week ago: | 17:18 |
| corvus | 2018-02-12 17:56:27,936 DEBUG zuul.Pipeline.openstack.gate: Loading dynamic layout (phase 2) | 17:18 |
| corvus | 2018-02-12 17:56:37,707 DEBUG zuul.Pipeline.openstack.gate: Loading dynamic layout complete | 17:18 |
| pabelanger | yah | 17:19 |
| clarkb | possible that also explains the test timeouts that we ran into trying to get that stack merged? | 17:19 |
| corvus | could be | 17:19 |
| pabelanger | clarkb: good point | 17:19 |
| *** jpena|brb is now known as jpena | 17:22 | |
| *** slaweq_ has joined #openstack-infra | 17:25 | |
| *** dsariel has quit IRC | 17:27 | |
| *** dhajare has quit IRC | 17:29 | |
| corvus | i'll look into speeding that up shortly. | 17:29 |
| *** slaweq_ has quit IRC | 17:29 | |
| *** lpetrut has joined #openstack-infra | 17:35 | |
| *** lpetrut has quit IRC | 17:36 | |
| *** lpetrut has joined #openstack-infra | 17:36 | |
| AJaeger | fungi, haven't seen this before with the tags - and indeed remote_src could be the difference. Why did we add that one? | 17:37 |
| *** ijw has joined #openstack-infra | 17:37 | |
| *** ijw has quit IRC | 17:38 | |
| *** ijw has joined #openstack-infra | 17:39 | |
| AJaeger | mordred: the remote_src is there from the initial commit - any idea why it was done? | 17:39 |
| *** dsariel has joined #openstack-infra | 17:40 | |
| pabelanger | clarkb: did you want to review https://review.openstack.org/545314/ again, glean patch to vendor distro | 17:40 |
| *** coolsvap has quit IRC | 17:41 | |
| AJaeger | fungi, mordred, clarkb, ah, see why we used the remote_src - we're renaming basically | 17:41 |
| *** shardy has quit IRC | 17:41 | |
| AJaeger | fungi, mordred, clarkb, and we do run publish-openstack-sphinx-docs-infra in release - but no other docs job, so never noticed ;( | 17:44 |
| clarkb | pabelanger: done. I've not approved it beacuse I think we should get more eyeballs on that to make sure people are comfortable vendoring that code | 17:45 |
| pabelanger | clarkb: wfm | 17:45 |
| pabelanger | mordred: mind looking at https://review.openstack.org/545314/ too | 17:46 |
| *** slaweq_ has joined #openstack-infra | 17:46 | |
| AJaeger | fungi, mordred, clarkb, unfortunately no direct idea on how to fix - besides writing a small shell script to do these steps. Can't help with that today | 17:46 |
| clarkb | we might be able to use the sync module instaed of copy? | 17:47 |
| *** yamamoto has joined #openstack-infra | 17:47 | |
| AJaeger | clarkb: sorry, didn't dig into that one yet. bbl | 17:47 |
| *** VW has quit IRC | 17:49 | |
| *** jpich has quit IRC | 17:49 | |
| *** slaweq_ has quit IRC | 17:51 | |
| *** yamamoto has quit IRC | 17:52 | |
| *** sshnaidm is now known as sshnaidm|off | 17:54 | |
| *** felipemonteiro has quit IRC | 17:54 | |
| *** felipemonteiro has joined #openstack-infra | 17:54 | |
| jlvillal | clarkb, I thought I would remind you on the gerritbot patch: https://review.openstack.org/#/c/545128/ You reviewed the two children of it. | 17:55 |
| jlvillal | It already has one +2. It is adding more unit tests to gerritbot. | 17:55 |
| clarkb | jlvillal: thanks | 17:55 |
| jlvillal | clarkb, Thank you | 17:55 |
| openstackgerrit | Monty Taylor proposed openstack-infra/puppet-zuul master: Update status backup to remove .json suffix https://review.openstack.org/545938 | 17:56 |
| jlvillal | clarkb, On a related note. Can I propose myself as a gerritbot core? And then wait for all they nay votes to come in? :) | 17:58 |
| *** felipemonteiro has quit IRC | 17:59 | |
| clarkb | if people are interested in resurrecting the bot work thats fine by me, but it might be better to start with the supybot replacement spec? | 17:59 |
| *** derekh has quit IRC | 18:00 | |
| jlvillal | clarkb, Hmm. I hadn't seen that. | 18:00 |
| *** e0ne has joined #openstack-infra | 18:01 | |
| * jlvillal has to go to a dentist appointment now | 18:01 | |
| *** ralonsoh has quit IRC | 18:02 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/puppet-zuul master: Update status backup to remove .json suffix https://review.openstack.org/545938 | 18:03 |
| openstackgerrit | Monty Taylor proposed openstack-infra/puppet-zuul master: Provide a backwards-compat rewrite rule for status.json https://review.openstack.org/545939 | 18:03 |
| *** tosky_ is now known as tosky | 18:04 | |
| clarkb | jlvillal: for when you get back does that just continuously try to connect to an non existant gerrit server? we might want some way of avoiding that to make the tests a little less noisy if you have to debug them? | 18:04 |
| mordred | infra-root: puppet changes in support of the zuul change to remove .json suffixes - and a backward compat one (did that one separate in case people don't like it) | 18:04 |
| mordred | infra-root: sorry ^^ 545938 and 545939 ... we should not land them until we've landed the zuul changes and restarted ... | 18:05 |
| *** armaan has quit IRC | 18:05 | |
| clarkb | mordred: why not support both /status and /status.json in apache then you can merge them in a less strict ordering | 18:07 |
| mordred | clarkb: I was going to do that - but I'd still need to know what to rewrite them _to_ | 18:07 |
| *** slaweq_ has joined #openstack-infra | 18:07 | |
| *** hongbin has joined #openstack-infra | 18:07 | |
| *** tosky has quit IRC | 18:07 | |
| clarkb | status.json -> status.json and /status to /status? | 18:08 |
| openstackgerrit | Merged openstack-infra/project-config master: [magnum] Add irc notifications for queens https://review.openstack.org/545786 | 18:08 |
| mordred | clarkb: oh - I get what you're saying, yeah | 18:08 |
| clarkb | then you only have to update the backups url when you are ready but zuul itself should transition seamlessly | 18:08 |
| *** wolverineav has quit IRC | 18:08 | |
| mordred | clarkb: ++ great idea | 18:08 |
| *** salv-orlando has joined #openstack-infra | 18:08 | |
| clarkb | also I'm checking one more thing | 18:08 |
| clarkb | so wait a sec on new patchset maybe | 18:08 |
| *** yamamoto has joined #openstack-infra | 18:08 | |
| *** yamamoto has quit IRC | 18:08 | |
| mordred | kk | 18:09 |
| clarkb | for RewriteRule ^/status.* - [F] you might want to make that more specific? ^/status(.json)? ? | 18:09 |
| clarkb | I don't actually knwo if that is valid rewrite rule regex though | 18:09 |
| mordred | clarkb: may just two rules | 18:10 |
| clarkb | I guess since we aren't mangling the path the .* should be relatively save | 18:10 |
| clarkb | ya two rules is nice and explicit too | 18:10 |
| *** slaweq_ has quit IRC | 18:12 | |
| *** hashar_ is now known as DockerIsA_Sin | 18:12 | |
| jlvillal | clarkb, It never connects to the gerrit server as far as the unit tests. At least I don't believe it does. | 18:12 |
| jlvillal | clarkb, We never execute the 'run' method. Or whatever the method for threading uses to start | 18:13 |
| clarkb | oh gotcha | 18:13 |
| openstackgerrit | Monty Taylor proposed openstack-infra/puppet-zuul master: Update apache and backups for json suffix removal https://review.openstack.org/545938 | 18:17 |
| openstackgerrit | Monty Taylor proposed openstack-infra/puppet-zuul master: Remove backwards compat rewrite rules https://review.openstack.org/545939 | 18:17 |
| openstackgerrit | Monty Taylor proposed openstack-infra/puppet-zuul master: Remove old status.json rewrite rule https://review.openstack.org/545942 | 18:17 |
| mordred | clarkb: ^^ how's that look now? | 18:17 |
| *** rwsu has quit IRC | 18:17 | |
| *** openstackgerrit has quit IRC | 18:18 | |
| clarkb | mordred: ya that looks good. Not sure if you want to update the backups source just yet but also thats a convenience tool so not end of the world if we just have it updated then zuul catches up with it later | 18:19 |
| *** jpena is now known as jpena|away | 18:19 | |
| dmsimard | clarkb: it turns out that we don't really need those to be available publicly anyway | 18:21 |
| clarkb | ya I'm not super concerned about it though we'll be restarting zuul soon (I hope) to address the performance issue | 18:21 |
| dmsimard | clarkb: I didn't know using file:// was a thing ( https://review.openstack.org/#/c/536622/ ) | 18:21 |
| clarkb | however in thatcase wei'll manually grab it | 18:21 |
| *** e0ne has quit IRC | 18:21 | |
| andreaf | clarkb corvus mordred btw is there something already in place to render README from roles into web doc pages? | 18:22 |
| mordred | andreaf: you mean like https://docs.openstack.org/infra/openstack-zuul-jobs/roles.html ? | 18:23 |
| dmsimard | clarkb: the thing we have to be careful about is mostly the rewrite rules, yeah.. I'm not really good at those. Do we have a reliable way of testing them before landing that ? | 18:23 |
| pabelanger | Yah, I've used file:// a few times now to load status.json from disk | 18:23 |
| andreaf | mordred: yeah something like that :) | 18:23 |
| mordred | andreaf: yes! :) | 18:24 |
| andreaf | mordred: we should produce that out of devstack and tempest roles | 18:24 |
| *** VW has joined #openstack-infra | 18:24 | |
| andreaf | mordred: so we can link from other documents into the roles docs as well | 18:24 |
| clarkb | dmsimard: tahts why I suggested adding them rather than swapping them | 18:24 |
| clarkb | dmsimard: we can get that in place then you can test it if you want | 18:24 |
| mordred | andreaf: http://git.openstack.org/cgit/openstack-infra/zuul-sphinx/ is the sphinx plugin | 18:24 |
| clarkb | (you shoudl see the rewrite in the access log?) | 18:24 |
| andreaf | mordred ack thx | 18:25 |
| mordred | andreaf: we've been discussing having a consolidated set of live docs in the dashboard - so that there can be a single browseable source of docs for all of the jobs | 18:26 |
| dmsimard | clarkb: maybe we should have a zuul-dev like we have a logs-dev | 18:26 |
| *** stevebaker has joined #openstack-infra | 18:26 | |
| mordred | andreaf: http://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/doc/source/roles.rst is using that sphinx plugin in the zuul-jobs repo ... along with http://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/doc/source/conf.py#n22 | 18:27 |
| clarkb | dmsimard: we've had one before but deploying from master like we do I think makes it useful only in some cases like testing against a nwe gerrit (which you can easily deploy a zuul for anyways)? | 18:27 |
| pabelanger | I'm off to run a few errands in town | 18:27 |
| fungi | just a heads up, construction work at the end of our driveway has apparently taken down our broadband for the past two hours now... i'm catching up via wireless modem for now | 18:27 |
| dmsimard | clarkb: well there's no delta between logs/logs-dev, right? just the vhost config is different | 18:28 |
| dmsimard | it's not a separate server or anything | 18:28 |
| *** jrist is now known as jrist-afk | 18:28 | |
| *** jrist-afk is now known as jrist | 18:28 | |
| clarkb | dmsimard: yes, btu in this case you actually need zuul to deploy unmerged code to test that the rewrites are working properly end to end | 18:28 |
| clarkb | dmsimard: so running a second vhost doesn't help that much | 18:29 |
| clarkb | especialyl since we can do this in an additive fashion for backward and forward compat | 18:29 |
| dmsimard | ah, bleh | 18:29 |
| *** rwsu has joined #openstack-infra | 18:29 | |
| fungi | clarkb: AJaeger: i did dig into the docs job failure on release and determined that it only sets remote_src in the tag playbook, which would explain why it's not broken in post (in effort to rename it, as you divined per scrollback) | 18:30 |
| clarkb | the problem here is less that we want to modify the vhost config and more that zuul's api is changing | 18:30 |
| dmsimard | smcginnis: did anyone hook you up? I have time now | 18:30 |
| *** tesseract has quit IRC | 18:30 | |
| *** openstackgerrit has joined #openstack-infra | 18:31 | |
| openstackgerrit | Merged openstack-infra/gerritbot master: Add more unit tests including testing of Gerrit() class https://review.openstack.org/545128 | 18:31 |
| openstackgerrit | Merged openstack-infra/gerritbot master: Test Gerrit message filtering https://review.openstack.org/545381 | 18:31 |
| openstackgerrit | Merged openstack-infra/gerritbot master: Support regexes in channel config yaml https://review.openstack.org/499367 | 18:31 |
| *** DockerIsA_Sin is now known as hashar | 18:33 | |
| openstackgerrit | Merged openstack-infra/zuul master: Remove .json suffix from web routes https://review.openstack.org/537010 | 18:34 |
| mnaser | ok is my irc client messed up | 18:36 |
| mnaser | or is the topic of #openstack one for a kolla meeting | 18:36 |
| dmsimard | mnaser: it is | 18:36 |
| mnaser | does anyone have the original topic | 18:36 |
| clarkb | mnaser: its probably in the eavesdrop channel logs | 18:36 |
| mnaser | im gonna look for it | 18:37 |
| *** eernst has quit IRC | 18:37 | |
| openstackgerrit | Andrey Kurilin proposed openstack-infra/project-config master: [rally] Add a separate repository for OpenStack plugins https://review.openstack.org/545824 | 18:37 |
| openstackgerrit | Andrey Kurilin proposed openstack-infra/project-config master: [rally] Configure CI for a new repo https://review.openstack.org/545828 | 18:37 |
| dmsimard | mnaser: probably someone that started a meeting there, meetbot doesn't enforce channels and it looks like tonyb abandoned https://review.openstack.org/#/c/246695/ | 18:37 |
| andreaf | mordred: thanks - that's really straight forward https://review.openstack.org/545952 | 18:37 |
| mnaser | dmsimard: #endmeeting didnt restore it | 18:37 |
| clarkb | if the bot dies afterwards it loses that state :/ | 18:38 |
| dmsimard | mnaser: when did the meeting start ? | 18:38 |
| mnaser | dmsimard: thats what im trying to figure out, im up to feb 1st | 18:38 |
| *** spligak has joined #openstack-infra | 18:38 | |
| mnaser | ok found it | 18:38 |
| dmsimard | yeah there's probably no meeting to cancel there, let's just put back the topic | 18:38 |
| mnaser | http://eavesdrop.openstack.org/irclogs/%23openstack/%23openstack.2018-01-31.log.html#t2018-01-31T17:44:57 | 18:39 |
| *** r-daneel has joined #openstack-infra | 18:39 | |
| mnaser | im gonna keep looking for the original topic | 18:39 |
| *** trown|lunch is now known as trown | 18:39 | |
| dmsimard | hmmm there might be a history in chanserv, let me check | 18:39 |
| mnaser | dmsimard: got it | 18:39 |
| mnaser | http://eavesdrop.openstack.org/irclogs/%23openstack/%23openstack.2018-01-29.log.html#t2018-01-29T16:18:52 | 18:39 |
| mnaser | and fixed it | 18:39 |
| dmsimard | cool | 18:40 |
| mnaser | back to regular programming | 18:40 |
| openstackgerrit | Merged openstack-infra/zuul master: Port per-change status to zuul-web https://review.openstack.org/535903 | 18:40 |
| openstackgerrit | Merged openstack-infra/zuul master: Re-enable test of returning 404 on unknown tenant https://review.openstack.org/545644 | 18:40 |
| *** e0ne has joined #openstack-infra | 18:40 | |
| openstackgerrit | Merged openstack-infra/zuul master: Add /info and /{tenant}/info route to zuul-web https://review.openstack.org/537011 | 18:40 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Avoid creating extra project schemas https://review.openstack.org/545953 | 18:45 |
| *** tosky has joined #openstack-infra | 18:46 | |
| *** rwsu has quit IRC | 18:46 | |
| dmsimard | andreaf: I know I'm late but I'm slowly catching up to my backlog... I'm looking into the failures you reported the other day with role inclusion: https://review.openstack.org/#/c/530642/12/playbooks/devstack-tempest.yaml | 18:50 |
| *** suhdood has joined #openstack-infra | 18:52 | |
| *** edmondsw has quit IRC | 18:55 | |
| *** edmondsw has joined #openstack-infra | 18:56 | |
| *** lucasagomes is now known as lucas-brb | 18:57 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Avoid creating extra project schemas https://review.openstack.org/545953 | 18:58 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Allow test_playbook to run long https://review.openstack.org/545955 | 18:58 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Fix training-labs publishing https://review.openstack.org/545956 | 19:00 |
| *** edmondsw has quit IRC | 19:00 | |
| *** pcichy has joined #openstack-infra | 19:01 | |
| *** rwsu has joined #openstack-infra | 19:01 | |
| *** felipemonteiro has joined #openstack-infra | 19:02 | |
| *** wolverineav has joined #openstack-infra | 19:02 | |
| AJaeger | mordred: as publishing script master, could you review the above, please? My earlier conversion was broken ^ | 19:03 |
| *** harlowja has joined #openstack-infra | 19:03 | |
| smcginnis | dmsimard: I have not seen anything, so I believe it still needs to be done/ | 19:04 |
| smcginnis | cc: fungi ^ | 19:04 |
| fungi | smcginnis: i missed it in here, same one from when you highlighted me in #openstack-release a couple hours ago? | 19:04 |
| fungi | smcginnis: http://logs.openstack.org/2f/2f55317ebc22b38f425e3bdcf2e6fff25e710155/release-post/tag-releases/2021bf7/ ? | 19:05 |
| *** gfidente is now known as gfidente|afk | 19:05 | |
| *** suhdood has quit IRC | 19:06 | |
| smcginnis | fungi, dmsimard: Yep, that'd be the one. | 19:06 |
| smcginnis | fungi: Not sure why our setting of the timeouts and limits don't appear to be working for that retry to work. | 19:06 |
| *** nhicher has quit IRC | 19:06 | |
| *** myoung|bbl is now known as myoung | 19:07 | |
| *** nhicher has joined #openstack-infra | 19:07 | |
| fungi | dmsimard: you want to go ahead and reenqueue that? the workstation with my openstack ssh keys is still offline due to roadwork | 19:07 |
| dmsimard | fungi: yup, I'll take care of it. | 19:07 |
| fungi | thanks!!! | 19:07 |
| dmsimard | fungi: your workstation, is that the DIY device ? :) | 19:07 |
| dmsimard | (I say device because I'm not sure what it's supposed to be called.... a laptop perhaps?) | 19:08 |
| smcginnis | Thanks for the help! | 19:08 |
| smcginnis | More like a kneetop. | 19:08 |
| fungi | nope, beefy machine with a mechanical switch keyboard and multiple large monitors | 19:08 |
| fungi | what i normally work from at home | 19:08 |
| fungi | the hackbook is a travel device and/or for knocking around in other rooms of the house | 19:09 |
| dmsimard | fungi: wow, multiple large monitors.. you allow yourself more than 22x80 lines ? :) | 19:09 |
| *** yamamoto has joined #openstack-infra | 19:09 | |
| smcginnis | fungi: I'm assuming the beefy machine is as similarly DIY constructed. ;) | 19:10 |
| *** dtantsur is now known as dtantsur|afk | 19:10 | |
| *** esberglu has quit IRC | 19:11 | |
| fungi | dmsimard: i allow myself 80x25 geometry in a dozen regions of one monitor and use the other for graphical applications | 19:12 |
| dmsimard | fungi: nice :) | 19:13 |
| *** ykarel|away has quit IRC | 19:13 | |
| fungi | smcginnis: not quite as hacked together. it's a barebones gigabyte brix mini box bolted to the back of a medical equipment stand and connected to 36" panels in portrait orientation via a displayport multi-streaming hub | 19:14 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Revert "Revert "Add zuul.{pipeline,nodepool.provider,executor.hostname} to job header"" https://review.openstack.org/514489 | 19:14 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/zuul-jobs master: Add Ansible version to job header https://review.openstack.org/532304 | 19:14 |
| fungi | er, not 36", 27" | 19:14 |
| *** jpena|away is now known as jpena|off | 19:14 | |
| smcginnis | fungi: Ooh, nice choice with the 36"... ah, that's less impressive but equally cool. :) | 19:15 |
| smcginnis | I used to have several monitors, now just one QHD 30-some inch curved one. | 19:15 |
| *** slaweq_ has joined #openstack-infra | 19:16 | |
| fungi | putting two in portrait side-by-side makes for a nice 2880x2560 workspace (and then i have a third 1280x1024 in landscape orientation off on an adjacent wall) | 19:16 |
| dmsimard | you are making my home office feel inadequate lol | 19:16 |
| smcginnis | haha | 19:16 |
| * mnaser still uses a single monitor | 19:17 | |
| clarkb | mnaser: me too. I find it easier to manage and understand | 19:17 |
| fungi | it takes up very little room... i got the portable equipment stand from a hospital supply | 19:17 |
| mnaser | clarkb: i am unable to manage multiple workspaces or use multiple screens efficently | 19:18 |
| smcginnis | fungi: Stand up? | 19:18 |
| mnaser | one just ends up being fairly idle | 19:18 |
| clarkb | also slightly better ergonomics as everything is right in front of me | 19:18 |
| dmsimard | smcginnis: going to requeue this: zuul enqueue-ref --tenant=openstack --trigger=gerrit --pipeline=release-post --project=openstack/releases --ref=refs/heads/master --newrev=2f55317ebc22b38f425e3bdcf2e6fff25e710155 | 19:18 |
| fungi | smcginnis: i usually use it standing up but it's one of those counterweighted adjustable kind made for sitting or standing | 19:18 |
| smcginnis | dmsimard: That does look correct. | 19:18 |
| fungi | so ~zero force needed for raising and lowering | 19:19 |
| smcginnis | fungi: Had a desk attachable one like that when I had a cubical. | 19:19 |
| fungi | though a high-end industrial anti-fatigue mat is a must in my opinion | 19:19 |
| fungi | since i tend to stand in front of it 10-12 hours a day | 19:20 |
| smcginnis | I remember going through some adjustment period getting used to standing so much. | 19:20 |
| dmsimard | I have a USB switch in my office to switch everything from my "PC" to my work laptop which has a dock. So really just two screens, my primary screen acts as a USB hub for everything including a USB sound card so I can use my speakers seamlessly between both. That was the important part for me. | 19:21 |
| *** slaweq_ has quit IRC | 19:21 | |
| dmsimard | my secondary screen is mostly the idle stuff I want to keep an eye on | 19:21 |
| dmsimard | Is zuul.openstack.org not loading for anyone else ? | 19:22 |
| smcginnis | I used to keep a small separate screen up for email and such. That was kind of useful at times. | 19:22 |
| *** ijw has quit IRC | 19:23 | |
| smcginnis | Nope | 19:23 |
| dmsimard | infra-root: ^ ? | 19:23 |
| dmsimard | It loaded for me *just now* with an error status.json not found | 19:23 |
| dmsimard | Did we merge any of that already mordred ? | 19:24 |
| clarkb | I got status: Not found | 19:24 |
| *** alexchadin has joined #openstack-infra | 19:24 | |
| clarkb | I think the zuul stuff must've merged which updated the js | 19:24 |
| clarkb | but we need to restart the web service to use the new api | 19:24 |
| dmsimard | smcginnis: I'm not sure if the requeue was successful, we'll need to check after we figure this one out | 19:24 |
| clarkb | mordred: ^ | 19:24 |
| smcginnis | dmsimard: ack | 19:25 |
| clarkb | dmsimard: you should be able to fetch the status.json directly http://zuul.openstack.org/status.json is accessable to me | 19:25 |
| mnaser | it's loading fine for me, status.json is accessible to me too | 19:25 |
| mnaser | unless we have cached versions | 19:25 |
| dmsimard | mnaser: clear your cache ? | 19:25 |
| AJaeger | clearing my cache *broke* it ;( | 19:26 |
| clarkb | mnaser: ya I bet you have cached the old js | 19:26 |
| clarkb | my hunch is new js is broken until we update the service | 19:26 |
| *** yamamoto has quit IRC | 19:26 | |
| clarkb | but old js will continue tow ork until refreshed | 19:26 |
| mnaser | i can investigate a tad | 19:26 |
| AJaeger | mnaser: AFAIU corvus will restart zuul soon - and then it should all work again... | 19:27 |
| dmsimard | smcginnis: there's *something* in the queue for 2f55317ebc22b38f425e3bdcf2e6fff25e7101 (which is what we requeued) from reading the raw status.json but other than that I dunno. | 19:27 |
| smcginnis | dmsimard: OK, I'll watch for progress. | 19:27 |
| mnaser | AJaeger: okay cool, what clarkb does seem to make sense though based on the changes im seeing | 19:28 |
| *** alexchadin has quit IRC | 19:28 | |
| AJaeger | config-core, could you review https://review.openstack.org/545617 to simplify our docs publishing, please? | 19:29 |
| *** esberglu has joined #openstack-infra | 19:29 | |
| clarkb | the new code is installed on zuul01.o.o | 19:32 |
| clarkb | so in theory we just have to systemctl restart zuul-web | 19:32 |
| clarkb | but I'm not sure if we have to restart any of the other services too | 19:32 |
| clarkb | mordred: corvus ^ | 19:32 |
| *** dsariel has quit IRC | 19:33 | |
| *** alexchadin has joined #openstack-infra | 19:33 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Fix publishing tagged docs https://review.openstack.org/545962 | 19:33 |
| AJaeger | clarkb, fungi, I found a minute to fix the tagged docs publishing ^ | 19:33 |
| *** mriedem has quit IRC | 19:34 | |
| mnaser | smcginnis: json parsing for you - http://zuul.openstack.org/stream.html?uuid=d7a553d426364ede9ccc6ef1a47d391e&logfile=console.log | 19:34 |
| mnaser | smcginnis: that's the tag-releases job | 19:34 |
| *** mriedem has joined #openstack-infra | 19:35 | |
| AJaeger | clarkb, fungi, didn't use synchronize after finding in http://git.openstack.org/cgit/openstack-infra/zuul-jobs/tree/roles/stage-output/tasks/main.yaml#n73 "remote_src copy does not work recursively, synchronise is restricted by zuul, using command" | 19:35 |
| mnaser | http://paste.openstack.org/show/677789/ | 19:35 |
| mnaser | jq is fun | 19:35 |
| *** alexchadin has quit IRC | 19:36 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Resolve Ansible variable precedence issue with include_vars https://review.openstack.org/536615 | 19:36 |
| mnaser | clarkb: i believe zuul-web restart should be enough given this is this is the patch that affected it - https://review.openstack.org/#/c/537010 | 19:37 |
| clarkb | mnaser: ya I'm more worried there are other changes since the last restart that may require a more complete restart | 19:38 |
| mnaser | clarkb: ah you're right that's a good point | 19:38 |
| clarkb | I know the upcoming performance fix (which hasn't merged yet) will require the scheduler to restart at least | 19:38 |
| * mnaser should get around doing the filtering work he wanted to do for the zuul status page | 19:38 | |
| clarkb | I suppose zuul-web is effectively broken for our users right now so restarting it and it not working isn't the worst thing | 19:39 |
| mnaser | clarkb: or if zuul-web sits behind a web server, you could probably do a small rewrite rule but that might be too much work for now :) | 19:40 |
| AJaeger | mnaser: time for a quick cleanup review, please? https://review.openstack.org/#/c/543330/ | 19:40 |
| vkmc | hey folks, I'm working on migrating to zuul v3 in manila... where should we locate zuul.yaml? manila or manila-tempest-plugin repo? | 19:41 |
| mnaser | AJaeger: done, thank you! | 19:41 |
| AJaeger | mnaser: thanks... | 19:41 |
| clarkb | andreaf: ^ do you have an opinion on vkmc's question? | 19:42 |
| clarkb | infra-root any other opinions on just restarting zuul-web for now? | 19:44 |
| fungi | restarting zuul-web seems like it should be non-impacting | 19:45 |
| fungi | your mention of recently-landed changes... you're concerned it won't actually start without restarting the scheduler on newer code too? | 19:47 |
| clarkb | fungi: ya or fail to function entirely whereas for now you can still get status.json if you know to look for it | 19:47 |
| *** suhdood has joined #openstack-infra | 19:48 | |
| fungi | got it. worth the risk, i think, given it only works with locally-cached static files for now | 19:48 |
| fungi | and we'll hopefully be restarting the scheduler again shortly anyway | 19:49 |
| clarkb | right thats why I'm cautious to break ability to get the status.json | 19:49 |
| clarkb | since we'll want to preserve queues | 19:49 |
| fungi | oh, that's a very good point | 19:49 |
| *** olaph1 has joined #openstack-infra | 19:50 | |
| *** olaph has quit IRC | 19:51 | |
| *** slaweq_ has joined #openstack-infra | 19:52 | |
| clarkb | corvus: ^ tl;dr is new js is deployed which wants /status but zuul-web is running on old code only serving status.json | 19:54 |
| clarkb | corvus: do you think we should hold off on restarting zuul-web to avoid issues with the upcoming restart for performance fix? | 19:54 |
| corvus | clarkb: hrm... i *think* it should be mostly okay to restart zuul-web now | 19:55 |
| corvus | i think the only things it may disagree with the scheduler about are nonexistent or not fully loaded tenants | 19:56 |
| clarkb | corvus: and so won't affect our ability to get queues for the existing openstack tenant? | 19:56 |
| corvus | clarkb: how about this | 19:56 |
| corvus | clarkb: maybe go ahead and grab a queue dump, then restart web | 19:57 |
| *** slaweq_ has quit IRC | 19:57 | |
| *** Goneri has quit IRC | 19:57 | |
| corvus | clarkb: if it breaks our ability to get another queue dump afterwords, we'll go ahead and restart scheduler with my change manually applied | 19:57 |
| fungi | that seems like a reasonable compromise | 19:57 |
| clarkb | ok let me try that | 19:57 |
| dmsimard | +1 | 19:58 |
| clarkb | heh I have to manually edit the script to use the old url | 19:58 |
| clarkb | so this might take a minute | 19:58 |
| *** lucas-brb is now known as lucasagomes | 19:58 | |
| clarkb | restarting zuul-web now | 20:00 |
| clarkb | corvus: ok its still not working | 20:01 |
| clarkb | I'm going to hit it directly rather than through apache to test | 20:01 |
| clarkb | curling localhost directly works | 20:01 |
| *** VW has quit IRC | 20:01 | |
| *** hongbin_ has joined #openstack-infra | 20:01 | |
| clarkb | so I don't think we need to emergency restart the scheduler | 20:01 |
| corvus | did we land the puppet changes? | 20:02 |
| *** VW has joined #openstack-infra | 20:02 | |
| clarkb | oh ha I should've checked that | 20:02 |
| clarkb | I assumed so because of the js change but I guess those may not be tied together | 20:02 |
| clarkb | ya puppet doesn't seem to have applied the apache changes yet | 20:03 |
| corvus | no, we're in a weird place with js installation right now until mordred is done with the webpack/yarn stuff | 20:03 |
| *** hongbin has quit IRC | 20:03 | |
| clarkb | ok so, apply puppet change to apache and all should be well | 20:03 |
| corvus | looks like it's https://review.openstack.org/545938 and child | 20:03 |
| clarkb | I think we got the order on these changes backwards | 20:04 |
| clarkb | we should've updated apache then updated zuul | 20:04 |
| *** alexchadin has joined #openstack-infra | 20:04 | |
| *** alexchadin has quit IRC | 20:04 | |
| corvus | -1 on 545938 | 20:05 |
| corvus | i have to go grab lunch now; maybe someone could push up a fix for that and go ahead and approve it and child? | 20:06 |
| clarkb | I'll push a new patchset I don't think the child is urgent is it? | 20:07 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-zuul master: Update apache and backups for json suffix removal https://review.openstack.org/545938 | 20:08 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-zuul master: Remove backwards compat rewrite rules https://review.openstack.org/545939 | 20:08 |
| openstackgerrit | Clark Boylan proposed openstack-infra/puppet-zuul master: Remove old status.json rewrite rule https://review.openstack.org/545942 | 20:08 |
| clarkb | new patchsets ^ | 20:08 |
| *** olaph has joined #openstack-infra | 20:09 | |
| *** olaph1 has quit IRC | 20:10 | |
| *** suhdood_ has joined #openstack-infra | 20:11 | |
| *** suhdood has quit IRC | 20:13 | |
| *** dave-mccowan has quit IRC | 20:14 | |
| *** lucasagomes is now known as lucas-afk | 20:14 | |
| clarkb | while waiting for reviews I too am going to grab lunch | 20:16 |
| clarkb | infra-root can you review ^ 545938 in particular | 20:16 |
| *** wolverineav has quit IRC | 20:19 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove legacy nodepool boot jobs https://review.openstack.org/543330 | 20:22 |
| *** amoralej is now known as amoralej|off | 20:23 | |
| *** janki has quit IRC | 20:28 | |
| mordred | corvus, clarkb: oh goodie - looks like I missed a bit of fun ... did we restart zuul before those puppet changes were live? | 20:28 |
| fungi | infra-root: should we zuul enqueue the zuul performance fix and puppet-zuul vhost fix into the gate pipeline so we can get things going without having to wait for backlogged node allocations in check? | 20:28 |
| fungi | mordred: no zuul restart yet | 20:28 |
| corvus | mordred: no, it's just the js changes went in to effect immediately because of the way we serve them currently | 20:29 |
| corvus | it's not worth worrying about since it's all about to change | 20:29 |
| mordred | corvus: OH. duh. | 20:29 |
| fungi | mordred: but the js gets deployed by puppet when the zuul repo updates | 20:29 |
| corvus | fungi: i'd be in favor of that | 20:29 |
| mordred | fungi: me too | 20:30 |
| *** edmondsw has joined #openstack-infra | 20:30 | |
| *** armaan has joined #openstack-infra | 20:31 | |
| dmsimard | I left comments in https://review.openstack.org/#/c/545938/ but nothing preventing it from merging | 20:31 |
| *** armaan has quit IRC | 20:31 | |
| dmsimard | Please do look at them though so we can maybe address some of them in follow up patches | 20:31 |
| corvus | dmsimard: which specifically would you like to see addressed? | 20:32 |
| *** felipemonteiro_ has joined #openstack-infra | 20:32 | |
| dmsimard | corvus: there's cleanliness nits (I can send a patch for that I guess) but otherwise questions about some of the rewrite rules | 20:33 |
| mordred | dmsimard: I'm not sure about your first question ... why would the [F] cause the second one to not work? | 20:34 |
| dansmith | is zuul known to be unreachable again? | 20:34 |
| corvus | dansmith: fix in progress | 20:34 |
| dansmith | ack | 20:34 |
| fungi | dansmith: specifically, just the status dashboard/json | 20:34 |
| mordred | dansmith: speaking of - do you still have a tool that fetches status.json directly? | 20:34 |
| dmsimard | mordred: the condition is the referer, not the requested URL | 20:34 |
| dansmith | fungi: ah okay | 20:35 |
| dansmith | mordred: I do | 20:35 |
| dmsimard | mordred: you're checking the referer once, then rewrite whatever requested url to status (or status.json) with a HTTP 403 .. and then you're checking referer a second time ? not sure if you see what I mean, unless I'm misreading it | 20:35 |
| dmsimard | checking the referer twice doesn't seem right | 20:35 |
| mordred | dansmith: k. so - we're in the process of stripping .json suffixes from the api (although status.json will stay around a bit longer for transition) | 20:35 |
| *** felipemonteiro has quit IRC | 20:35 | |
| dansmith | mordred: okay | 20:35 |
| mordred | dansmith: so you might want to s/status.json/status/ in a bit - once we do the next zuul-web restart (so whenever you see the status page again it should be safe to start pulling from /status) | 20:36 |
| dansmith | mordred: okay, is there some reason for that change? | 20:36 |
| dansmith | I thought it was hip to use .ext on dynamic resources like that | 20:37 |
| mriedem | AJaeger: i'm not sure what i should do with this bug nova is running legacy tempest-full and zuulv3 tempest-full on all changes, which is pretty wasteful https://review.openstack.org/#/c/542484/ | 20:37 |
| mriedem | s/bug/because/ | 20:37 |
| *** dsariel has joined #openstack-infra | 20:37 | |
| mordred | dansmith: it's more related to us growing a much larger rest api in v3 - and it was starting to look weird | 20:38 |
| fungi | dansmith: part of it is because it's going to start accepting parameters and, yeah, aesthetics related to that i suppose | 20:38 |
| mordred | dansmith: https://review.openstack.org/#/c/537010/ <-- there's the patch that landed for it | 20:38 |
| dansmith | okay | 20:38 |
| fungi | things like /status/change/123456 | 20:38 |
| fungi | might look stranger as /status.json/change/123456 | 20:39 |
| mordred | status.json/change/542484,2.json seemed like a weird url, but /status.json and /status/change/542484,2.json did too | 20:39 |
| mordred | fungi: jinx :) | 20:39 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/puppet-zuul master: Nitfix: Simplify if/else condition for status.json https://review.openstack.org/545977 | 20:39 |
| dmsimard | ^ | 20:39 |
| dmsimard | brb | 20:39 |
| mnaser | dang | 20:40 |
| mnaser | my cached working-but-old js is gone :< | 20:40 |
| corvus | heh, "nobody hit reload" :) | 20:41 |
| AJaeger | mriedem: let's ask corvus and other config-cores for help on 542484. I don't understand why you see those jobs running. | 20:44 |
| pabelanger | and back | 20:44 |
| corvus | AJaeger: my understanding is that mriedem was not asking a technical question, but a policy one, about whether 542484 should be merged | 20:45 |
| mriedem | well, | 20:45 |
| mnaser | corvus: just found an old lost tab that's still working, ha. | 20:45 |
| mriedem | not really, i'm saying "wtf do i do?" :) | 20:45 |
| mriedem | on master, nova is running identical CI jobs, wasting resources | 20:45 |
| mriedem | i can drop the override in project-config and we'll always run that job on nova stable changes <queens, but that's probably ok | 20:46 |
| mriedem | the reason nova has an override in project-config is because of the irrelevant-files, which apparently you can't override | 20:46 |
| mriedem | and it somehow screws up the branch designation in the template - hence my patch to copy the branch restrictions into project-config | 20:47 |
| *** eharney has quit IRC | 20:49 | |
| fungi | infra-root: there seemed to be some consensus for zuul enqueue'ing the zuul performance fix and puppet-zuul vhost fix into the gate pipeline so we can get things going without having to wait for backlogged node allocations in check if someone wants to do that (i can't since the machine with my ssh keys is still offline due to construction) | 20:49 |
| corvus | mriedem: that's roughly it, yeah. that patch seems fine to me. | 20:49 |
| corvus | fungi: oh ok, i'll do it | 20:49 |
| *** kjackal has quit IRC | 20:49 | |
| fungi | thanks corvus! | 20:49 |
| *** wolverineav has joined #openstack-infra | 20:50 | |
| *** kjackal has joined #openstack-infra | 20:50 | |
| fungi | i'm hoping they don't leave my house internetless all night | 20:50 |
| corvus | mriedem: though... the irrelevant-files bit there isn't going to apply, since you also use the integrated-gate template | 20:51 |
| mriedem | corvus: if it doesn't apply then i should just remove that from project-config, | 20:51 |
| mriedem | and the job, via the template in openstack-zuul-jobs, will use the branch restriction for stable only | 20:51 |
| corvus | mriedem: iow, if someone pushes a change to nova/hacking/foo.py on ocata, it's not going to match the job defined on nova in project-config, but it is going to match the job defined in the template, so it'll run | 20:52 |
| mriedem | and nova patches on stable will just run the job regardless of the files being relevant or not | 20:52 |
| mriedem | corvus: so i should just remove the faulty override from project-config, yeah? | 20:52 |
| mriedem | and let the template handle the branch | 20:52 |
| corvus | mriedem: correct, if you drop those local job invocations and rely on the template, you'll only have whatever irrelevant-files are on the job definition itself. | 20:53 |
| corvus | mriedem: which is the same state we're in now anyway | 20:53 |
| mriedem | alright | 20:53 |
| ianw | fungi: i have all day ... just syncing on things from scrollback | 20:54 |
| corvus | mriedem: the alternative is to drop the template and keep the local invocation with the nova-specific irrelevant-files. then it'll run less often, but i'm not sure andreaf, gmann_, AJaeger are convinced that's how we should structure things. | 20:54 |
| *** wolverineav has quit IRC | 20:54 | |
| corvus | good news is, we can sort all this out over beers next week :) | 20:54 |
| mriedem | corvus: plus then if integrated-gate changes in openstack-zuul-jobs, | 20:54 |
| mriedem | nova won't get those changes, and we're duplicating job defs | 20:55 |
| corvus | yeah, that's the downside to that approach | 20:55 |
| mriedem | which sucks | 20:55 |
| corvus | i gate enqueued 545955 545953 545938 | 20:57 |
| fungi | thanks corvus! | 20:57 |
| smcginnis | I notice when someone cherry picks using the gerrit UI, it does not include the "(cherry-picked from xyz)" message in the commit message. | 21:00 |
| clarkb | ok I'm back | 21:00 |
| smcginnis | Is there a template somewhere with gerrit that can be updated so that is included? | 21:00 |
| pabelanger | ianw: anything blocking a new tag of diskimage-builder? | 21:01 |
| clarkb | smcginnis: that is actually a flag to the cherry pick command. I'm not aware of a gerrit setting to have it include that but in theory you could make that a configuration option and have cherry pick do the right thing that way | 21:02 |
| smcginnis | clarkb: So that gerrit would know to do the -x flag? | 21:02 |
| clarkb | ya (or the equivalent for that flag in jgit) | 21:02 |
| fungi | "make" meaning upstream in gerrit via google-style java spaghetti | 21:02 |
| smcginnis | clarkb: But that's not a config option - it would actually require modifying gerrit I'm assuming? | 21:03 |
| *** wolverineav has joined #openstack-infra | 21:03 | |
| clarkb | smcginnis: yes | 21:03 |
| fungi | correct | 21:03 |
| * smcginnis backs away | 21:03 | |
| fungi | and then waiting for us to upgrade to a new enough gerrit to be able to turn that on | 21:03 |
| fungi | or at least a new enough gerrit for us to carry a clean backport temporarily | 21:03 |
| smcginnis | Oh well. Worth checking I guess, but not that big of a deal. | 21:04 |
| clarkb | has anyone promoted the changes we need to promote? | 21:04 |
| fungi | corvus did a few minutes ago | 21:04 |
| fungi | at 20:57z | 21:04 |
| clarkb | thanks | 21:04 |
| dansmith | corvus: fungi: is zuul running jobs just not responding to status queries, or are we going for another reset when the issue is fixed? | 21:05 |
| clarkb | dansmith: zuul is running jobs, just not responding to status queries. But there is a separate unrelated issue that will require a full scheduler restart to pick up the fix for (so little of both at the end of the day) | 21:06 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/project-config master: Don't run legacy-tempest-dsvm-neutron-full for nova on queens+ https://review.openstack.org/542484 | 21:06 |
| mriedem | corvus: AJaeger: here we go ^ | 21:06 |
| dansmith | clarkb: okay | 21:06 |
| *** slaweq_ has joined #openstack-infra | 21:07 | |
| *** wolverineav has quit IRC | 21:08 | |
| *** slaweq_ has quit IRC | 21:12 | |
| *** ldnunes has quit IRC | 21:15 | |
| corvus | despite the performance hit, we're actually not falling far behind based on the node utilization graphs | 21:15 |
| openstackgerrit | Merged openstack-infra/puppet-zuul master: Update apache and backups for json suffix removal https://review.openstack.org/545938 | 21:17 |
| AJaeger | mriedem: I'm happy with that change, +2 | 21:18 |
| mordred | clarkb: ok - once that puppet change runs the dashboard should be back because of the updated rewrite rules I believe, yeah? | 21:18 |
| clarkb | mordred: ya | 21:18 |
| openstackgerrit | Merged openstack-infra/zuul master: Allow test_playbook to run long https://review.openstack.org/545955 | 21:19 |
| mordred | awesome | 21:19 |
| *** wolverineav has joined #openstack-infra | 21:20 | |
| clarkb | looks like we started a puppet run 5 minutes ago | 21:20 |
| clarkb | so we won't use the new repo state until this run is done I Think | 21:20 |
| clarkb | :/ | 21:20 |
| AJaeger | mriedem, corvus, I'm also unhappy with https://review.openstack.org/538908 - this is relevant since it *removes* the integrated-gate template as you explained. I agree a discussion on how to structure these all is needed. I won't +2 a change that removes the template completely and replaces it with the single jobs - that is IMHO the wrong approach | 21:20 |
| rm_work | did zuul's url change from http://zuul.openstack.org/ ? | 21:20 |
| clarkb | rm_work: no, its just broken until we get the chagne above that just merged deployed | 21:21 |
| rm_work | oh ok | 21:21 |
| rm_work | missed that :) thanks | 21:21 |
| clarkb | rm_work: https://review.openstack.org/545938 specifically | 21:21 |
| AJaeger | config-core, could you review two changes to fix publishing, please? https://review.openstack.org/545956 and https://review.openstack.org/545962 | 21:21 |
| * AJaeger waves good night | 21:21 | |
| rm_work | kk | 21:21 |
| rm_work | my timing is just bad i guess, lol | 21:21 |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool master: Hack for test_delete_now rare failures https://review.openstack.org/545982 | 21:22 |
| *** suhdood_ has quit IRC | 21:23 | |
| *** wolverineav has quit IRC | 21:24 | |
| mriedem | AJaeger: ack, -1'ed gibi_off's change | 21:25 |
| *** andreas_s has joined #openstack-infra | 21:26 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Avoid creating extra project schemas https://review.openstack.org/545953 | 21:26 |
| *** clayton has quit IRC | 21:28 | |
| *** hongbin has joined #openstack-infra | 21:29 | |
| *** mhayden has quit IRC | 21:29 | |
| *** hongbin_ has quit IRC | 21:30 | |
| *** mhayden has joined #openstack-infra | 21:30 | |
| *** andreas_s has quit IRC | 21:31 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Fix publishing tagged docs https://review.openstack.org/545962 | 21:31 |
| *** dprince has quit IRC | 21:33 | |
| *** rcernin has joined #openstack-infra | 21:36 | |
| openstackgerrit | Merged openstack-infra/project-config master: Fix training-labs publishing https://review.openstack.org/545956 | 21:36 |
| *** threestrands has joined #openstack-infra | 21:37 | |
| clarkb | zuul apache should be updating in about 10 minutes | 21:37 |
| corvus | i'm re-enqueueing 545953 in gate | 21:38 |
| pabelanger | oh, speaking of apache updating, https://review.openstack.org/497948/ should make it a little more friendly | 21:40 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config master: gerritbot: Change x-vrif-minus-2 to x-verify-minus-2 https://review.openstack.org/545626 | 21:41 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config master: gerritbot: Use new regex feature for ironic stable branches https://review.openstack.org/545990 | 21:41 |
| *** salv-orl_ has joined #openstack-infra | 21:42 | |
| * jlvillal unhappy | 21:42 | |
| *** Goneri has joined #openstack-infra | 21:42 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config master: gerritbot: Use new regex feature for ironic stable branches https://review.openstack.org/545990 | 21:43 |
| jlvillal | clarkb, ^^^ I figured we (Ironic) can be the test subjects for the regex support in gerrit bot | 21:43 |
| clarkb | jlvillal: cool, we'll need to double check the deployment of gerritbot to make sure that the changes made it into running service (might need to make a tag/release to do that) then we can make ironic be the test subjects | 21:43 |
| jlvillal | clarkb, Sounds good :) | 21:44 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config master: gerritbot: Change x-vrif-minus-2 to x-verified-minus-2 https://review.openstack.org/545626 | 21:44 |
| *** salv-orlando has quit IRC | 21:44 | |
| *** suhdood_ has joined #openstack-infra | 21:45 | |
| *** felipemonteiro_ has quit IRC | 21:45 | |
| *** felipemonteiro_ has joined #openstack-infra | 21:45 | |
| *** suhdood_ has quit IRC | 21:45 | |
| fungi | at this point, i suppose we want to postpone the zuul scheduler restart until after the zuul meeting | 21:50 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-httpd master: Reload vhost changes, not restart https://review.openstack.org/497948 | 21:50 |
| *** aeng has joined #openstack-infra | 21:51 | |
| clarkb | zuul status is back | 21:51 |
| pabelanger | fungi: thanks, updated ^ | 21:51 |
| clarkb | rm_work: dansmith smcginnis ^ fyi zuul status should be working agin | 21:51 |
| rm_work | cool thanks | 21:51 |
| dansmith | clarkb: indeed; but we have a reset coming soon? | 21:51 |
| *** armaan has joined #openstack-infra | 21:52 | |
| clarkb | dansmith: yes, I expect that as soon as https://review.openstack.org/#/c/545953/ manages to get in we will work on a scheduler restart | 21:52 |
| clarkb | probably after the zuul meeting so ~an hour from now? | 21:52 |
| dansmith | clarkb: does my thing that is already in the gate get put back into the gate? :) | 21:52 |
| clarkb | dansmith: yes | 21:52 |
| dansmith | clarkb: okay then I give my approval :P | 21:52 |
| clarkb | when we scheduel the restarts we save queues then restore them | 21:52 |
| clarkb | there is a small window where things approved during the outage will have to be reapproved | 21:53 |
| clarkb | (eventually we'll fix that by using something like mqtt to queue up gerrit events rather than just missing them due to lack of ssh connection) | 21:53 |
| *** wolverineav has joined #openstack-infra | 21:53 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Extract an abstract base Parser class https://review.openstack.org/545610 | 21:53 |
| *** felipemonteiro__ has joined #openstack-infra | 21:53 | |
| *** priteau has quit IRC | 21:53 | |
| *** priteau has joined #openstack-infra | 21:54 | |
| *** jcoufal has quit IRC | 21:55 | |
| *** hashar has quit IRC | 21:56 | |
| *** trown is now known as trown|outtypewww | 21:56 | |
| *** felipemonteiro_ has quit IRC | 21:57 | |
| fungi | pabelanger: oh! so it was /usr/bin/service on centos then? | 21:57 |
| *** wolverineav has quit IRC | 21:58 | |
| pabelanger | fungi: yah, I used my local env as default, which was fedora. /usr/sbin/service | 21:58 |
| pabelanger | fungi: good review :D | 21:58 |
| *** priteau has quit IRC | 21:59 | |
| fungi | pabelanger: cool, i'll try to update my memory about what was different there | 21:59 |
| smcginnis | clarkb: Thanks! | 21:59 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Added endpoint add summit event type https://review.openstack.org/545993 | 22:00 |
| corvus | meeting time in #openstack-meeting-alt | 22:01 |
| corvus | er zuul meeting time even | 22:01 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Do not merge: Test review for include_role issue https://review.openstack.org/545994 | 22:02 |
| openstackgerrit | Merged openstack-infra/project-config master: [rally] Add a separate repository for OpenStack plugins https://review.openstack.org/545824 | 22:04 |
| *** lpetrut has quit IRC | 22:05 | |
| *** e0ne has quit IRC | 22:06 | |
| *** VW_ has joined #openstack-infra | 22:06 | |
| *** hamzy has quit IRC | 22:07 | |
| *** florianf has quit IRC | 22:09 | |
| *** florianf has joined #openstack-infra | 22:09 | |
| *** florianf has quit IRC | 22:09 | |
| *** VW has quit IRC | 22:09 | |
| *** eharney has joined #openstack-infra | 22:09 | |
| *** olaph1 has joined #openstack-infra | 22:10 | |
| *** VW_ has quit IRC | 22:11 | |
| *** olaph has quit IRC | 22:11 | |
| *** slaweq_ has joined #openstack-infra | 22:13 | |
| *** abelur_ has joined #openstack-infra | 22:15 | |
| *** hamzy has joined #openstack-infra | 22:16 | |
| openstackgerrit | Merged openstack-infra/zuul master: Avoid creating extra project schemas https://review.openstack.org/545953 | 22:18 |
| *** slaweq_ has quit IRC | 22:18 | |
| *** openstackgerrit has quit IRC | 22:18 | |
| *** priteau has joined #openstack-infra | 22:22 | |
| *** wolverineav has joined #openstack-infra | 22:22 | |
| *** masber has joined #openstack-infra | 22:23 | |
| andreykurilin | hi folks, what are my next steps should be after https://review.openstack.org/545824 ? | 22:23 |
| *** openstackgerrit has joined #openstack-infra | 22:23 | |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Added endpoint add summit event type https://review.openstack.org/545993 | 22:23 |
| clarkb | andreykurilin: we wait for the deployment automation to create and import your new project then we seed the initial group user | 22:27 |
| andreykurilin | clarkb: thanks | 22:27 |
| *** wolverineav has quit IRC | 22:27 | |
| * fungi just saw e-mail from github for it, so probably mostly done in automation now | 22:27 | |
| pabelanger | infra-root: when you have time, https://review.openstack.org/545704/ will help bring ubuntu-bionic reprepro online, I've confirmed the changes work on mirror-update.o.o | 22:28 |
| andreykurilin | fungi: wow. cool | 22:33 |
| *** slaweq has quit IRC | 22:36 | |
| *** dave-mccowan has joined #openstack-infra | 22:37 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Use a status code to detect unknown vs. missing tenant https://review.openstack.org/545879 | 22:38 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul master: Remove Paste from the dependencies https://review.openstack.org/546000 | 22:38 |
| *** priteau has quit IRC | 22:38 | |
| *** armaan has quit IRC | 22:44 | |
| *** slaweq has joined #openstack-infra | 22:45 | |
| *** armaan has joined #openstack-infra | 22:46 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-httpd master: Reload vhost changes, not restart https://review.openstack.org/497948 | 22:50 |
| *** slaweq has quit IRC | 22:50 | |
| *** HeOS has quit IRC | 22:50 | |
| pabelanger | fungi: I was wrong, /usr/sbin/service is both on ubuntu and centos^. The patch with path, didn't pass testing | 22:51 |
| fungi | pabelanger: ahh, so it's just /usr/bin/service on fedora i guess? | 22:53 |
| fungi | i would have loggged into one of the git servers to check, but internet is still down for the system with my ssh keys | 22:54 |
| pabelanger | fungi: no, just /usr/sbin/service. ubuntu has both /usr/bin/service but a symlink to /usr/sbin/service | 22:55 |
| fungi | ahh | 22:55 |
| corvus | fungi, clarkb, mordred, pabelanger: the zuul perf improvement has landed, should we restart now or wait? | 22:57 |
| clarkb | corvus: I just checked that it is installed on zuul01 ( it is) | 22:58 |
| pabelanger | I still have some time to help | 22:58 |
| clarkb | I think we should go ahead and restart | 22:58 |
| pabelanger | ++ | 22:58 |
| clarkb | there is nothing in the release queue currently | 22:58 |
| corvus | cool. this is just a scheduler restart, so should be simple. shall i go ahead and do it then? | 22:59 |
| clarkb | corvus: wfm | 22:59 |
| pabelanger | wfm | 22:59 |
| mordred | corvus: we also landed changes to zuul-web - I'd love to restart zuul-web too - after you restart thescheduler | 23:00 |
| clarkb | mordred: since the last restart? | 23:00 |
| mordred | clarkb: the last thing was just the javascript | 23:00 |
| mordred | there's also a change to the python in there | 23:00 |
| clarkb | mordred: ya I restarted zuul-web already | 23:01 |
| corvus | mordred: well, fortunately, zuul-web is broken and needs a restart everytime zuul-scheduler restarts anyway | 23:01 |
| clarkb | otherwise javascript wouldn't work | 23:01 |
| mordred | clarkb: oh! ok. nevermind then | 23:01 |
| pabelanger | corvus: is that an easy fix for 3.0 | 23:01 |
| *** bobh has quit IRC | 23:02 | |
| pabelanger | I can look into it with some direction | 23:02 |
| corvus | hrm... i think the gearman server failed to start | 23:02 |
| *** aeng has quit IRC | 23:03 | |
| corvus | restarting again | 23:03 |
| corvus | i wonder if i was just too quick restarting the first time... | 23:03 |
| corvus | mordred: zuul-web restarted, and is correctly returning tenant not ready | 23:04 |
| mordred | corvus: woot | 23:04 |
| corvus | mordred: did the 204 change land? | 23:04 |
| mordred | corvus: also - the info endpoint works (although I didn't manage to add it to that puppet-zuul change) | 23:04 |
| mordred | corvus: no it did not - I had to update it | 23:04 |
| *** dave-mccowan has quit IRC | 23:05 | |
| mordred | I've also discovered a bug in the info endpoint which I will now go fix - and figure out how to test | 23:05 |
| corvus | w00t | 23:05 |
| mordred | "endpoint": "http://127.0.0.1:9000", <-- request.url is returning the aiohttp url, not the proxied url | 23:06 |
| fungi | i'm good with going ahead with the restart, but am not in a position to be able to assist | 23:06 |
| * fungi catches up on scrollback | 23:06 | |
| clarkb | mordred: the reserse proxy stuff on apache will only affect headers not body content | 23:06 |
| clarkb | if that is part of the json (looks like it) you'll have to configure zuul to return that I think | 23:07 |
| *** wolverineav has joined #openstack-infra | 23:07 | |
| mordred | clarkb: yah, that is correct | 23:07 |
| corvus | re-enqueuing | 23:07 |
| mordred | clarkb: but I should be able to get the effective url from the apache headers right? | 23:07 |
| mordred | clarkb: like, doens't it add something to the headers when it's doing proxy pass stuff | 23:07 |
| mordred | ? | 23:07 |
| clarkb | mordred: there may be an x-forwarded-for | 23:07 |
| clarkb | mordred: we might have to configure that | 23:07 |
| clarkb | but ya thats potentially another option | 23:07 |
| corvus | that's a good option which should work across multiple proxy setups (not just apache) | 23:08 |
| corvus | 2018-02-19 23:08:03,169 DEBUG zuul.Pipeline.openstack.gate: Loading dynamic layout (phase 2) | 23:09 |
| corvus | 2018-02-19 23:08:14,213 DEBUG zuul.Pipeline.openstack.gate: Loading dynamic layout complete | 23:09 |
| clarkb | that looks a lot more like the old numbers | 23:09 |
| corvus | ya | 23:09 |
| pabelanger | nice | 23:09 |
| pabelanger | now to see if memory stay around 14GB | 23:09 |
| pabelanger | stays* | 23:09 |
| fungi | x-forwarded-for should be safe in this case, yes. if you have the actual endpoint exposed to the internet and make any authorization/filtering decisions based on x-forwarded-for, you have to be aware that clients can spoof it of course | 23:10 |
| corvus | 11,12,12,12 seconds so far | 23:11 |
| corvus | so we seem to be pretty consistently averaging 12s | 23:12 |
| *** wolverineav has quit IRC | 23:12 | |
| corvus | last week looked to be a pretty consistent 10s | 23:12 |
| openstackgerrit | Trevor McCasland proposed openstack-infra/subunit2sql master: WIP: add subunit2sql-meta for reading metadata with runs https://review.openstack.org/546006 | 23:13 |
| corvus | so we've grown by 2s. *that* might be due to the increased memory efficiency though, which does a little bit more work to only store half the data | 23:13 |
| *** olaph1 has quit IRC | 23:13 | |
| *** olaph has joined #openstack-infra | 23:13 | |
| *** agopi has quit IRC | 23:15 | |
| mordred | clarkb, fungi: X-Forwarded-Host is the one we're thinking about | 23:19 |
| *** aeng has joined #openstack-infra | 23:20 | |
| fungi | ahh, you don't care about preserving the client identity i guess | 23:21 |
| fungi | well, client ip address anyway | 23:21 |
| mordred | nope. not even a little bit :) | 23:22 |
| *** jbadiapa has quit IRC | 23:22 | |
| *** felipemonteiro__ has quit IRC | 23:25 | |
| pabelanger | corvus: ack | 23:26 |
| pabelanger | so far memory looks real good | 23:27 |
| pabelanger | lets see what the next few days look like | 23:27 |
| *** hongbin has quit IRC | 23:27 | |
| openstackgerrit | Merged openstack-infra/system-config master: Update VerifyRelease keys for debuntu in reprepro https://review.openstack.org/545704 | 23:28 |
| corvus | pabelanger: good. i haven't actually really started on my work to improve memory use yet; that was just something i noticed as i was preparing. i'm expecting some significant changes to come yet. | 23:28 |
| pabelanger | \o/ | 23:29 |
| *** wolverineav has joined #openstack-infra | 23:33 | |
| *** eernst has joined #openstack-infra | 23:34 | |
| *** masber has quit IRC | 23:34 | |
| *** jbadiapa has joined #openstack-infra | 23:35 | |
| pabelanger | mordred: do you mind adding https://review.openstack.org/545314/ to your reviews | 23:36 |
| *** r-daneel has quit IRC | 23:36 | |
| *** masber has joined #openstack-infra | 23:37 | |
| *** panda is now known as panda|off | 23:37 | |
| mordred | pabelanger: lgtm - I left +A off - not sure if you want to control the landing of that or not | 23:38 |
| mordred | pabelanger: I mean, it makes me sad it's needed, but such is life | 23:38 |
| pabelanger | mordred: ack, thanks | 23:38 |
| *** wolverineav has quit IRC | 23:39 | |
| *** xarses has joined #openstack-infra | 23:42 | |
| *** andreww has joined #openstack-infra | 23:43 | |
| *** andreww has quit IRC | 23:43 | |
| *** andreww has joined #openstack-infra | 23:44 | |
| *** xarses has quit IRC | 23:46 | |
| gmann_ | mriedem: corvus andreaf AJaeger i am in favor of not using the integrated-gate template for nova otherwise it leads to run all master jobs (tempest-full, neutron-grenade ) on api-ref, hacking, ut patches which is more costly i think | 23:50 |
| pabelanger | clarkb: ianw: we'll need to land https://review.openstack.org/545183/ before glean changes. I added depends-on that way, to show that glean opensuse nodepool jobs, actually built tumbleweed | 23:51 |
| pabelanger | I can flip depends-on if you like too | 23:53 |
| *** slaweq has joined #openstack-infra | 23:53 | |
| gmann_ | mriedem: corvus andreaf AJaeger i am sure same issue exist on other project case also not only nova. IMO either 1. we stop using integrated-gate template which means do not define it 2. allow each project specific irrelevant_files into template. 2nd one is not good and might create issue due to conflict in few irrelevant_files across projects | 23:53 |
| corvus | gmann_: can nova's irrelevant-files be added to the template and be used everywhere? | 23:53 |
| corvus | gmann_: ok | 23:54 |
| gmann_ | corvus: it can but i am afraid some of them might conflict with other projects but need to check case by case | 23:54 |
| corvus | gmann_: yeah, i seem to remember maybe something like cinder has a doc subdirectory carved out while nova doesn't so it doesn't really work to share them globally | 23:54 |
| gmann_ | corvus: true | 23:54 |
| *** aeng has quit IRC | 23:55 | |
| gmann_ | corvus: it think that is just for functional job, integrated jobs is ok for doc dir at least | 23:55 |
| gmann_ | corvus: i mean nova does not want to run integrated jobs for doc/* | 23:56 |
| *** masber has quit IRC | 23:57 | |
| clarkb | pabelanger: I dont think order matters unless the job is voting. Otherwise just get them in together ish | 23:57 |
| *** slaweq has quit IRC | 23:58 | |
| gmann_ | in 2nd approach, integrated-gate template might contain a huge list of irrelevant_files for all projects (at least <project>/tests) | 23:58 |
| corvus | gmann_: yeah, my feeling is that this stuff is so different for each project and the template is so small, that it's not really worthwhile. but i think we all need to have a discussion about it to figure out the best way forward, but not everyone is convinced. so maybe we should try to find some time for that next week. | 23:58 |
| gmann_ | corvus: nice idea. how about during infra help hour where we all are there together | 23:59 |
| corvus | er, i mis-edited that line, i meant to say not everyone is convinced we should drop the template, therefore we should have a discussion about it next week. in case that wasn't clear. :) | 23:59 |
| corvus | gmann_: ++ | 23:59 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
