| *** jamesmcarthur has joined #openstack-infra | 00:02 | |
| *** slaweq has quit IRC | 00:03 | |
| *** yamamoto has joined #openstack-infra | 00:06 | |
| *** jamesmcarthur has quit IRC | 00:10 | |
| *** bobh has quit IRC | 00:11 | |
| *** bobh has joined #openstack-infra | 00:11 | |
| *** jamesmca_ has joined #openstack-infra | 00:14 | |
| *** bobh has quit IRC | 00:16 | |
| *** jcoufal_ has joined #openstack-infra | 00:18 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] GPT partitioning support https://review.openstack.org/533490 | 00:19 |
|---|---|---|
| *** erlon has quit IRC | 00:20 | |
| *** jcoufal has quit IRC | 00:22 | |
| *** Keitaro has joined #openstack-infra | 00:22 | |
| *** tosky has quit IRC | 00:23 | |
| *** esberglu has quit IRC | 00:24 | |
| *** jcoufal_ has quit IRC | 00:28 | |
| *** edmondsw has joined #openstack-infra | 00:35 | |
| *** slaweq has joined #openstack-infra | 00:36 | |
| *** edmondsw has quit IRC | 00:40 | |
| *** slaweq has quit IRC | 00:41 | |
| *** gema has quit IRC | 00:47 | |
| *** gema has joined #openstack-infra | 00:48 | |
| *** threestrands_ has quit IRC | 00:48 | |
| *** threestrands has joined #openstack-infra | 00:48 | |
| *** persia has joined #openstack-infra | 00:49 | |
| *** ganso has joined #openstack-infra | 00:52 | |
| *** caphrim007_ has quit IRC | 00:53 | |
| *** caphrim007 has joined #openstack-infra | 00:55 | |
| *** camunoz has quit IRC | 00:58 | |
| *** felipemonteiro_ has joined #openstack-infra | 01:01 | |
| *** caphrim007 has quit IRC | 01:03 | |
| *** daidv has quit IRC | 01:04 | |
| *** daidv has joined #openstack-infra | 01:04 | |
| *** felipemonteiro has quit IRC | 01:04 | |
| *** jamesmca_ has quit IRC | 01:08 | |
| *** slaweq has joined #openstack-infra | 01:12 | |
| *** jamesmcarthur has joined #openstack-infra | 01:13 | |
| *** jamesmcarthur has quit IRC | 01:16 | |
| *** slaweq has quit IRC | 01:16 | |
| *** armax has quit IRC | 01:18 | |
| *** liujiong has joined #openstack-infra | 01:22 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: zk: check for client in properties https://review.openstack.org/534988 | 01:24 |
| *** Kevin_Zheng has joined #openstack-infra | 01:24 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: Add a plugin interface for drivers https://review.openstack.org/524620 | 01:25 |
| *** liujiong has quit IRC | 01:28 | |
| *** jamesmcarthur has joined #openstack-infra | 01:29 | |
| *** liujiong has joined #openstack-infra | 01:30 | |
| *** jamesmcarthur has quit IRC | 01:31 | |
| *** jamesmcarthur has joined #openstack-infra | 01:31 | |
| *** jamesmcarthur has quit IRC | 01:49 | |
| *** stakeda has joined #openstack-infra | 01:50 | |
| *** jamesmcarthur has joined #openstack-infra | 01:51 | |
| *** slaweq has joined #openstack-infra | 01:51 | |
| *** zhurong has joined #openstack-infra | 01:52 | |
| *** jamesmcarthur has quit IRC | 01:53 | |
| *** eharney has quit IRC | 01:54 | |
| *** openstackstatus has quit IRC | 01:55 | |
| *** harlowja has quit IRC | 01:55 | |
| *** berendt has quit IRC | 01:55 | |
| *** akscram1 has quit IRC | 01:55 | |
| *** mhayden has quit IRC | 01:55 | |
| *** thorre has quit IRC | 01:55 | |
| *** johnsom has quit IRC | 01:55 | |
| *** pots has quit IRC | 01:55 | |
| *** robcresswell has quit IRC | 01:55 | |
| *** jmccrory has quit IRC | 01:55 | |
| *** Roamer` has quit IRC | 01:55 | |
| *** rajinir has quit IRC | 01:55 | |
| *** vipul has quit IRC | 01:55 | |
| *** jlvillal has quit IRC | 01:55 | |
| *** cargonza has quit IRC | 01:55 | |
| *** mnaser has quit IRC | 01:55 | |
| *** vdrok has quit IRC | 01:55 | |
| *** fdegir has quit IRC | 01:55 | |
| *** tbarron has quit IRC | 01:55 | |
| *** mandre has quit IRC | 01:55 | |
| *** Roamer` has joined #openstack-infra | 01:55 | |
| *** jmccrory has joined #openstack-infra | 01:55 | |
| *** mandre has joined #openstack-infra | 01:55 | |
| *** thorre has joined #openstack-infra | 01:55 | |
| *** pots has joined #openstack-infra | 01:55 | |
| *** tbarron has joined #openstack-infra | 01:55 | |
| *** jlvillal has joined #openstack-infra | 01:55 | |
| *** akscram1 has joined #openstack-infra | 01:55 | |
| *** rajinir has joined #openstack-infra | 01:55 | |
| *** slaweq has quit IRC | 01:55 | |
| *** berendt has joined #openstack-infra | 01:56 | |
| *** cargonza has joined #openstack-infra | 01:56 | |
| *** vdrok has joined #openstack-infra | 01:56 | |
| *** mnaser has joined #openstack-infra | 01:56 | |
| *** openstackstatus has joined #openstack-infra | 01:56 | |
| *** ChanServ sets mode: +v openstackstatus | 01:56 | |
| *** mhayden has joined #openstack-infra | 01:56 | |
| *** johnsom has joined #openstack-infra | 01:56 | |
| *** robcresswell has joined #openstack-infra | 01:56 | |
| *** fdegir has joined #openstack-infra | 01:56 | |
| *** bandini has quit IRC | 01:57 | |
| *** vipul has joined #openstack-infra | 01:58 | |
| *** bandini has joined #openstack-infra | 01:59 | |
| TheJulia | Out of curiosity, has anyone seen jobs hang on "TASK [fetch-zuul-cloner : Install zuul-cloner shim dependencies]" ? | 02:01 |
| *** hongbin has joined #openstack-infra | 02:01 | |
| *** stakeda has quit IRC | 02:07 | |
| *** annp has joined #openstack-infra | 02:19 | |
| *** caphrim007 has joined #openstack-infra | 02:23 | |
| *** edmondsw has joined #openstack-infra | 02:24 | |
| pabelanger | fungi: thanks! | 02:25 |
| pabelanger | TheJulia: I have not, do you have a log or which job? | 02:25 |
| TheJulia | pabelanger: there is a networking-generic-switch job in the gate, showing no progress, that has restarted twice after hanging at the same place for like a half hour | 02:26 |
| TheJulia | pabelanger: right now it reports "END OF STREAM". Last time, it looked like the job was automatically restarted in the background | 02:27 |
| *** jamesmcarthur has joined #openstack-infra | 02:27 | |
| TheJulia | pabelanger: since times incremented from the previous hang. | 02:27 |
| openstackgerrit | lvxianguo proposed openstack/python-jenkins master: trivial: modify spelling error of version https://review.openstack.org/535001 | 02:27 |
| *** edmondsw has quit IRC | 02:28 | |
| *** caphrim007 has quit IRC | 02:28 | |
| ianw | TheJulia: what's the id (that you get in the console url?) | 02:29 |
| TheJulia | http://zuulv3.openstack.org/stream.html?uuid=0715299bb82f4fd8bb77a1b4020ab412&logfile=console.log | 02:29 |
| TheJulia | looks like it is running again | 02:29 |
| *** slaweq has joined #openstack-infra | 02:30 | |
| * TheJulia blinks | 02:30 | |
| pabelanger | odd, that task just installs a script via ansible | 02:30 |
| pabelanger | I wonder if the node had a networking issue via SSH | 02:31 |
| pabelanger | and job aborted and run on a new cloud | 02:31 |
| TheJulia | ahh, connection timeout | 02:31 |
| TheJulia | mirror.bhs1.ovh.openstack.org seems unreachable based on the logs | 02:32 |
| pabelanger | fungi: Warning: nodepool-builder.service changed on disk. Run 'systemctl daemon-reload' to reload units. | 02:33 |
| pabelanger | something to fix in the morning | 02:33 |
| pabelanger | I'll update puppet-nodepool to account for systemd | 02:33 |
| ianw | TheJulia: urgh, indeed ... let me see if there's anything on the console | 02:34 |
| *** slaweq has quit IRC | 02:35 | |
| TheJulia | hmm, looks like the next run seems to be moving forward | 02:35 |
| TheJulia | Yeah, moving forward on an ovh node | 02:36 |
| Jeffrey4l | could any one check this link https://review.openstack.org/531641 it can not be opened. | 02:38 |
| *** hrw has quit IRC | 02:38 | |
| pabelanger | odd | 02:39 |
| pabelanger | The page you requested was not found, or you do not have permission to view this page. | 02:39 |
| pabelanger | not sure why that is | 02:39 |
| pabelanger | I should have admin permissions | 02:39 |
| *** apetrich has quit IRC | 02:41 | |
| clarkb | drafts can do that but they should be disabled | 02:41 |
| ianw | mirror.bhs1.ovh.openstack.org ... console log doesn't show anything odd, but i can't get into a live console via the url, just sits at "connecting" | 02:41 |
| *** apetrich has joined #openstack-infra | 02:41 | |
| ianw | i'm just going to hard reboot it | 02:41 |
| pabelanger | #status log nb04.o.o stopped to prepare for nb01.o.o replacement tomorrow | 02:41 |
| openstackstatus | pabelanger: finished logging | 02:41 |
| *** yamahata has quit IRC | 02:42 | |
| *** iyamahat has quit IRC | 02:42 | |
| pabelanger | clarkb: any way to check it is disabled? | 02:42 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: zk: automatically retry command when connection is lost https://review.openstack.org/523640 | 02:43 |
| clarkb | its an acl in all projects config we dont allow pushes to refs/for/draft or whatever the magic dest is | 02:43 |
| ianw | A start job is running for OpenAFS client (21s / 1min 44s) ... hmm | 02:44 |
| pabelanger | ! [remote rejected] HEAD -> refs/drafts/master (draft workflow is disabled) | 02:45 |
| pabelanger | atleast on sandbox | 02:45 |
| *** jamesmcarthur has quit IRC | 02:45 | |
| pabelanger | I'll look more in the morning on 531641 | 02:45 |
| pabelanger | ianw: which? | 02:46 |
| clarkb | possibly a corrupt git object or repo could do tha | 02:46 |
| clarkb | ? | 02:46 |
| *** jamesmcarthur has joined #openstack-infra | 02:47 | |
| ianw | #status log mirror.bhs1.ovh.openstack.org was unresponsive ... hard reboot and it has reappeared. nothing useful in console logs unfortunately | 02:47 |
| openstackstatus | ianw: finished logging | 02:47 |
| ianw | pabelanger: might have just been a bit impatient with it, anyway, back now | 02:48 |
| pabelanger | ah, I see | 02:48 |
| *** rhallisey has quit IRC | 02:50 | |
| clarkb | Jeffrey4l: do you know what project or commit that change maps to? | 02:51 |
| Jeffrey4l | clarkb, it is a patch of kolla-ansible project | 02:52 |
| Jeffrey4l | last night, some guy make changes on the gerrit web page, then it become unreachable. | 02:52 |
| Jeffrey4l | it is the first one in https://review.openstack.org/#/q/topic:bp/support-custom-policy-yaml+(status:open+OR+status:merged) | 02:52 |
| Jeffrey4l | seem only the last patch is unaccessable. https://review.openstack.org/#/c/531641/6 , patch 6 is OK. | 02:53 |
| Jeffrey4l | clarkb, ^ | 02:53 |
| clarkb | http://paste.openstack.org/show/646780/ there we go, logs had rolled over had to look at yesterday's | 02:54 |
| clarkb | I'm not sure that there is going to be a fix here | 02:55 |
| Jeffrey4l | clarkb, so click edit and then close the browser will trigger this issue? | 02:56 |
| clarkb | it looks like maybe there are incomplete edits which has confused gerrit | 02:56 |
| *** andreas_s has joined #openstack-infra | 02:56 | |
| clarkb | you might be able to "move" it along by pushing a new patchset? | 02:56 |
| clarkb | Jeffrey4l: I don't know, thats what gerrit has logged, I have no idea what the user did | 02:56 |
| Jeffrey4l | let me try to push a new one. | 02:56 |
| Jeffrey4l | ok. patch 10 make it back to life. thanks clarkb . | 02:57 |
| *** andreas_s has quit IRC | 03:01 | |
| *** kiennt26 has joined #openstack-infra | 03:06 | |
| *** armax has joined #openstack-infra | 03:07 | |
| *** slaweq has joined #openstack-infra | 03:10 | |
| *** slaweq has quit IRC | 03:10 | |
| *** witek_ has joined #openstack-infra | 03:10 | |
| *** witek has quit IRC | 03:10 | |
| *** abelur_ has quit IRC | 03:11 | |
| *** ekcs has quit IRC | 03:16 | |
| *** abelur_ has joined #openstack-infra | 03:16 | |
| *** ganso has quit IRC | 03:20 | |
| *** jamesmcarthur has quit IRC | 03:20 | |
| *** ramishra has joined #openstack-infra | 03:21 | |
| *** sree has joined #openstack-infra | 03:29 | |
| *** sdoran has quit IRC | 03:30 | |
| *** alex_xu has quit IRC | 03:30 | |
| *** gmann has quit IRC | 03:30 | |
| *** therve has quit IRC | 03:30 | |
| *** ajo has quit IRC | 03:30 | |
| *** RuiChen has quit IRC | 03:30 | |
| *** rm_work has quit IRC | 03:30 | |
| *** jpmaxman has quit IRC | 03:30 | |
| *** ildikov has quit IRC | 03:30 | |
| *** TheJulia has quit IRC | 03:30 | |
| *** zxiiro has quit IRC | 03:30 | |
| *** evgenyl has quit IRC | 03:30 | |
| *** coreycb has quit IRC | 03:30 | |
| *** csatari has quit IRC | 03:30 | |
| *** beisner has quit IRC | 03:30 | |
| *** samueldmq has quit IRC | 03:30 | |
| *** briancurtin has quit IRC | 03:30 | |
| *** fmccrthy has quit IRC | 03:30 | |
| *** filler has quit IRC | 03:30 | |
| *** wendar has quit IRC | 03:30 | |
| *** timrc has quit IRC | 03:30 | |
| *** jamielennox has quit IRC | 03:30 | |
| *** gus has quit IRC | 03:30 | |
| *** wznoinsk has quit IRC | 03:30 | |
| *** timrc has joined #openstack-infra | 03:30 | |
| *** wendar has joined #openstack-infra | 03:30 | |
| *** filler has joined #openstack-infra | 03:30 | |
| *** therve has joined #openstack-infra | 03:30 | |
| *** gus has joined #openstack-infra | 03:30 | |
| *** rm_work has joined #openstack-infra | 03:30 | |
| *** rm_work has quit IRC | 03:30 | |
| *** rm_work has joined #openstack-infra | 03:30 | |
| *** wznoinsk has joined #openstack-infra | 03:30 | |
| *** alex_xu has joined #openstack-infra | 03:30 | |
| *** jpmaxman has joined #openstack-infra | 03:31 | |
| *** sdoran has joined #openstack-infra | 03:31 | |
| *** ajo has joined #openstack-infra | 03:31 | |
| *** samueldmq has joined #openstack-infra | 03:31 | |
| *** gmann has joined #openstack-infra | 03:31 | |
| *** RuiChen has joined #openstack-infra | 03:31 | |
| *** beisner has joined #openstack-infra | 03:31 | |
| *** evgenyl has joined #openstack-infra | 03:31 | |
| *** ildikov has joined #openstack-infra | 03:31 | |
| *** briancurtin has joined #openstack-infra | 03:31 | |
| *** fmccrthy has joined #openstack-infra | 03:31 | |
| *** csatari has joined #openstack-infra | 03:31 | |
| *** TheJulia has joined #openstack-infra | 03:31 | |
| *** coreycb has joined #openstack-infra | 03:31 | |
| *** zxiiro has joined #openstack-infra | 03:31 | |
| *** mlavalle has quit IRC | 03:32 | |
| *** rossella_s has quit IRC | 03:32 | |
| *** armax has quit IRC | 03:36 | |
| *** rossella_s has joined #openstack-infra | 03:36 | |
| *** jamielennox has joined #openstack-infra | 03:37 | |
| *** armax has joined #openstack-infra | 03:37 | |
| *** dingyichen has joined #openstack-infra | 03:44 | |
| *** slaweq has joined #openstack-infra | 03:44 | |
| *** vhosakot has joined #openstack-infra | 03:46 | |
| *** slaweq has quit IRC | 03:49 | |
| *** vhosakot has quit IRC | 03:52 | |
| *** daidv has quit IRC | 04:00 | |
| *** daidv has joined #openstack-infra | 04:01 | |
| *** zhurong has quit IRC | 04:08 | |
| *** bhavik1 has joined #openstack-infra | 04:09 | |
| *** psachin has joined #openstack-infra | 04:10 | |
| *** armax has quit IRC | 04:10 | |
| *** bhavik1 has quit IRC | 04:13 | |
| *** slaweq has joined #openstack-infra | 04:15 | |
| *** dhajare has joined #openstack-infra | 04:19 | |
| *** slaweq has quit IRC | 04:20 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] GPT partitioning support https://review.openstack.org/533490 | 04:20 |
| *** bobh has joined #openstack-infra | 04:24 | |
| *** gouthamr has joined #openstack-infra | 04:26 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/nodepool feature/zuulv3: zk: use kazoo retry facilities https://review.openstack.org/523640 | 04:27 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: [WIP] GPT partitioning support https://review.openstack.org/533490 | 04:28 |
| *** gouthamr has quit IRC | 04:30 | |
| *** coolsvap has joined #openstack-infra | 04:36 | |
| *** ykarel|away has joined #openstack-infra | 04:36 | |
| *** zhongjun has quit IRC | 04:46 | |
| *** slaweq has joined #openstack-infra | 04:51 | |
| *** slaweq has quit IRC | 04:55 | |
| *** dave-mccowan has quit IRC | 04:59 | |
| *** ameliac has joined #openstack-infra | 04:59 | |
| *** kiennt26 has quit IRC | 05:04 | |
| *** hongbin has quit IRC | 05:05 | |
| *** ameliac has quit IRC | 05:07 | |
| *** gcb has quit IRC | 05:07 | |
| *** jamesmcarthur has joined #openstack-infra | 05:11 | |
| *** bobh has quit IRC | 05:12 | |
| *** gcb has joined #openstack-infra | 05:14 | |
| *** jamesmcarthur has quit IRC | 05:16 | |
| AJaeger | ianw, frickler , could you review https://review.openstack.org/#/c/534880/ and https://review.openstack.org/#/c/534896/ to cleanup jobs in project-config, please? | 05:19 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Link to zuul-base-jobs docs from User's Guide https://review.openstack.org/531912 | 05:21 |
| *** slaweq has joined #openstack-infra | 05:23 | |
| *** slaweq has quit IRC | 05:27 | |
| *** links has joined #openstack-infra | 05:31 | |
| *** eumel8 has joined #openstack-infra | 05:42 | |
| *** cuongnv has joined #openstack-infra | 05:42 | |
| *** jamesmcarthur has joined #openstack-infra | 05:47 | |
| *** spzala has quit IRC | 05:48 | |
| *** zhurong has joined #openstack-infra | 05:48 | |
| *** spzala has joined #openstack-infra | 05:48 | |
| *** claudiub has joined #openstack-infra | 05:49 | |
| *** jamesmcarthur has quit IRC | 05:51 | |
| *** spzala has quit IRC | 05:53 | |
| *** caphrim007 has joined #openstack-infra | 05:54 | |
| *** caphrim007 has quit IRC | 05:58 | |
| *** slaweq has joined #openstack-infra | 06:01 | |
| *** pgadiya has joined #openstack-infra | 06:04 | |
| *** namnh has joined #openstack-infra | 06:05 | |
| *** armaan has quit IRC | 06:05 | |
| *** slaweq has quit IRC | 06:05 | |
| *** armaan has joined #openstack-infra | 06:06 | |
| frickler | AJaeger: done | 06:07 |
| frickler | infra-root: devstack-gate still needs reviews https://review.openstack.org/529000 | 06:07 |
| rakhmerov | hi, is our CI broken again? We have several approved patches but CI jobs are not starting for them | 06:09 |
| frickler | rakhmerov: which patches? may be some dependency issue. or the zuul restart yesterday | 06:10 |
| rakhmerov | sec | 06:13 |
| openstackgerrit | Merged openstack-infra/project-config master: Rename legacy-tempest-dsvm-neutron-linuxbridge https://review.openstack.org/534896 | 06:13 |
| rakhmerov | frickler: for example, https://review.openstack.org/#/c/534949/ | 06:13 |
| rakhmerov | I approved it more than 2 hours ago | 06:14 |
| eumel8 | a recheck could help | 06:15 |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: GPT partitioning support https://review.openstack.org/533490 | 06:17 |
| *** janki has joined #openstack-infra | 06:17 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove trove experimental jobs https://review.openstack.org/534880 | 06:17 |
| frickler | rakhmerov: it seems the patch was submitted during the zuul downtime yesterday, so didn't get an initial verified+1 vote. I did a recheck and it now seems to be proceeding | 06:17 |
| *** rosmaita has quit IRC | 06:18 | |
| rakhmerov | ok | 06:18 |
| rakhmerov | what about others? | 06:18 |
| rakhmerov | what do we need to do? | 06:18 |
| rakhmerov | just recheck? | 06:18 |
| frickler | if they don't have the verified vote, submitting a recheck will help, yes | 06:18 |
| frickler | if that doesn't work, please tell us the patch IDs and we can take another look | 06:19 |
| *** gouthamr has joined #openstack-infra | 06:20 | |
| *** aeng has quit IRC | 06:20 | |
| *** felipemonteiro_ has quit IRC | 06:21 | |
| *** jbadiapa has quit IRC | 06:29 | |
| *** jtomasek has quit IRC | 06:30 | |
| *** janki has quit IRC | 06:30 | |
| openstackgerrit | Ian Wienand proposed openstack/diskimage-builder master: GPT partitioning support https://review.openstack.org/533490 | 06:33 |
| *** jtomasek has joined #openstack-infra | 06:38 | |
| *** slaweq has joined #openstack-infra | 06:40 | |
| openstackgerrit | Merged openstack-dev/pbr master: emit warning correctly https://review.openstack.org/534927 | 06:44 |
| *** dhajare has quit IRC | 06:44 | |
| *** slaweq has quit IRC | 06:45 | |
| *** kiennt26 has joined #openstack-infra | 06:47 | |
| *** dhajare has joined #openstack-infra | 06:49 | |
| *** zhongjun has joined #openstack-infra | 06:49 | |
| openstackgerrit | Merged openstack-dev/pbr master: Updated from global requirements https://review.openstack.org/534900 | 06:50 |
| *** makowals has joined #openstack-infra | 06:56 | |
| *** gcb has quit IRC | 06:58 | |
| AJaeger | thanks, frickler | 07:01 |
| *** ianw has quit IRC | 07:02 | |
| *** edmondsw has joined #openstack-infra | 07:06 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Rename trove grenade job https://review.openstack.org/535191 | 07:07 |
| *** andreas_s has joined #openstack-infra | 07:09 | |
| *** stakeda has joined #openstack-infra | 07:09 | |
| *** edmondsw has quit IRC | 07:11 | |
| *** dims has quit IRC | 07:12 | |
| *** janki has joined #openstack-infra | 07:13 | |
| *** dims has joined #openstack-infra | 07:14 | |
| *** pcaruana has joined #openstack-infra | 07:15 | |
| AJaeger | frickler, next ones in the queue if you have time: https://review.openstack.org/535191 and https://review.openstack.org/#/c/531496 and https://review.openstack.org/513589, please. | 07:18 |
| rakhmerov | frickler: ok, thanks | 07:18 |
| *** dims has quit IRC | 07:19 | |
| *** dims has joined #openstack-infra | 07:20 | |
| *** stakeda has quit IRC | 07:22 | |
| *** stakeda has joined #openstack-infra | 07:28 | |
| openstackgerrit | Debo Zhang proposed openstack/diskimage-builder master: Correct link address https://review.openstack.org/535199 | 07:28 |
| *** cshastri has joined #openstack-infra | 07:30 | |
| *** wendar has quit IRC | 07:31 | |
| *** alexchadin has joined #openstack-infra | 07:31 | |
| prometheanfire | mordred: was it you I was talking to about mailman3/dmarc? | 07:33 |
| prometheanfire | anyway, testing out mm3, and working well, weith dmarc strict (rewrite from in our case is acceptable) | 07:34 |
| *** ykarel|away is now known as ykarel | 07:38 | |
| *** slaweq has joined #openstack-infra | 07:38 | |
| *** dhajare is now known as dhajare|BRB | 07:40 | |
| *** ekcs has joined #openstack-infra | 07:42 | |
| prometheanfire | nn | 07:48 |
| *** vsaienk0 has joined #openstack-infra | 07:48 | |
| *** alexchadin has quit IRC | 07:49 | |
| *** alexchadin has joined #openstack-infra | 07:52 | |
| *** dhajare|BRB has quit IRC | 07:52 | |
| *** ekcs has quit IRC | 07:53 | |
| *** ekcs has joined #openstack-infra | 07:53 | |
| *** wendar has joined #openstack-infra | 07:53 | |
| *** ekcs has quit IRC | 07:54 | |
| *** namnh has quit IRC | 07:55 | |
| *** cuongnv has quit IRC | 07:55 | |
| *** annp has quit IRC | 07:56 | |
| *** kiennt26 has quit IRC | 07:56 | |
| *** namnh has joined #openstack-infra | 08:01 | |
| *** cuongnv has joined #openstack-infra | 08:01 | |
| *** agopi has quit IRC | 08:01 | |
| *** threestrands has quit IRC | 08:03 | |
| *** florianf has joined #openstack-infra | 08:04 | |
| *** armaan has quit IRC | 08:07 | |
| *** jamesmcarthur has joined #openstack-infra | 08:08 | |
| *** tesseract has joined #openstack-infra | 08:08 | |
| *** b_bezak has joined #openstack-infra | 08:12 | |
| *** jamesmcarthur has quit IRC | 08:12 | |
| *** kiennt26 has joined #openstack-infra | 08:12 | |
| *** annp has joined #openstack-infra | 08:13 | |
| *** notmyname has quit IRC | 08:13 | |
| *** notmyname has joined #openstack-infra | 08:16 | |
| *** namnh has quit IRC | 08:16 | |
| *** cuongnv has quit IRC | 08:16 | |
| *** ralonsoh has joined #openstack-infra | 08:17 | |
| *** kiennt26 has quit IRC | 08:18 | |
| *** namnh has joined #openstack-infra | 08:18 | |
| *** cuongnv has joined #openstack-infra | 08:18 | |
| *** kiennt26 has joined #openstack-infra | 08:18 | |
| *** aviau has quit IRC | 08:19 | |
| *** aviau has joined #openstack-infra | 08:19 | |
| *** pgadiya has quit IRC | 08:22 | |
| *** dingyichen has quit IRC | 08:22 | |
| *** oidgar has joined #openstack-infra | 08:23 | |
| *** witek_ is now known as witek | 08:24 | |
| *** alexchadin has quit IRC | 08:26 | |
| *** shardy has joined #openstack-infra | 08:27 | |
| *** rcernin has quit IRC | 08:28 | |
| *** gouthamr has quit IRC | 08:28 | |
| *** ralonsoh has quit IRC | 08:28 | |
| *** jbadiapa has joined #openstack-infra | 08:29 | |
| *** thorre_se has joined #openstack-infra | 08:35 | |
| *** namnh has quit IRC | 08:35 | |
| *** namnh has joined #openstack-infra | 08:35 | |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Add buildset-artifacts-location and fetch roles https://review.openstack.org/530679 | 08:37 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Add linters job and role https://review.openstack.org/530682 | 08:37 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Add ansible-lint job https://review.openstack.org/532083 | 08:37 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Add ansible-import-to-galaxy job https://review.openstack.org/532084 | 08:37 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Add ansible-spec job https://review.openstack.org/532085 | 08:37 |
| openstackgerrit | Tristan Cacqueray proposed openstack-infra/zuul-jobs master: Add ansible-review job https://review.openstack.org/535223 | 08:37 |
| *** ralonsoh has joined #openstack-infra | 08:37 | |
| *** thorre has quit IRC | 08:38 | |
| *** thorre_se is now known as thorre | 08:38 | |
| *** pgadiya has joined #openstack-infra | 08:38 | |
| *** cuongnv has quit IRC | 08:39 | |
| *** pcaruana has quit IRC | 08:43 | |
| *** pcaruana has joined #openstack-infra | 08:43 | |
| *** namnh_ has joined #openstack-infra | 08:44 | |
| *** namnh has quit IRC | 08:44 | |
| *** namnh_ has quit IRC | 08:44 | |
| *** jpena|off is now known as jpena | 08:46 | |
| *** namnh has joined #openstack-infra | 08:46 | |
| *** zoli is now known as zoli|trng | 08:48 | |
| *** zoli|trng is now known as zoli | 08:48 | |
| *** amoralej|off is now known as amoralej | 08:49 | |
| *** namnh has quit IRC | 08:53 | |
| *** shardy has quit IRC | 08:53 | |
| *** edmondsw has joined #openstack-infra | 08:54 | |
| *** daidv has quit IRC | 08:55 | |
| *** sshnaidm is now known as sshnaidm|afk | 08:55 | |
| *** shardy has joined #openstack-infra | 08:56 | |
| *** ethfci has joined #openstack-infra | 08:56 | |
| *** kiennt26 has quit IRC | 08:57 | |
| *** namnh has joined #openstack-infra | 08:57 | |
| *** kiennt26 has joined #openstack-infra | 08:57 | |
| *** ianw has joined #openstack-infra | 08:58 | |
| *** edmondsw has quit IRC | 08:59 | |
| *** _Cyclone has quit IRC | 09:02 | |
| *** jpich has joined #openstack-infra | 09:03 | |
| *** arxcruz|rover is now known as arxcruz | 09:06 | |
| *** dtantsur|afk is now known as dtantsur | 09:07 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove legacy job for heat-templates https://review.openstack.org/513589 | 09:08 |
| *** rossella_s has quit IRC | 09:10 | |
| *** rossella_s has joined #openstack-infra | 09:12 | |
| *** daidv has joined #openstack-infra | 09:13 | |
| *** liujiong has quit IRC | 09:13 | |
| *** ganso has joined #openstack-infra | 09:15 | |
| *** jamesmcarthur has joined #openstack-infra | 09:16 | |
| *** hrw has joined #openstack-infra | 09:20 | |
| *** jamesmcarthur has quit IRC | 09:20 | |
| *** dulek has quit IRC | 09:21 | |
| *** zoli has quit IRC | 09:21 | |
| *** dsariel has quit IRC | 09:22 | |
| *** e0ne has joined #openstack-infra | 09:23 | |
| *** dulek has joined #openstack-infra | 09:24 | |
| *** pgadiya has quit IRC | 09:27 | |
| *** tobberydberg has joined #openstack-infra | 09:27 | |
| *** zoli has joined #openstack-infra | 09:28 | |
| *** pgadiya has joined #openstack-infra | 09:32 | |
| *** pgadiya has quit IRC | 09:38 | |
| *** derekh has joined #openstack-infra | 09:39 | |
| *** cuongnv has joined #openstack-infra | 09:39 | |
| *** rossella_s has quit IRC | 09:40 | |
| *** vsaienk0 has quit IRC | 09:43 | |
| *** rossella_s has joined #openstack-infra | 09:44 | |
| *** pgadiya has joined #openstack-infra | 09:50 | |
| *** spzala has joined #openstack-infra | 09:50 | |
| *** vsaienk0 has joined #openstack-infra | 09:52 | |
| *** spzala has quit IRC | 09:54 | |
| *** caphrim007 has joined #openstack-infra | 09:55 | |
| *** pcichy has joined #openstack-infra | 09:55 | |
| *** alexchadin has joined #openstack-infra | 09:57 | |
| *** caphrim007 has quit IRC | 09:59 | |
| *** ramishra has quit IRC | 10:00 | |
| *** sambetts|afk is now known as sambetts | 10:06 | |
| *** cuongnv has quit IRC | 10:06 | |
| *** pgadiya has quit IRC | 10:08 | |
| *** kjackal has joined #openstack-infra | 10:10 | |
| *** rossella_s has quit IRC | 10:10 | |
| *** rossella_s has joined #openstack-infra | 10:12 | |
| *** tosky has joined #openstack-infra | 10:15 | |
| *** sree has quit IRC | 10:16 | |
| *** pgadiya has joined #openstack-infra | 10:19 | |
| *** dsariel has joined #openstack-infra | 10:23 | |
| *** pbourke has quit IRC | 10:26 | |
| *** tesseract has quit IRC | 10:27 | |
| *** pbourke has joined #openstack-infra | 10:27 | |
| *** tesseract has joined #openstack-infra | 10:29 | |
| cmurphy | I think proposal bot is broken, it put three change-ids on this patch https://review.openstack.org/#/c/534025/24 | 10:30 |
| frickler | cmurphy: there are two other patches for the second and third ID, might have been a race condition during the initial run. https://review.openstack.org/#/c/534026/1 https://review.openstack.org/#/c/534027/1 | 10:35 |
| *** ldnunes has joined #openstack-infra | 10:36 | |
| frickler | cmurphy: I'd say you can abandon those and we will wait whether we see it happen more often. unless someone wants to dig further right now | 10:36 |
| cmurphy | okay | 10:38 |
| *** ramishra has joined #openstack-infra | 10:38 | |
| *** dbecker has joined #openstack-infra | 10:41 | |
| *** edmondsw has joined #openstack-infra | 10:42 | |
| *** erlon has joined #openstack-infra | 10:46 | |
| *** vsaienk0 has quit IRC | 10:47 | |
| *** edmondsw has quit IRC | 10:47 | |
| *** vsaienk0 has joined #openstack-infra | 10:47 | |
| tobberydberg | Hi! Question: Do you have any information about how I setup a meetbot for the openstack-publiccloud channel? | 10:48 |
| *** ssbarnea has quit IRC | 10:50 | |
| *** wznoinsk has quit IRC | 10:50 | |
| *** armaan has joined #openstack-infra | 10:51 | |
| openstackgerrit | Tobias Rydberg proposed openstack-infra/irc-meetings master: Updating the list of active chairs for the Public Cloud WG https://review.openstack.org/535286 | 10:54 |
| *** andreas_s has quit IRC | 10:55 | |
| *** ssbarnea has joined #openstack-infra | 10:56 | |
| *** wznoinsk has joined #openstack-infra | 10:57 | |
| *** panda|ruck is now known as panda | 11:00 | |
| *** andreas_s has joined #openstack-infra | 11:00 | |
| *** armaan has quit IRC | 11:01 | |
| *** armaan has joined #openstack-infra | 11:01 | |
| openstackgerrit | Ghanshyam Mann proposed openstack-infra/project-config master: Add tempest-stress project repo https://review.openstack.org/533025 | 11:02 |
| gmann | andreaf: done, https://review.openstack.org/#/c/533025/3 | 11:03 |
| frickler | tobberydberg: you can propose a patch to add your channel to this list: https://git.openstack.org/cgit/openstack-infra/system-config/tree/hiera/common.yaml#n162 also see https://docs.openstack.org/infra/system-config/irc.html#id3 | 11:05 |
| *** saop has joined #openstack-infra | 11:05 | |
| *** andreas_s has quit IRC | 11:05 | |
| *** andreas_s has joined #openstack-infra | 11:06 | |
| *** gouthamr has joined #openstack-infra | 11:09 | |
| *** rossella_s has quit IRC | 11:10 | |
| *** rossella_s has joined #openstack-infra | 11:11 | |
| *** janki has quit IRC | 11:16 | |
| *** tobiash has quit IRC | 11:19 | |
| *** andreas_s has quit IRC | 11:20 | |
| *** andreas_s has joined #openstack-infra | 11:21 | |
| *** tobiash has joined #openstack-infra | 11:22 | |
| *** pgadiya has quit IRC | 11:24 | |
| *** wolverineav has joined #openstack-infra | 11:28 | |
| *** gouthamr has quit IRC | 11:29 | |
| *** gouthamr has joined #openstack-infra | 11:29 | |
| *** gouthamr has quit IRC | 11:30 | |
| *** gouthamr_ has joined #openstack-infra | 11:30 | |
| *** pgadiya has joined #openstack-infra | 11:31 | |
| *** zhurong has quit IRC | 11:34 | |
| *** gouthamr has joined #openstack-infra | 11:34 | |
| *** namnh has quit IRC | 11:35 | |
| *** andreas_s has quit IRC | 11:35 | |
| *** andreas_s has joined #openstack-infra | 11:35 | |
| *** gouthamr_ has quit IRC | 11:36 | |
| *** jistr is now known as jistr|mtg | 11:41 | |
| *** alexchadin has quit IRC | 11:41 | |
| *** pgadiya has quit IRC | 11:45 | |
| *** alexchadin has joined #openstack-infra | 11:48 | |
| *** cshastri has quit IRC | 11:48 | |
| *** janki has joined #openstack-infra | 11:51 | |
| *** yamamoto has quit IRC | 11:51 | |
| *** jkilpatr has joined #openstack-infra | 11:53 | |
| *** sree has joined #openstack-infra | 11:53 | |
| *** jkilpatr has quit IRC | 11:53 | |
| *** jkilpatr_ has joined #openstack-infra | 11:54 | |
| *** yamamoto has joined #openstack-infra | 11:55 | |
| *** cshastri has joined #openstack-infra | 11:56 | |
| *** sree has quit IRC | 11:58 | |
| *** jpena is now known as jpena|lunch | 11:59 | |
| *** annp has quit IRC | 12:00 | |
| *** smatzek has joined #openstack-infra | 12:05 | |
| *** amoralej is now known as amoralej|lunch | 12:07 | |
| *** shardy has quit IRC | 12:07 | |
| *** andreas_s has quit IRC | 12:08 | |
| tobberydberg | Thanks frickler! I will have a look at that! | 12:08 |
| *** dhajare|BRB has joined #openstack-infra | 12:10 | |
| *** rosmaita has joined #openstack-infra | 12:12 | |
| *** andreas_s has joined #openstack-infra | 12:13 | |
| *** dhajare|BRB has quit IRC | 12:16 | |
| *** gouthamr has quit IRC | 12:16 | |
| *** gouthamr has joined #openstack-infra | 12:17 | |
| *** dhajare|BRB has joined #openstack-infra | 12:17 | |
| AJaeger | cmurphy: this happens if several requirements jobs just merged and race in psot queue. This is second time this week I see it, we should use a mutex to run them one after the other... | 12:18 |
| *** eharney has joined #openstack-infra | 12:19 | |
| *** andreas_s has quit IRC | 12:22 | |
| *** andreas_s has joined #openstack-infra | 12:23 | |
| *** spzala has joined #openstack-infra | 12:29 | |
| *** edmondsw has joined #openstack-infra | 12:30 | |
| *** andreas_s has quit IRC | 12:33 | |
| *** spzala has quit IRC | 12:33 | |
| *** andreas_s has joined #openstack-infra | 12:34 | |
| *** andreas_s has quit IRC | 12:34 | |
| *** andreas_s has joined #openstack-infra | 12:35 | |
| *** edmondsw has quit IRC | 12:35 | |
| *** rossella_s has quit IRC | 12:39 | |
| openstackgerrit | Tobias Rydberg proposed openstack-infra/system-config master: Adding meetbot for #openstack-publiccloud channel https://review.openstack.org/535318 | 12:40 |
| *** flepied has joined #openstack-infra | 12:40 | |
| *** rossella_s has joined #openstack-infra | 12:41 | |
| openstackgerrit | Merged openstack-infra/zuul-jobs master: Remove testr and stestr specific roles https://review.openstack.org/529340 | 12:43 |
| *** kgiusti has quit IRC | 12:48 | |
| *** rhallisey has joined #openstack-infra | 12:53 | |
| AJaeger | mordred: done ;) ^ | 12:53 |
| *** rossella_s has quit IRC | 12:54 | |
| *** caphrim007 has joined #openstack-infra | 12:55 | |
| openstackgerrit | Merged openstack-infra/irc-meetings master: Updating the list of active chairs for the Public Cloud WG https://review.openstack.org/535286 | 12:55 |
| *** rossella_s has joined #openstack-infra | 12:57 | |
| *** yamamoto has quit IRC | 12:58 | |
| *** caphrim007 has quit IRC | 12:59 | |
| *** yamamoto has joined #openstack-infra | 13:01 | |
| *** agopi has joined #openstack-infra | 13:04 | |
| *** dsariel has quit IRC | 13:04 | |
| *** dsariel has joined #openstack-infra | 13:05 | |
| *** coolsvap has quit IRC | 13:05 | |
| *** jpena|lunch is now known as jpena | 13:06 | |
| *** bobh has joined #openstack-infra | 13:10 | |
| *** ianychoi has quit IRC | 13:12 | |
| *** psachin has quit IRC | 13:15 | |
| *** cshastri_ has joined #openstack-infra | 13:16 | |
| *** yamamoto has quit IRC | 13:16 | |
| *** cshastri has quit IRC | 13:18 | |
| *** _Cyclone_ has joined #openstack-infra | 13:19 | |
| *** dave-mccowan has joined #openstack-infra | 13:23 | |
| *** edmondsw has joined #openstack-infra | 13:23 | |
| *** rlandy has joined #openstack-infra | 13:27 | |
| *** gouthamr_ has joined #openstack-infra | 13:28 | |
| *** ykarel is now known as ykarel|away | 13:28 | |
| *** ldnunes has quit IRC | 13:29 | |
| *** gouthamr has quit IRC | 13:31 | |
| *** oidgar has quit IRC | 13:31 | |
| *** dhajare|BRB has quit IRC | 13:31 | |
| *** bobh has quit IRC | 13:33 | |
| *** yamamoto has joined #openstack-infra | 13:35 | |
| *** kjackal has quit IRC | 13:38 | |
| *** ykarel|away has quit IRC | 13:40 | |
| rosmaita | AJaeger, mordred if either of you have a minute, i'm getting post failures on glance_store experimental functional gates, i'm probably doing something stupid in .zuul.yaml that you will hopefully notice right away: https://review.openstack.org/#/c/526956/13 | 13:40 |
| *** ldnunes has joined #openstack-infra | 13:41 | |
| AJaeger | config-core, please review https://review.openstack.org/535191 and https://review.openstack.org/511150 for removal of trove jobs. | 13:41 |
| *** zoli is now known as zoli|dentist | 13:42 | |
| *** zoli|dentist is now known as zoli | 13:42 | |
| AJaeger | rosmaita: something for mordred - see http://logs.openstack.org/56/526956/13/experimental/glance_store-dsvm-functional-filesystem/f8ed233/ara/result/a31ddac1-bd53-46a5-bfe7-2b461738e2e7/ | 13:43 |
| AJaeger | rosmaita: ara really helped to find this quickly... | 13:43 |
| AJaeger | mordred: error is "/bin/sh: 1: .tox/functional-filesystem/bin/testr: not found" | 13:44 |
| *** kgiusti has joined #openstack-infra | 13:45 | |
| *** alexchadin has quit IRC | 13:46 | |
| rosmaita | AJaeger that is much better than reading through the log! | 13:46 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove Neutron legacy jobs definition https://review.openstack.org/531496 | 13:48 |
| *** alexchadin has joined #openstack-infra | 13:48 | |
| *** jamesmcarthur has joined #openstack-infra | 13:49 | |
| *** chandankumar has quit IRC | 13:51 | |
| *** weshay is now known as weshay|rover | 13:52 | |
| *** chandankumar has joined #openstack-infra | 13:52 | |
| *** gouthamr_ has quit IRC | 13:53 | |
| *** vsaienk0 has quit IRC | 13:53 | |
| *** gouthamr has joined #openstack-infra | 13:53 | |
| *** ianychoi has joined #openstack-infra | 13:55 | |
| *** zoli is now known as zoli|dentist | 13:56 | |
| *** zoli|dentist is now known as zoli | 13:56 | |
| *** oidgar has joined #openstack-infra | 13:57 | |
| *** sshnaidm|afk is now known as sshnaidm | 13:57 | |
| *** oidgar has quit IRC | 13:59 | |
| *** dprince has joined #openstack-infra | 13:59 | |
| *** rfolco has quit IRC | 13:59 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/system-config master: Add nb01.o.o to zookeeper firewall rules https://review.openstack.org/535332 | 13:59 |
| *** rfolco has joined #openstack-infra | 14:00 | |
| *** spzala has joined #openstack-infra | 14:00 | |
| pabelanger | infra-root: ^ bring opens firewalls for nb01.o.o to talk with zookeeper. Now running under python3 and feature/zuulv3 branch | 14:01 |
| *** vsaienk0 has joined #openstack-infra | 14:02 | |
| fungi | pabelanger: did you also have a patch for the systemd reconfig? | 14:02 |
| pabelanger | fungi: ya, working on that now | 14:02 |
| fungi | cool, didn't know whether i should go looking yet | 14:02 |
| *** kiennt26_ has joined #openstack-infra | 14:04 | |
| *** kiennt26_ has quit IRC | 14:04 | |
| openstackgerrit | Merged openstack-infra/project-config master: Rename trove grenade job https://review.openstack.org/535191 | 14:04 |
| *** spzala has quit IRC | 14:05 | |
| *** dhill_ has joined #openstack-infra | 14:06 | |
| *** kiennt26 has quit IRC | 14:06 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-nodepool master: Support systemdctl daemon-reload for nodepool services https://review.openstack.org/535337 | 14:06 |
| *** sshnaidm is now known as sshnaidm|afk | 14:06 | |
| *** sshnaidm|afk is now known as sshnaidm|mtg | 14:07 | |
| *** jistr|mtg is now known as jistr | 14:08 | |
| pabelanger | fungi: ^should fix things up | 14:09 |
| *** Goneri has joined #openstack-infra | 14:10 | |
| *** nikhil has joined #openstack-infra | 14:11 | |
| openstackgerrit | Kiran Thyagaraja proposed openstack-infra/project-config master: Add ansible-role-k8s-cinder https://review.openstack.org/534594 | 14:14 |
| *** oidgar has joined #openstack-infra | 14:15 | |
| *** claudiub has quit IRC | 14:15 | |
| *** saop has quit IRC | 14:15 | |
| *** flaper87 has quit IRC | 14:16 | |
| *** flaper87 has joined #openstack-infra | 14:16 | |
| *** dayou has joined #openstack-infra | 14:17 | |
| *** ramishra_ has joined #openstack-infra | 14:17 | |
| *** ramishra has quit IRC | 14:17 | |
| *** ramishra_ is now known as ramishra | 14:18 | |
| *** shardy has joined #openstack-infra | 14:20 | |
| *** eumel8 has quit IRC | 14:21 | |
| *** smatzek has quit IRC | 14:21 | |
| *** andreas_s has quit IRC | 14:24 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Retire craton (1/2) https://review.openstack.org/535344 | 14:25 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Retire craton (2/2) https://review.openstack.org/535345 | 14:25 |
| *** ykarel|away has joined #openstack-infra | 14:25 | |
| *** jaosorior has quit IRC | 14:25 | |
| *** esberglu has joined #openstack-infra | 14:26 | |
| AJaeger | config-core, could you review https://review.openstack.org/535344 , please? I don't like to patch dead projects anymore... | 14:27 |
| smcginnis | ++ | 14:27 |
| AJaeger | thanks, smcginnis | 14:28 |
| * AJaeger just abandoned his latest review after trying to get some reply for the Nth time ;( | 14:28 | |
| *** andreas_s has joined #openstack-infra | 14:28 | |
| *** lathiat has quit IRC | 14:29 | |
| smcginnis | I'm all for cleaning up old things. | 14:29 |
| *** danpawlik has quit IRC | 14:29 | |
| AJaeger | especially if they are dead... | 14:29 |
| *** esberglu has quit IRC | 14:30 | |
| *** dayou has quit IRC | 14:30 | |
| *** gouthamr has quit IRC | 14:31 | |
| *** esberglu has joined #openstack-infra | 14:33 | |
| *** kjackal has joined #openstack-infra | 14:34 | |
| AJaeger | dmsimard: thanks for recheck on https://review.openstack.org/511150, it's ready now to +2A - could you review, please? | 14:37 |
| AJaeger | thanks, dmsimard ! | 14:38 |
| *** kjackal has quit IRC | 14:38 | |
| dmsimard | AJaeger: sorry for not reviewing much recently :( | 14:39 |
| dave-mccowan | dhellman ping | 14:39 |
| mordred | morning everybody | 14:39 |
| dave-mccowan | Hi AJaeger. That gate failure in python-barbican client yesterday is due to a change in cliff. (they added a -s parameter that conflicts with our -s parameter). | 14:40 |
| AJaeger | dave-mccowan: I saw patches by dhellmann for cliff - let's ask him whether a new release is out yet | 14:40 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/storyboard master: suppress warnings from wsmeext.sphinxext https://review.openstack.org/534872 | 14:41 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/storyboard master: treat doc build warnings as errors https://review.openstack.org/534873 | 14:41 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/storyboard master: import blog post "Mapping Launchpad to StoryBoard" https://review.openstack.org/534420 | 14:41 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/storyboard master: import blog post "Why StoryBoard for OpenStack?" https://review.openstack.org/534421 | 14:41 |
| AJaeger | dmsimard: mnaser had a long flight ;) Still quite a few open again, so if you can help, it would be great. | 14:41 |
| AJaeger | morning, mordred | 14:41 |
| dmsimard | AJaeger: I should have some time today, will try and go through as many as I can | 14:42 |
| dhellmann | AJaeger, dave-mccowan : we have a fix waiting to be approved in cliff and I've queued up a release | 14:42 |
| mordred | AJaeger: any fires I should be looking at? | 14:42 |
| *** ykarel|away has quit IRC | 14:42 | |
| AJaeger | dmsimard, mordred, pabelanger, could you put https://review.openstack.org/535344 on your queue, please? | 14:42 |
| AJaeger | not a fire ;) | 14:42 |
| dave-mccowan | dhellmann thanks! | 14:42 |
| dhellmann | AJaeger, dave-mccowan : if no one has approved the fix in an hour or two I'll self-approve so we can move on | 14:42 |
| AJaeger | mordred: see backscroll from rosmaita - something strange going on in glance with the testr role | 14:42 |
| AJaeger | dhellmann: thanks | 14:43 |
| rosmaita | "something strange going on in glance" -- it's the story of my life! | 14:43 |
| *** gouthamr has joined #openstack-infra | 14:43 | |
| AJaeger | sorry, rosmaita ! | 14:43 |
| AJaeger | ;) | 14:43 |
| *** andreas_s has quit IRC | 14:43 | |
| *** dayou has joined #openstack-infra | 14:44 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove Trove legacy jobs https://review.openstack.org/511150 | 14:47 |
| dhellmann | could I get someone to add me to the new review group for the goal-tools repo, please? https://review.openstack.org/#/admin/groups/1859,members | 14:49 |
| *** oidgar has quit IRC | 14:49 | |
| *** smatzek has joined #openstack-infra | 14:50 | |
| *** makowals has quit IRC | 14:52 | |
| *** makowals has joined #openstack-infra | 14:54 | |
| *** olaph has joined #openstack-infra | 14:54 | |
| *** olaph1 has quit IRC | 14:55 | |
| *** jamesmcarthur has quit IRC | 14:55 | |
| *** andreas_s has joined #openstack-infra | 15:00 | |
| *** oidgar has joined #openstack-infra | 15:03 | |
| dmsimard | dhellmann: done | 15:03 |
| *** jcoufal has joined #openstack-infra | 15:05 | |
| *** amoralej|lunch is now known as amoralej | 15:06 | |
| *** efoley has joined #openstack-infra | 15:09 | |
| openstackgerrit | Merged openstack-infra/system-config master: Add nb01.o.o to zookeeper firewall rules https://review.openstack.org/535332 | 15:09 |
| mtreinish | infra-root: it looks like the mqtt_statsd daemon stopped updating statsd on 1/10: http://grafana.openstack.org/dashboard/db/mosquitto-status?from=1511104296926&to=1516288296926 | 15:12 |
| mtreinish | can someone check if it died and maybe kick it? | 15:12 |
| dmsimard | that sounds like when the meltdown/spectre patches were applied, I'll look | 15:12 |
| *** makowals has quit IRC | 15:12 | |
| dhellmann | dmsimard : thanks! | 15:15 |
| * fungi wishes the systemd/journald devs knew people still used terminals | 15:16 | |
| fungi | mtreinish: dmsimard: http://paste.openstack.org/show/646913/ | 15:16 |
| mtreinish | fungi: :/ nothing in the logs about the failure? | 15:17 |
| fungi | would it be in mosquitto.log? | 15:17 |
| *** jamesmcarthur has joined #openstack-infra | 15:17 | |
| *** pahuang has quit IRC | 15:18 | |
| mtreinish | probably not, iirc it would just print to stdout so it'd be in the journal | 15:18 |
| * fungi figures out which manpage he should be looking at | 15:19 | |
| mtreinish | fungi: 'journalctl --unit mqtt_statsd' ? | 15:19 |
| fungi | cool, so i guessed the right manpage, just hadn't read far enough yet to find that services are called units | 15:20 |
| fungi | -- No entries -- | 15:20 |
| fungi | i'll see if maybe it gets copied to a real file like syslog or something | 15:21 |
| *** hamzy has quit IRC | 15:21 | |
| fungi | that won't help either, we only keep a week of syslog | 15:21 |
| fungi | oldest syslog entries are from the rotation at Jan 11 06:25:01 | 15:22 |
| sean-k-mooney | qq is the requirement repo owned/reviewed by infra or is there a seperate irc channel for requirements changes? | 15:22 |
| fungi | sean-k-mooney: #openstack-requirements | 15:22 |
| dmsimard | sean-k-mooney: #openstack-requirements ? | 15:22 |
| *** armax has joined #openstack-infra | 15:22 | |
| sean-k-mooney | ah thank you :) | 15:22 |
| fungi | it's an official project team under the technical committee | 15:22 |
| sean-k-mooney | that makes sense | 15:22 |
| *** hongbin has joined #openstack-infra | 15:22 | |
| mtreinish | fungi: hmm, I guess just restart it and cross our fingers | 15:22 |
| fungi | sean-k-mooney: https://governance.openstack.org/tc/reference/projects/requirements.html | 15:22 |
| fungi | mtreinish: yep, that's what i'm going to try next | 15:22 |
| *** alexchadin has quit IRC | 15:22 | |
| *** jbadiapa has quit IRC | 15:22 | |
| fungi | sean-k-mooney: it was under the remit of the release management team until it got split out (similar to the stable branch team in that regard) | 15:23 |
| dmsimard | mtreinish: I see those in mosquitto.log but not sure if it's normal 1516288249: OpenSSL Error: error:140F3042:SSL routines:ssl_undefined_const_function:called a function you should not call | 15:23 |
| AJaeger | fungi, dmsimard, could you put https://review.openstack.org/535344 on your review queue, please? | 15:23 |
| dmsimard | AJaeger: I wonder, is there some sort of inactivity "timeout" where we eventually retired unmaintained projects ? | 15:24 |
| mtreinish | dmsimard: we've had that problem with some clients when the CA cert isn't setup properly and they try to use the ssl port (which is a requirement for clients publishing to mqtt) | 15:25 |
| *** felipemonteiro_ has joined #openstack-infra | 15:25 | |
| mtreinish | but I thought we fixed the services that had that issue | 15:25 |
| AJaeger | dmsimard: none - I asked their release team after pinging them several times in reviews and seeing no reacion | 15:25 |
| fungi | mtreinish: it's running now. i wonder if we have a startup race at boot where it attempts to run before unbound has dns resolution up and running | 15:25 |
| AJaeger | dmsimard: so, he told me to retire ;) | 15:26 |
| *** felipemonteiro__ has joined #openstack-infra | 15:26 | |
| mtreinish | fungi: it could be, I didn't declare any dependencies in the systemd unit file. So who knows when systemd will start it up | 15:26 |
| dmsimard | AJaeger: ah ok, maybe we should have some kind of rule like that -- otherwise people are expecting the project to work, be maintained and especially be secure | 15:27 |
| *** b_bezak has quit IRC | 15:27 | |
| dmsimard | Would probably help cleaning up a lot of the cruft that has accumulated over the years too. | 15:27 |
| fungi | dmsimard: identifying "inactivity" is non-trivial (what variety of activity do you monitor? just lack of new changes/releases, or reviews and bugs going unaddressed or failure to hold meetings or dead irc channels or lack of communication specific to that project on mailing lists, or...) | 15:27 |
| dmsimard | fungi: yeah I'm not implying it's easy | 15:27 |
| *** b_bezak has joined #openstack-infra | 15:27 | |
| openstackgerrit | Eric Berglund proposed openstack-infra/os-loganalyze master: WIP: Support ANSI color codes in HTMLView https://review.openstack.org/524658 | 15:28 |
| dmsimard | but the other extreme is do you just keep everything forever ? | 15:28 |
| fungi | yep | 15:28 |
| AJaeger | dmsimard: if I push infra changes around and get no answers, I get curious ;) | 15:28 |
| AJaeger | dmsimard: I prefer not to do this twice... | 15:28 |
| fungi | or to look at similar examples, how does github decide which repos should get automatically retired/removed? | 15:28 |
| *** oidgar has quit IRC | 15:29 | |
| fungi | being able to find software on the internet doesn't automatically mean it's a good idea to use and rely on it | 15:29 |
| dmsimard | fungi: Yeah, I see what you mean | 15:29 |
| fungi | or that it necessarily ever even worked to start with | 15:30 |
| *** felipemonteiro__ has quit IRC | 15:30 | |
| *** felipemonteiro_ has quit IRC | 15:30 | |
| dmsimard | I've been staying mostly quiet about this but all this IRC stuff recently made me realize I don't know what a large percentage of our IRC channels are about | 15:30 |
| *** felipemonteiro__ has joined #openstack-infra | 15:30 | |
| dmsimard | this list of channels: http://paste.openstack.org/raw/629168/ | 15:30 |
| AJaeger | dmsimard: let'S retire, so you have one topic less to learn ;) | 15:31 |
| *** pahuang has joined #openstack-infra | 15:31 | |
| fungi | take the openstack/meghdwar repo for exmaple... ;) | 15:31 |
| fungi | created 10 months ago, still has only its initial .gitreview commit | 15:31 |
| fungi | er, 19 months ago i mean | 15:31 |
| dmsimard | I saw an openstack project once that was like... cat as a service or something like that, and it made error pages cats or something | 15:32 |
| dmsimard | forgot the name | 15:32 |
| *** b_bezak has quit IRC | 15:32 | |
| *** spzala has joined #openstack-infra | 15:32 | |
| fungi | at least it _did_ something | 15:33 |
| dmsimard | yeah :D | 15:33 |
| *** jamesmcarthur has quit IRC | 15:36 | |
| *** jamesmcarthur has joined #openstack-infra | 15:37 | |
| *** links has quit IRC | 15:38 | |
| *** slaweq has quit IRC | 15:39 | |
| *** olaph1 has joined #openstack-infra | 15:40 | |
| *** olaph has quit IRC | 15:41 | |
| *** camunoz has joined #openstack-infra | 15:42 | |
| *** jamesmcarthur has quit IRC | 15:42 | |
| *** ramishra has quit IRC | 15:44 | |
| openstackgerrit | Merged openstack-infra/project-config master: Retire craton (1/2) https://review.openstack.org/535344 | 15:45 |
| mtreinish | fungi: what do you think between: https://raw.githubusercontent.com/mtreinish/firehose/lca-sysadmin-miniconf/sub_throughput.png (our original from boston) and https://i.imgur.com/DAgmKqS.png | 15:47 |
| *** hjensas is now known as hjensas|afk | 15:48 | |
| fungi | mtreinish: like a heatmap? neat way of presenting it | 15:48 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Migrate nodepool.yaml to new syntax https://review.openstack.org/535382 | 15:48 |
| fungi | mtreinish: i also wonder if some sort of volumetric characterization would be relevant? we probably don't have enough axes and datapoints for that to really be warranted i guess | 15:49 |
| *** alex_xu has quit IRC | 15:50 | |
| *** alex_xu has joined #openstack-infra | 15:52 | |
| mtreinish | fungi: yeah we'd probably need another axis for that. There also aren't that many datapoints which is why that 2d plot looks so blocky | 15:52 |
| fungi | yup | 15:53 |
| corvus | mtreinish: you could also do a multiple line graph, where each line corresponds to a specific number of publishers, X is number of subscribers, and Y is throughput. | 15:53 |
| pabelanger | infra-root: nodepool.yaml changes for feature/zuulv3 builders^. I also like to discuss how we can manage nodepool configs moving forward too. Maybe not right now, but now we have 3 different nodepool yaml files and things like our diskimages and providers sections now are copy and pasted between them. i think supporting something like nodepool.d (split configure) or starting to template it via | 15:53 |
| pabelanger | puppet (harder to managing I think) are some options on moving forward. | 15:53 |
| corvus | mtreinish: (i've been looking at a lot of frequency response graphs lately...) | 15:53 |
| fungi | mtreinish: i think what you have there is at least likely more able to get the point across than our sheet graph from before | 15:54 |
| *** janki has quit IRC | 15:55 | |
| fungi | mtreinish: also from a presenter notes standpoint, it's helpful to mention that the vacant triangle is simply indicating values which would be above the available network bandwidth of the system under test | 15:55 |
| mtreinish | corvus: I'd definitely give that a try if we had data for all subscriber counts for each number of publisher. But, we were hitting up against node bandwidth limits in our tests and couldn't scale both past a certain point. Which is where that diagonal comes from in the graphs | 15:56 |
| mtreinish | heh, what fungi said :) | 15:56 |
| corvus | fungi, mtreinish: oh! i did not get that from the graph. i wondered about that. | 15:56 |
| corvus | (maybe a diagonal line there, and on the right "exceeded available bandwidth" would help) | 15:57 |
| fungi | same reason the sheet graph is hard to interpret, because it's a sheet stretched between three points instead of four | 15:57 |
| mtreinish | corvus: yeah, I have to figure out how to do overlay annotation in matplotlib | 15:58 |
| * mtreinish doesn't want to cheat and use gimp | 15:59 | |
| mordred | AJaeger, rosmaita: looking | 15:59 |
| tosky | AJaeger: if I observe a legacy post job, automatically converted, which does not publish anymore the generate tarball, should I try to fix it or rewrite it or hope for some help? | 16:00 |
| corvus | mtreinish: if you write a function that uses gimp-fu (or whatever it's called this century), it's not cheating! | 16:00 |
| fungi | tosky: i think it depends on where the fault in the build lies. a link to an example failure log would be useful | 16:03 |
| *** sree has joined #openstack-infra | 16:03 | |
| tosky | fungi: I didn't see a build failure, and the post job passed for what I can see, but no upload | 16:05 |
| tosky | let me find the build | 16:05 |
| *** hamzy has joined #openstack-infra | 16:05 | |
| mtreinish | corvus: luckily it doesn't look that hard to do in matplotlib: https://matplotlib.org/users/annotations.html | 16:06 |
| openstackgerrit | Bernard Cafarelli proposed openstack-infra/system-config master: Add #networking-sfc to statusbot and meetbot https://review.openstack.org/535390 | 16:06 |
| *** hamzy has quit IRC | 16:07 | |
| *** hamzy has joined #openstack-infra | 16:07 | |
| *** makowals has joined #openstack-infra | 16:08 | |
| tosky | oh, I know where the problem is | 16:08 |
| tosky | fungi: this is the build: https://review.openstack.org/#/c/532690 and this is the post-build: http://zuulv3.openstack.org/builds.html?job_name=legacy-post-sahara-extra-artifacts | 16:08 |
| tosky | but it uploaded to sahara-extra/, not sahara/ as it happened before | 16:09 |
| *** yamamoto has quit IRC | 16:09 | |
| tosky | so http://tarballs.openstack.org/sahara-extra/dist/ instead of http://tarballs.openstack.org/sahara/dist/ | 16:09 |
| *** dbecker has quit IRC | 16:09 | |
| *** olaph1 has quit IRC | 16:10 | |
| *** olaph has joined #openstack-infra | 16:10 | |
| *** makowals has quit IRC | 16:12 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: set up governance repo for storyboard https://review.openstack.org/535393 | 16:12 |
| tosky | now, let's see if I can fix it... | 16:12 |
| tosky | or if it's even possible to restore the old behavior | 16:13 |
| clarkb | pabelanger: if we split it into base config (zk connection details), config for each provider so they can be composed where necessary, then label and diskimage config we would probably be good. Another option is to have the launchers stop being distinct sets and have them overlap then we can use the same config everywhere | 16:13 |
| AJaeger | config-core, could you review https://review.openstack.org/533762 to setup dhellmann's new repo goal-tools, please? | 16:13 |
| AJaeger | config-core, two review requests for new repos: https://review.openstack.org/531604 and https://review.openstack.org/531608, please | 16:13 |
| dhellmann | AJaeger : do you know if turning on the flag for storyboard in gerrit/projects.yaml is all I need for https://review.openstack.org/535393 ? | 16:13 |
| dhellmann | I suppose I should look at the migrating doc | 16:14 |
| AJaeger | dhellmann: fungi is the expert ^ | 16:14 |
| *** yamamoto has joined #openstack-infra | 16:14 | |
| AJaeger | dhellmann: AFAIK yes | 16:14 |
| pabelanger | clarkb: Yah, I think launchers would need to be updates to support same config. IIRC, we cannot share providers across launchers today | 16:14 |
| clarkb | pabelanger: I don't think the new quota checking allows us to do shared config between launchers but I think its a step towards that | 16:14 |
| *** hamzy_ has joined #openstack-infra | 16:14 | |
| pabelanger | yah | 16:15 |
| *** gema has quit IRC | 16:15 | |
| *** hamzy has quit IRC | 16:17 | |
| corvus | clarkb, pabelanger: the old quota checking (in nodepool v3) didn't allow that either. a provider can only be in one launcher. | 16:18 |
| AJaeger | fungi, can you confirm that https://review.openstack.org/#/c/535393/ is fine, please? | 16:19 |
| *** sree_ has joined #openstack-infra | 16:19 | |
| *** sree_ is now known as Guest65258 | 16:19 | |
| clarkb | ya we'd have to account for the quota available and used in zookeeper so that all launchers could see a common view of that | 16:19 |
| *** sree has quit IRC | 16:22 | |
| *** e0ne has quit IRC | 16:23 | |
| openstackgerrit | Merged openstack-infra/project-config master: add zuul settings for goal-tools repository https://review.openstack.org/533762 | 16:23 |
| thingee | Hi infra, i have three small patches with 2 +2's... just need an approval. https://review.openstack.org/#/q/status:open+topic:statusbot-pageurl | 16:25 |
| fungi | dhellmann: AJaeger: yes, though you may also want to add it to one or more (perhaps new?) project groups, though that can also be done later by adding to gerrit/project.yaml | 16:26 |
| fungi | looking at 535393 | 16:26 |
| dhellmann | fungi : "project groups"? | 16:26 |
| *** ramishra has joined #openstack-infra | 16:27 | |
| fungi | dhellmann: you'll see some projects in there with a "groups" key and a list of one or more groups | 16:27 |
| clarkb | In the proposed PTG schedule we have a similar setup to denver. 2 days of help room then 3 days of infra hacking time | 16:27 |
| fungi | dhellmann: sb has a project group concept in its data model where you can look at a set of related projects or list their collective stories by group | 16:27 |
| *** yamamoto has quit IRC | 16:27 | |
| dhellmann | fungi : so maybe something like "tc"? | 16:27 |
| dhellmann | or "governance"? | 16:27 |
| clarkb | My expectation is that the 2 days of helproom will be busier this time around due to the zuulv3 rollout | 16:27 |
| fungi | dhellmann: sure, or governance or yeah | 16:27 |
| dhellmann | ok, let me add that | 16:28 |
| fungi | dhellmann: but as i said, you can also just save that idea for later when there's more than one to worry about grouping | 16:28 |
| fungi | entirely up to you | 16:28 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: set up governance repo for storyboard https://review.openstack.org/535393 | 16:28 |
| dhellmann | meh, done ^^ | 16:28 |
| clarkb | does anyone see a problem with the proposed schedule being 2 days of helproom then 3 days of infra hacking time during the dublin ptg? cc infra-root config-core | 16:28 |
| *** felipemonteiro__ has quit IRC | 16:29 | |
| fungi | clarkb: sounds basically like denver? fine by me | 16:29 |
| *** pcaruana has quit IRC | 16:29 | |
| fungi | i was just looking over that schedule a little while ago | 16:29 |
| fungi | we seem to have a room big enough for 30 people on wednesday through friday | 16:30 |
| *** liusheng has quit IRC | 16:30 | |
| fungi | and a 50-person room shared with the other horizontal teams for community help on monday/tuesday | 16:30 |
| *** liusheng has joined #openstack-infra | 16:31 | |
| clarkb | ya the rooms should be big enough. We probably don't need quite as much as a 30 person room wed-fri but I expect 16 person room may be too small | 16:32 |
| pabelanger | fungi: clarkb: I think it is fine, but I did hear some feedback that people didn't know infra was in the shared room on monday/tuesday for getting support. So maybe this time around we need to over communicate that? I know a few projects shows up on wed/thurs/fri to talk design stuff | 16:32 |
| clarkb | pabelanger: ya, though last time I specifically sent email to the dev list re specific topics that would be perfect for the helproom days like tripleo and that was ignored :/ so not sure how to communicate that better | 16:33 |
| persia | From visiting past infra rooms, it seemed like there were often many things going on simultaneously: maybe 2 16-person rooms could be useful? | 16:33 |
| clarkb | persia: ya I think the shared room with infra, qa, release, requirements etc does sort of force other projects to be assertive to sort out whats what to get what they need. Making that easier would like help | 16:34 |
| *** ramishra has quit IRC | 16:34 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Make subunit processing more resilient https://review.openstack.org/534431 | 16:35 |
| pabelanger | clarkb: yah, guess we need to send it twice? But agree | 16:35 |
| mordred | AJaeger, rosmaita: ^^ + pushed up a glance_store DNM patch with depends-on | 16:35 |
| persia | clarkb: Also forces out-of-room time for folk who want to have a meeting. I remember release team especially doing this a couple times, where they didn't want to disturb others (so just were not there). | 16:35 |
| clarkb | persia: probably something that would help if not changing the rooms is better identification "I'm Infra!" and better "I'm Idle!" advertising | 16:36 |
| clarkb | another idea I had was for us to float more | 16:37 |
| clarkb | so rather than people coming to us we go to them | 16:37 |
| clarkb | that might actually just be simplest. Ping in the ptg channel for help and someone wanders off to help rather than forcing people into the helproom | 16:38 |
| clarkb | and then the big room gives infra, qa, rel, req folk a place to work when not otherwise roaming | 16:39 |
| *** yamamoto has joined #openstack-infra | 16:39 | |
| *** oidgar has joined #openstack-infra | 16:42 | |
| persia | clarkb: Maybe not time for Dublin, but maybe in future have an infra rep go to a weekly meeting for all the other attending projects and ask "When is someone from that project coming to the infra room?" or "When should someone from infra come to the project room?" | 16:42 |
| pabelanger | clarkb: yah, float is a good idea. Maybe we should try and work up some sort of schedule of interested people | 16:42 |
| pabelanger | last PTG we really were head down on zuulv3, I feel this time around we might have more room to float | 16:42 |
| persia | key is to avoid float-as-clump :) | 16:43 |
| *** yamamoto has quit IRC | 16:44 | |
| SamYaple | would you prefer to float-as-cloud instead? | 16:45 |
| persia | SamYaple: Absolutely: on-demand API-instantiable infra-rep delivered to any project room as needed. Might be hard on the folk actually floating though. | 16:46 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Migrate nodepool.yaml to new syntax https://review.openstack.org/535382 | 16:46 |
| SamYaple | hahaha | 16:47 |
| SamYaple | thats a good answer | 16:47 |
| SamYaple | all core members must where pagers. done | 16:47 |
| *** yamamoto has joined #openstack-infra | 16:49 | |
| *** yamamoto has quit IRC | 16:49 | |
| clarkb | I think we'd all quit | 16:50 |
| fungi | yeah, i had that job for 15 years... not going back ;) | 16:51 |
| tosky | sooo, going back to my post-job, which published to a different place than before | 16:51 |
| *** andreas_s has quit IRC | 16:51 | |
| tosky | can I simply ask to have http://tarballs.openstack.org/sahara/dist/ changed as symlink to http://tarballs.openstack.org/sahara-extra/dist/, or can I change the job to publish to sahara/, or should I update all references? | 16:52 |
| *** andreas_s has joined #openstack-infra | 16:52 | |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: turn off python 3.5 jobs for goal-tools repo https://review.openstack.org/535405 | 16:53 |
| *** andreas_s_ has joined #openstack-infra | 16:53 | |
| dhellmann | AJaeger , clarkb : I jumped the gun adding the python35 jobs to the goal-tools repo ^^ | 16:54 |
| dhellmann | we have to figure out what structure makes sense in there before we start adding test jobs | 16:54 |
| *** andreas__ has joined #openstack-infra | 16:54 | |
| *** jamesmcarthur has joined #openstack-infra | 16:54 | |
| clarkb | dhellmann: reviewed, lbragstad is right you need something there (maybe just noop-jobs) | 16:55 |
| dhellmann | ah, ok | 16:55 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/project-config master: turn off python 3.5 jobs for goal-tools repo https://review.openstack.org/535405 | 16:55 |
| dhellmann | clarkb : thanks | 16:56 |
| *** sshnaidm|mtg has quit IRC | 16:56 | |
| *** andreas_s has quit IRC | 16:56 | |
| *** andreas_s_ has quit IRC | 16:57 | |
| *** andreas__ has quit IRC | 16:59 | |
| pabelanger | clarkb: fungi: Shrews: corvus: mind adding https://review.openstack.org/535382/ to review pipeline. That will allow us to bring nb01 online today | 17:00 |
| *** ldnunes has quit IRC | 17:00 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul feature/zuulv3: Handle nodesets in branches https://review.openstack.org/534974 | 17:00 |
| clarkb | pabelanger: are both of the old builders off at this point? | 17:01 |
| pabelanger | clarkb: yes | 17:01 |
| pabelanger | once nb01 is online and working, I was going to delete nb03 / nb04 and bring nb02 online | 17:02 |
| pabelanger | then shift effort to maybe rebuilding nodepool-launchers to smaller instances(2GB) and seeing if we want to add more | 17:02 |
| clarkb | pabelanger: for the builder config should we have min-ready and max-servers set to 0? | 17:03 |
| openstackgerrit | Eric Berglund proposed openstack-infra/os-loganalyze master: WIP: Support ANSI color codes in HTMLView https://review.openstack.org/524658 | 17:03 |
| *** linkmark has joined #openstack-infra | 17:03 | |
| pabelanger | clarkb: I think we could remove that, but opted to keep the syntax the same as nl01 / nl02 right now. I think we could have the discussion how to best share the data, after we landed it | 17:04 |
| *** Guest65258 has quit IRC | 17:05 | |
| corvus | pabelanger, clarkb: the only thing the builders need are "providers: ...: diskimages:" and "diskimages:" | 17:05 |
| corvus | don't need labels or provider-pool configs at all | 17:05 |
| clarkb | in that case it would probably be nice to make the config minimal | 17:06 |
| *** andreas_s has joined #openstack-infra | 17:06 | |
| *** sree has joined #openstack-infra | 17:06 | |
| pabelanger | okay, I am do that now if you'd like | 17:06 |
| clarkb | wfm either way | 17:06 |
| openstackgerrit | Bernard Cafarelli proposed openstack-infra/release-tools master: Add StoryBoard filter and tag tools https://review.openstack.org/534808 | 17:06 |
| clarkb | (but I think that makes it more clear what we actually need to sort out when ti comes time to refactor the config management for the configs) | 17:07 |
| pabelanger | clarkb: I'll push up a follow since corvus has +2 | 17:07 |
| clarkb | ok | 17:07 |
| corvus | also, fwiw, i don't think the launchers need a "diskimages:" section. but they currently have one (and so are not minimal) | 17:07 |
| pabelanger | yah, i was hoping to test that too | 17:08 |
| *** vsaienk0 has quit IRC | 17:11 | |
| clarkb | the old tox siblings + release notes job issues were sorted out right? | 17:11 |
| clarkb | AJaeger: ^ you probably know | 17:11 |
| *** andreas_s has quit IRC | 17:12 | |
| *** ldnunes has joined #openstack-infra | 17:12 | |
| *** jcoufal has quit IRC | 17:13 | |
| pabelanger | clarkb: corvus: any preference on yaml anchors to help reduce duplication more in a nodepool.yaml file? | 17:13 |
| corvus | pabelanger: for the provider-diskimages section? | 17:14 |
| clarkb | pabelanger: for the builders I'm not sure it will help much except for providers: diskimages: right? and in the launchers we have the flavor problem with pools | 17:14 |
| pabelanger | corvus: yes | 17:14 |
| pabelanger | I don't have a preference myself, but figure if we wanted to try them, I'd ask | 17:15 |
| corvus | wow do we set config-drive:true on everything? | 17:15 |
| *** vsaienk0 has joined #openstack-infra | 17:15 | |
| clarkb | corvus: ya because its not an openstack default (but we could/maybe should make it a nodepool default) | 17:15 |
| pabelanger | yah, I think we talked about making that true by default before | 17:15 |
| corvus | a de-facto de-fault | 17:16 |
| corvus | pabelanger: i'm okay with anchors | 17:16 |
| corvus | i don't have a strong feeling about it; i'll go with the flow. | 17:16 |
| AJaeger | clarkb, mordred release notes are fixed - siblings might not yet. mordred ? | 17:16 |
| clarkb | the downside to it is you don't get live mgiration on nodes with config drives | 17:16 |
| clarkb | which for most uses of nodepool is probably fine | 17:17 |
| openstackgerrit | Merged openstack-infra/project-config master: turn off python 3.5 jobs for goal-tools repo https://review.openstack.org/535405 | 17:17 |
| clarkb | pabelanger: do you know what the status of ansible linting on our zuul configs is? this is last job related item on zuulv3-issues etherpad. YOu had a change that you abandoned because you thought it was done somewhere else? | 17:18 |
| *** kjackal has joined #openstack-infra | 17:18 | |
| pabelanger | clarkb: yah, i believe we have jobs running. They leverage tox, but we talk about make fixing that in the future | 17:19 |
| pabelanger | but believe everything is working properly last I checked | 17:19 |
| openstackgerrit | Merged openstack-infra/project-config master: Migrate nodepool.yaml to new syntax https://review.openstack.org/535382 | 17:20 |
| *** felipemonteiro has joined #openstack-infra | 17:21 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Switch to tempest-plugin-jobs for ec2api-tempest-plugin https://review.openstack.org/532835 | 17:23 |
| mordred | AJaeger: I'm not aware of any tox-siblings issues remaining | 17:23 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Only fail requests if no cloud can service them https://review.openstack.org/533372 | 17:24 |
| AJaeger | mordred: good. No open changes? Let me check... | 17:24 |
| AJaeger | mordred: what about https://review.openstack.org/#/c/514058/ ? | 17:24 |
| mordred | AJaeger: I should abandon that | 17:25 |
| AJaeger | mordred: and https://review.openstack.org/#/c/513933/ ? Do we need those | 17:25 |
| *** vsaienk0 has quit IRC | 17:25 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Minimize nodepool-builder configuration https://review.openstack.org/535416 | 17:25 |
| openstackgerrit | Merged openstack-infra/project-config master: set up governance repo for storyboard https://review.openstack.org/535393 | 17:26 |
| AJaeger | mordred: see also https://review.openstack.org/#/c/516076/ - can we revert that install-sibling: false? | 17:26 |
| pabelanger | corvus: clarkb: ^how does that look | 17:26 |
| *** sree has quit IRC | 17:26 | |
| corvus | pabelanger: i love any patch that's +12 -699 | 17:26 |
| pabelanger | :) | 17:27 |
| clarkb | pabelanger: you might want to define the anchor outside of a provider so taht we don't screw up a provider removal and break the anchor? | 17:27 |
| clarkb | otherwise lgtm | 17:27 |
| *** kjackal has quit IRC | 17:27 | |
| pabelanger | good point | 17:28 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Switch to tempest-plugin-jobs for ec2api-tempest-plugin https://review.openstack.org/532835 | 17:28 |
| *** hashar has joined #openstack-infra | 17:31 | |
| *** hashar is now known as hasharDinner | 17:31 | |
| *** sshnaidm|mtg has joined #openstack-infra | 17:31 | |
| *** sshnaidm|mtg is now known as sshnaidm | 17:32 | |
| clarkb | pabelanger: I +2'd it but didn't approve in case you wanted to make that update. I'll let you decide. | 17:32 |
| *** gyee has joined #openstack-infra | 17:32 | |
| *** rfolco is now known as rfolco|rover|lun | 17:32 | |
| *** kjackal has joined #openstack-infra | 17:32 | |
| *** rfolco|rover|lun is now known as rfolco|rover|afk | 17:32 | |
| *** lathiat has joined #openstack-infra | 17:32 | |
| *** vsaienk0 has joined #openstack-infra | 17:34 | |
| pabelanger | clarkb: yah, I am just trying to under syntax for that way. Examples I see use <<: *provider_diskaimges. Trying to see why <<: is needed | 17:34 |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Add test_launcher test https://review.openstack.org/533771 | 17:34 |
| pabelanger | https://learnxinyminutes.com/docs/yaml/ | 17:34 |
| TheJulia | fyi, ssh changed identity warning spotted: http://logs.openstack.org/91/535391/1/check/openstack-tox-cover/0393f2a/job-output.txt.gz#_2018-01-18_16_31_30_432282 | 17:36 |
| mordred | AJaeger: we should be able to remove it - lemme poke a few seconds | 17:36 |
| pabelanger | clarkb: actually, I am not sure if voluptuous will let me move that outside of current location. get extra keys not allowed exception | 17:36 |
| clarkb | pabelanger: ha ok, in that case maybe we use it as is for now but think about relaxing the schema checks for this reason? | 17:37 |
| pabelanger | clarkb: sure, might be a reason why | 17:37 |
| *** jpich has quit IRC | 17:38 | |
| *** sambetts is now known as sambetts|afk | 17:39 | |
| pabelanger | clarkb: left comment about issue, with +3 now | 17:39 |
| *** cshastri_ has quit IRC | 17:40 | |
| *** yamahata has joined #openstack-infra | 17:40 | |
| *** esberglu has quit IRC | 17:41 | |
| *** shardy has quit IRC | 17:42 | |
| *** oidgar has quit IRC | 17:42 | |
| *** caphrim007 has joined #openstack-infra | 17:43 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/nodepool feature/zuulv3: WIP: Add provider info command https://review.openstack.org/535423 | 17:44 |
| *** rfolco|rover|afk is now known as rfolco|rover | 17:44 | |
| openstackgerrit | Merged openstack-infra/project-config master: Minimize nodepool-builder configuration https://review.openstack.org/535416 | 17:46 |
| *** Swami has joined #openstack-infra | 17:46 | |
| *** kjackal has quit IRC | 17:47 | |
| *** SumitNaiksatam has joined #openstack-infra | 17:47 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Remove diskimages section from nodepool-launchers https://review.openstack.org/535424 | 17:48 |
| *** yamamoto has joined #openstack-infra | 17:50 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/system-config master: Remove feature/zuulv3 references https://review.openstack.org/535426 | 17:52 |
| *** kjackal has joined #openstack-infra | 17:53 | |
| pabelanger | clarkb: unsure if we want to bring nb01.o.o online now, or just wait until we merge master and land ^ changes | 17:53 |
| *** owalsh_ has joined #openstack-infra | 17:53 | |
| *** owalsh has quit IRC | 17:55 | |
| *** esberglu has joined #openstack-infra | 17:56 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/system-config master: Update for feature/zuulv3 merging into master https://review.openstack.org/535429 | 17:56 |
| *** rfolco|rover is now known as rfolco|rover|brb | 17:56 | |
| clarkb | pabelanger: I think we can probably bring it up. I imagine we won't delete feature/zuulv3 until after we repoint things to master | 17:57 |
| clarkb | so should be fine | 17:57 |
| *** jamesmcarthur has quit IRC | 17:57 | |
| *** dsariel has quit IRC | 17:58 | |
| *** dtantsur is now known as dtantsur|afk | 17:58 | |
| *** jamesmcarthur has joined #openstack-infra | 17:58 | |
| pabelanger | clarkb: okay, will wait for changes to land and start service | 17:58 |
| pabelanger | clarkb: https://review.openstack.org/535337/ something we should land too for puppet-nodepool | 17:58 |
| pabelanger | systemd related changes | 17:59 |
| *** tesseract has quit IRC | 18:00 | |
| *** yamamoto has quit IRC | 18:00 | |
| *** derekh has quit IRC | 18:00 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/infra-manual master: Remove feature/zuulv3 references https://review.openstack.org/535430 | 18:02 |
| *** iyamahat has joined #openstack-infra | 18:02 | |
| *** efoley has quit IRC | 18:03 | |
| *** armaan has quit IRC | 18:04 | |
| *** armaan has joined #openstack-infra | 18:04 | |
| *** kjackal has quit IRC | 18:04 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-base-jobs master: Remove feature/zuulv3 from pip depend https://review.openstack.org/535432 | 18:05 |
| *** ralonsoh has quit IRC | 18:05 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/openstack-zuul-jobs master: Remove branch specifier from tox-py35-on-zuul job https://review.openstack.org/535433 | 18:06 |
| AJaeger | fungi, could you put reviewing of infra-manual on your review list, please? | 18:07 |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Remove feature/zuulv3 branch references https://review.openstack.org/535434 | 18:08 |
| pabelanger | okay, nb01.o.o is updated, starting service | 18:08 |
| pabelanger | look to be building fedora-27 DIB now | 18:11 |
| *** snapiri- has quit IRC | 18:12 | |
| openstackgerrit | James E. Blair proposed openstack-infra/nodepool master: Replace master with feature/zuulv3 https://review.openstack.org/535435 | 18:14 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Replace master with feature/zuulv3 https://review.openstack.org/535436 | 18:17 |
| openstackgerrit | Merged openstack-infra/zuul feature/zuulv3: Stop running tox-cover job https://review.openstack.org/534458 | 18:18 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Replace master with feature/zuulv3 https://review.openstack.org/535436 | 18:19 |
| fungi | AJaeger: putting it on my review list likely won't help much (since it's already on there, along with a couple hundred other repos i rarely get a chance to check on) but i'll try to remember to up my priority for it | 18:26 |
| AJaeger | thanks for trying, fungi | 18:27 |
| fungi | :( | 18:27 |
| *** nikhil has quit IRC | 18:31 | |
| *** jamesmcarthur has quit IRC | 18:33 | |
| *** armaan has quit IRC | 18:34 | |
| *** jamesmcarthur has joined #openstack-infra | 18:35 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Replace master with feature/zuulv3 https://review.openstack.org/535436 | 18:38 |
| *** gyee has quit IRC | 18:40 | |
| fungi | is there a list of remaining blockers and enablers like ^ for the branch merge? | 18:42 |
| * fungi suddenly remembers today is mergeday | 18:42 | |
| clarkb | fungi: https://etherpad.openstack.org/p/3mSln6ATMP is the etherpad | 18:42 |
| fungi | thanks, i've clearly been pulled in more directions than usual today | 18:43 |
| dmsimard | infra-root: I sent this to add sql reporter to some pipelines where it wasn't enabled: https://review.openstack.org/#/c/534877/ | 18:44 |
| *** jpena is now known as jpena|off | 18:46 | |
| mordred | dmsimard: lgtm. | 18:46 |
| dmsimard | mordred: I'm not 100% sure of the syntax (yaml empty dict/lists confuses me) but zuul doesn't seem to be complaining | 18:47 |
| dmsimard | and I pattern'd it off of the other pipelines | 18:48 |
| dmsimard | ¯\_(ツ)_/¯ | 18:48 |
| corvus | dmsimard: that's actuall a yaml null, which is fine | 18:48 |
| corvus | probably could take an empty dict which might be more readable. but consistency is best. | 18:49 |
| dmsimard | corvus: yeah, but it seems to expect a list (I tried with {} at first and zuul complained that it should be a list) | 18:49 |
| dmsimard | so list or null ? dunno | 18:49 |
| corvus | interesting... that merits a closer look. later. changes is fine for now. :) | 18:49 |
| dmsimard | corvus: maybe it's one of those list-dicts things.. like .projects and ._projects | 18:50 |
| corvus | oh that reminds me, that's a v3.0 blocker we should track | 18:50 |
| dmsimard | that's not finished yet ? I lost track | 18:51 |
| corvus | ianw: ^ what is the status of _projects? | 18:51 |
| dmsimard | oh, iirc we had to revert because inbetween two reverts more people had started using the (deprecated) syntax and then when we wanted to look, hound wasn't indexing new repos and down the rabbithole we went | 18:52 |
| corvus | dmsimard, ianw: it looks like we're close. it looks like projects and _projects are both dicts, so we're at the final stage where we just need to remove the underscore. | 18:55 |
| corvus | filed https://storyboard.openstack.org/#!/story/2001487 | 18:55 |
| openstackgerrit | Merged openstack-infra/project-config master: Add SQL reporter to Zuul experiemental and tripleo pipelines https://review.openstack.org/534877 | 18:56 |
| *** gema has joined #openstack-infra | 18:59 | |
| *** hjensas|afk is now known as hjensas | 19:00 | |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul feature/zuulv3: Do not call merger:cat when all config items are excluded https://review.openstack.org/535450 | 19:02 |
| dmsimard | I guess patches should no longer be going to feature/zuulv3 | 19:07 |
| dmsimard | how does that work ? do we put an ACL to prevent people from posting new changes to feature/zuulv3 ? | 19:07 |
| *** nikhil has joined #openstack-infra | 19:08 | |
| clarkb | dmsimard: we haven't, we'll merge to master then abanson everythign on feature/zuulv3 before deleting the branch | 19:08 |
| clarkb | and in the mean time are trying to advertise we are frozen and in the process of transitioning | 19:08 |
| dmsimard | clarkb: you can delete a branch even though there are changes on it ? | 19:09 |
| dmsimard | I mean, in gerrit | 19:09 |
| clarkb | no, you have to abandon all of the changes | 19:09 |
| pabelanger | then git-review will just complain | 19:10 |
| dmsimard | hmm, and what happens if you do a restore on an abandoned patch ? Just curious how gerrit works from that perspective | 19:10 |
| clarkb | dmsimard: I think its an error | 19:10 |
| AJaeger | if anybody wants to review a few config changes, please consider https://review.openstack.org/532835 https://review.openstack.org/531058 https://review.openstack.org/534878 https://review.openstack.org/531608 https://review.openstack.org/531604 https://review.openstack.org/521697 | 19:10 |
| dmsimard | okay | 19:10 |
| AJaeger | clarkb: I remember somebody tried and it even worked - when we retired stable branches | 19:11 |
| *** iyamahat has quit IRC | 19:11 | |
| AJaeger | clarkb: so there might be a bug | 19:11 |
| clarkb | interesting | 19:11 |
| *** yamahata has quit IRC | 19:11 | |
| *** vsaienk0 has quit IRC | 19:12 | |
| *** amoralej is now known as amoralej|off | 19:12 | |
| clarkb | changes are distinct from the branches they are proposed to in git at least | 19:12 |
| clarkb | so it may be happy to do that because it doesn't break there but I imagine once you try to merge something it would break? | 19:12 |
| fungi | odds are we would also have trouble running ci jobs | 19:14 |
| fungi | on such a change | 19:14 |
| openstackgerrit | Merged openstack-infra/puppet-nodepool master: Support systemdctl daemon-reload for nodepool services https://review.openstack.org/535337 | 19:14 |
| openstackgerrit | Merged openstack-infra/infra-manual master: Remove feature/zuulv3 references https://review.openstack.org/535430 | 19:15 |
| dmsimard | clarkb: maybe it's just a ref from a gerrit perspective, not necessarily a branch | 19:15 |
| clarkb | dmsimard: ya thats what chagnes are in git | 19:15 |
| clarkb | and so unless they explicitly check the target branch is open when restoring it probably just works (tm) | 19:16 |
| *** dprince has quit IRC | 19:18 | |
| *** jbadiapa has joined #openstack-infra | 19:18 | |
| AJaeger | mordred: is the jeepyb change still on your radar? Would love to see it - https://review.openstack.org/531898 | 19:20 |
| *** olaph1 has joined #openstack-infra | 19:22 | |
| *** olaph has quit IRC | 19:23 | |
| *** iyamahat has joined #openstack-infra | 19:29 | |
| clarkb | as a heads up I've approved the IRC bot spec. Looks like we reached agreement on it sodon't need another round of reviews | 19:31 |
| *** dave-mccowan has quit IRC | 19:32 | |
| *** caphrim007 has quit IRC | 19:34 | |
| *** caphrim007 has joined #openstack-infra | 19:35 | |
| openstackgerrit | Merged openstack-infra/infra-specs master: Add IRC bot improvement spec https://review.openstack.org/531242 | 19:37 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/project-config master: Only run one requirements propose post job at a time https://review.openstack.org/535462 | 19:42 |
| AJaeger | fungi, thanks for reviewing! | 19:43 |
| openstackgerrit | Merged openstack-infra/infra-manual master: zuul v3: Expand section on what *to* convert https://review.openstack.org/524566 | 19:44 |
| openstackgerrit | Merged openstack-infra/infra-manual master: Remove version and release https://review.openstack.org/524567 | 19:44 |
| *** yamahata has joined #openstack-infra | 19:48 | |
| openstackgerrit | Merged openstack-infra/infra-manual master: trivial: Remove cruft from 'conf.py' https://review.openstack.org/524568 | 19:49 |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Run project-config nodepool test on all nodepool changes https://review.openstack.org/535464 | 19:51 |
| AJaeger | pabelanger: looking at your change https://review.openstack.org/#/c/535424/ triggered ^ | 19:51 |
| openstackgerrit | Merged openstack-infra/infra-manual master: Update Code Review instruction due to Gerrit UI change https://review.openstack.org/520289 | 19:51 |
| AJaeger | pabelanger: let's get that in first... | 19:51 |
| *** gouthamr has quit IRC | 19:52 | |
| *** harlowja has joined #openstack-infra | 19:52 | |
| *** dprince has joined #openstack-infra | 19:54 | |
| *** rfolco|rover|brb is now known as rfolco|rover | 19:55 | |
| *** vsaienk0 has joined #openstack-infra | 19:55 | |
| AJaeger | fungi, I see "Patch Set" on https://review.openstack.org/#/c/535434/ - but not change set - and https://gerrit-documentation.storage.googleapis.com/Documentation/2.14.5.1/intro-quick.html talks about patch sets as well. | 19:58 |
| AJaeger | this is in response to https://review.openstack.org/528375 | 19:58 |
| AJaeger | fungi: ah, now getting your comment hopefully... | 20:00 |
| *** caphrim007_ has joined #openstack-infra | 20:01 | |
| *** caphrim007_ has quit IRC | 20:01 | |
| *** caphrim007_ has joined #openstack-infra | 20:01 | |
| *** slaweq has joined #openstack-infra | 20:03 | |
| fungi | AJaeger: yeah, in gerrit parlance, 535434 is a "change" while 535434,1 or 535434/1 is a "patch set" | 20:03 |
| fungi | and if you're talking about either in the context of git, they're both a "commit" | 20:04 |
| *** caphrim007 has quit IRC | 20:04 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/infra-manual master: Better explain needed changes https://review.openstack.org/528375 | 20:05 |
| AJaeger | fungi, got it - is the above better? ^ | 20:05 |
| fungi | and change 526955 followed by change 535434 pushed into gerrit as a dependent set of git commits is something we've tended to refer to as a "change series" though i don't know whether that's a real gerrit term | 20:05 |
| *** vsaienk0 has quit IRC | 20:05 | |
| fungi | AJaeger: thanks, will rereview! | 20:05 |
| fungi | my main concern was that if you refer to multiple patch sets, they can be sequential patch sets (revisions) within a single change | 20:07 |
| fungi | rather than as distinct (but dependent) changes | 20:07 |
| AJaeger | fungi: understood | 20:07 |
| AJaeger | pabelanger, config-core, we did not run the project-config-nodepool test for changes on nl01 and nl02, please review https://review.openstack.org/#/c/535464/ and https://review.openstack.org/#/c/535424 | 20:09 |
| pabelanger | AJaeger: let me update depends on | 20:12 |
| AJaeger | pabelanger: let's just ask for a second reviewer instead ;) | 20:13 |
| *** dave-mccowan has joined #openstack-infra | 20:13 | |
| AJaeger | pabelanger: we can merge independ, the job run... | 20:13 |
| *** e0ne has joined #openstack-infra | 20:15 | |
| *** ldnunes has quit IRC | 20:15 | |
| *** fultonj has quit IRC | 20:16 | |
| AJaeger | fungi, do you still have some review cycles, please? ^ | 20:17 |
| *** SumitNaiksatam has quit IRC | 20:17 | |
| *** felipemonteiro_ has joined #openstack-infra | 20:18 | |
| openstackgerrit | Merged openstack-infra/infra-manual master: promote IRC channel creation info into its own section https://review.openstack.org/531133 | 20:18 |
| *** felipemonteiro has quit IRC | 20:21 | |
| *** fultonj has joined #openstack-infra | 20:25 | |
| *** rfolco|rover is now known as rfolco|rover|off | 20:27 | |
| *** jamesmcarthur has quit IRC | 20:27 | |
| fungi | AJaeger: pabelanger: yep, both lgtm | 20:28 |
| fungi | thanks for spotting/fixing | 20:28 |
| *** jlvillal has quit IRC | 20:31 | |
| *** jlvillal has joined #openstack-infra | 20:31 | |
| *** dhajare|BRB has joined #openstack-infra | 20:33 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Run project-config nodepool test on all nodepool changes https://review.openstack.org/535464 | 20:38 |
| *** e0ne has quit IRC | 20:39 | |
| AJaeger | cmurphy, frickler, here's a workaround for the requirements race we saw this morning - https://review.openstack.org/535462 - we only run one requirements proposal job at a time | 20:44 |
| AJaeger | config-core, what do you think? ^ | 20:44 |
| *** dsariel has joined #openstack-infra | 20:45 | |
| *** dhajare|BRB has quit IRC | 20:45 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove diskimages section from nodepool-launchers https://review.openstack.org/535424 | 20:45 |
| *** dhajare|BRB has joined #openstack-infra | 20:45 | |
| *** jamesmcarthur has joined #openstack-infra | 20:46 | |
| openstackgerrit | Merged openstack-infra/zuul master: Replace master with feature/zuulv3 https://review.openstack.org/535436 | 20:47 |
| cmurphy | AJaeger: i think it's neat that zuul has a semaphore feature :) | 20:49 |
| AJaeger | cmurphy: it is | 20:49 |
| *** jamesmcarthur has quit IRC | 20:50 | |
| *** camunoz has quit IRC | 20:51 | |
| fungi | s/neat/extremely useful/ | 20:55 |
| fungi | and in fact necessary, since we used to rely on some jobs being unable to run multiple copies concurrently due to only having one static node assigned | 20:56 |
| *** erlon has quit IRC | 20:56 | |
| AJaeger | so, https://review.openstack.org/535462 looks extremly useful? ;) | 20:57 |
| * fungi was just reviewing to that effect, yes | 20:58 | |
| AJaeger | ok | 20:59 |
| *** caphrim007_ has quit IRC | 21:00 | |
| *** rhallisey has quit IRC | 21:04 | |
| *** caphrim007 has joined #openstack-infra | 21:05 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Remove feature/zuulv3 references from README https://review.openstack.org/535481 | 21:06 |
| *** sbra has joined #openstack-infra | 21:06 | |
| *** witek has quit IRC | 21:10 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Replace master with feature/zuulv3 https://review.openstack.org/535435 | 21:10 |
| *** caphrim007 has quit IRC | 21:10 | |
| *** witek has joined #openstack-infra | 21:12 | |
| openstackgerrit | Merged openstack-infra/zuul-base-jobs master: Remove feature/zuulv3 from pip depend https://review.openstack.org/535432 | 21:13 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove branch specifier from tox-py35-on-zuul job https://review.openstack.org/535433 | 21:13 |
| pabelanger | I've stopped nb01.o.o, having issues with /opt space | 21:14 |
| *** pcichy has quit IRC | 21:15 | |
| pabelanger | http://paste.openstack.org/show/646956/ | 21:16 |
| pabelanger | also an issue with nodepool-launchers | 21:16 |
| AJaeger | #success zuul and nodepool feature/zuulv3 branches have merged into master | 21:16 |
| openstackstatus | AJaeger: Added success to Success page | 21:16 |
| pabelanger | think we need to revet https://review.openstack.org//535424/ | 21:16 |
| *** caphrim007 has joined #openstack-infra | 21:16 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Revert "Remove diskimages section from nodepool-launchers" https://review.openstack.org/535483 | 21:17 |
| pabelanger | clarkb: fungi: AJaeger: ^ | 21:17 |
| AJaeger | pabelanger: completely? | 21:17 |
| corvus | sorry :( | 21:17 |
| AJaeger | pabelanger: including the tox.ini change? | 21:18 |
| Shrews | pabelanger: looks like centos-7 was left in the provider diskimages section | 21:18 |
| Shrews | and labels | 21:18 |
| pabelanger | AJaeger: I can propose them again | 21:18 |
| AJaeger | pabelanger: please do | 21:19 |
| pabelanger | Shrews: yah, we still what that, since we launcher nodes. But thought we might be able to remove diskimages section since we don't build images | 21:19 |
| openstackgerrit | Clark Boylan proposed openstack-infra/nodepool master: Fix node data retrieval race in test_failed_provider https://review.openstack.org/535484 | 21:20 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/system-config master: Have statusbot join #zuul https://review.openstack.org/535485 | 21:20 |
| *** caphrim007 has quit IRC | 21:21 | |
| *** caphrim007 has joined #openstack-infra | 21:22 | |
| *** smatzek has quit IRC | 21:23 | |
| *** weshay|rover is now known as weshay|rover|den | 21:24 | |
| *** weshay|rover|den is now known as weshay|dentist | 21:24 | |
| *** caphrim007 has quit IRC | 21:27 | |
| *** slaweq has quit IRC | 21:27 | |
| *** kgiusti has quit IRC | 21:28 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove feature/zuulv3 branch references https://review.openstack.org/535434 | 21:30 |
| *** slaweq has joined #openstack-infra | 21:32 | |
| *** eharney has quit IRC | 21:33 | |
| *** sbra has quit IRC | 21:33 | |
| *** openstackgerrit has quit IRC | 21:33 | |
| *** olaph has joined #openstack-infra | 21:33 | |
| *** slaweq has quit IRC | 21:34 | |
| *** olaph1 has quit IRC | 21:34 | |
| *** sshnaidm is now known as sshnaidm|off | 21:36 | |
| *** tpsilva has quit IRC | 21:36 | |
| *** openstackgerrit has joined #openstack-infra | 21:38 | |
| openstackgerrit | Merged openstack-infra/system-config master: Update for feature/zuulv3 merging into master https://review.openstack.org/535429 | 21:38 |
| openstackgerrit | Merged openstack-infra/project-config master: Revert "Remove diskimages section from nodepool-launchers" https://review.openstack.org/535483 | 21:44 |
| fungi | pabelanger: ^ | 21:44 |
| *** hasharDinner has quit IRC | 21:45 | |
| *** agopi has quit IRC | 21:45 | |
| *** agopi has joined #openstack-infra | 21:45 | |
| *** caphrim007 has joined #openstack-infra | 21:48 | |
| *** rlandy is now known as rlandy|brb | 21:48 | |
| *** dave-mccowan has quit IRC | 21:49 | |
| *** caphrim007 has quit IRC | 21:49 | |
| *** mattoliverau has quit IRC | 21:52 | |
| *** ihrachys has quit IRC | 21:54 | |
| *** hamzy_ has quit IRC | 21:54 | |
| *** linkmark has quit IRC | 21:55 | |
| *** threestrands has joined #openstack-infra | 21:56 | |
| *** threestrands has quit IRC | 21:56 | |
| *** threestrands has joined #openstack-infra | 21:56 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Updated endpoint get attendee by id https://review.openstack.org/535494 | 21:56 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Updated endpoint get attendee by id https://review.openstack.org/535494 | 21:58 |
| *** dhajare|BRB has quit IRC | 21:59 | |
| *** nikhil has quit IRC | 22:01 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/project-config master: Build and publish releasenotes for openstacksdk https://review.openstack.org/535497 | 22:02 |
| corvus | #status log deleted nodepool and zuul feature/zuulv3 branches | 22:05 |
| openstackstatus | corvus: finished logging | 22:05 |
| *** rcernin has joined #openstack-infra | 22:06 | |
| *** ihrachys has joined #openstack-infra | 22:09 | |
| *** rlandy|brb is now known as rlandy | 22:09 | |
| *** hongbin has quit IRC | 22:12 | |
| *** Goneri has quit IRC | 22:14 | |
| *** hongbin has joined #openstack-infra | 22:14 | |
| *** makowals has joined #openstack-infra | 22:15 | |
| openstackgerrit | Merged openstack-infra/nodepool master: Fix node data retrieval race in test_failed_provider https://review.openstack.org/535484 | 22:18 |
| *** edmondsw has quit IRC | 22:24 | |
| *** smatzek has joined #openstack-infra | 22:25 | |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Handle secrets in branches https://review.openstack.org/535501 | 22:25 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Handle nodesets in branches https://review.openstack.org/535502 | 22:25 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Normalize semaphore branch handling https://review.openstack.org/535503 | 22:25 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Added new endpoint delete attendee https://review.openstack.org/535504 | 22:26 |
| *** smatzek has quit IRC | 22:27 | |
| *** smatzek has joined #openstack-infra | 22:27 | |
| *** kjackal has joined #openstack-infra | 22:28 | |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Added new endpoint delete attendee https://review.openstack.org/535504 | 22:29 |
| pots | anyone know what the sshd.idleTimeout setting is for Gerrit? I seem to be locked out because my CI was leaking ssh connections and now that I've fixed it, I can't get back in. | 22:33 |
| pots | ssh is failing with "Too many concurrent connections (64) - max. allowed: 64" | 22:37 |
| *** smatzek has quit IRC | 22:38 | |
| *** smatzek has joined #openstack-infra | 22:38 | |
| *** smatzek has quit IRC | 22:39 | |
| clarkb | pots: this is for review.openstack.org? | 22:39 |
| pots | yes | 22:39 |
| clarkb | we don't appear to set a value for that | 22:41 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: Do not call merger:cat when all config items are excluded https://review.openstack.org/535509 | 22:43 |
| pots | i'm not sure why they would've survived me killing the processes on this end. is there some sort of penalty box meant to stop DOS attacks? | 22:43 |
| fungi | pots: odds are something dropped the sockets at the client end without following up with a tcp close and as there is no dead peer detection the server thinks those sessions are still open | 22:44 |
| clarkb | pots: yes iirc but higher than that I thought. | 22:44 |
| clarkb | limit is 100 or a 1k connections? we were generous because NAT iirc | 22:45 |
| openstackgerrit | Fabien Boucher proposed openstack-infra/zuul master: Make Zuul able to start with a broken config https://review.openstack.org/535511 | 22:45 |
| pots | fungi clarkb: any chance it gets restarted on a schedule? | 22:45 |
| clarkb | it being gerrit? | 22:46 |
| fungi | and yeah, the per-ip-address blocking we do to avoid dos situations happens once you reach 100 concurrent connections and isn't on right now because we've rebooted for a new kernel and https://review.openstack.org/529712 has yet to get approvd | 22:46 |
| pots | yes. it's telling me "max. allowed: 64" which is the default for Gerrit. | 22:46 |
| fungi | we should be able to close them via the admin api command | 22:47 |
| *** dprince has quit IRC | 22:47 | |
| fungi | what's your gerrit account id number? it'll be easiest to filter by that | 22:47 |
| fungi | should be able to see it on the main preferences page in the gerrit webui | 22:48 |
| fungi | when logged in | 22:48 |
| pots | the accounts that caused the problem are hpmsa-ci and lenovo-ci, not sure how to find the number | 22:48 |
| pots | oh ok, one sec | 22:48 |
| fungi | i can dig them up, just a sec | 22:48 |
| *** esberglu has quit IRC | 22:49 | |
| fungi | 24814 and 24815 respectivly | 22:50 |
| fungi | checking the connections list next | 22:50 |
| pots | yep | 22:51 |
| fungi | yep, gerrit thinks there are 64 active connections for each of those | 22:51 |
| clarkb | I wonder if the gerrit upgrade changed the gerrit behavior and put that limit in place | 22:51 |
| fungi | i'll tell it to close them | 22:51 |
| pots | thanks! | 22:51 |
| clarkb | oh no I remember stackalytics ran into this | 22:51 |
| clarkb | its an account vs IP thing | 22:51 |
| pots | in my case i am using a modified version of jgriffith's sos-ci and it doesn't close ssh connections in a timely fashion, instead relying on GC which didn't seem to be happening. | 22:52 |
| clarkb | pots: does it use paramiko? | 22:52 |
| pots | yes | 22:52 |
| clarkb | if so then ya GC doesn't work with paramiko anymor eyou have to explicitly close connections | 22:52 |
| pots | i will let jgriffith know | 22:53 |
| openstackgerrit | James E. Blair proposed openstack-infra/zuul master: Normalize semaphore branch handling https://review.openstack.org/535503 | 22:53 |
| fungi | pots: jgriffith: i'm closing all 128 of those now, will let you know once it completes | 22:54 |
| pots | fungi: great, thanks so much | 22:54 |
| *** wolverineav has quit IRC | 22:55 | |
| *** wolverineav has joined #openstack-infra | 22:56 | |
| *** dsariel has quit IRC | 22:57 | |
| *** smatzek has joined #openstack-infra | 22:57 | |
| pots | fungi: maybe the sshd.idleTimeout parameter could be set to a conservative value (3 hours?) or would that break things over a slow holiday weekend? | 22:58 |
| *** Swimingly has quit IRC | 22:58 | |
| fungi | pots: jgriffith: connections have been closed now | 22:59 |
| pots | fungi: thanks again! | 23:00 |
| *** ameliac has joined #openstack-infra | 23:00 | |
| *** dhajare|BRB has joined #openstack-infra | 23:00 | |
| *** smatzek has quit IRC | 23:00 | |
| fungi | pots: checking gerrit 2.13 docs for sshd.idleTimeout now | 23:01 |
| *** wolverineav has quit IRC | 23:01 | |
| clarkb | pots: ya that could potentially make zuul lose events? | 23:03 |
| fungi | https://review.openstack.org/Documentation/config-gerrit.html#sshd.idleTimeout | 23:04 |
| *** jamesmcarthur has joined #openstack-infra | 23:04 | |
| fungi | i don't think we'd lose events unless they came across less frequently than the idle timeout | 23:04 |
| *** felipemonteiro_ has quit IRC | 23:05 | |
| fungi | oh, huh | 23:05 |
| fungi | so gerrit no longer uses minasshd? | 23:05 |
| fungi | sshd.backend now defaults to NIO2 | 23:06 |
| clarkb | fungi: we could lose events if we timed out, and before zuul reconnected again an event was emitted | 23:06 |
| fungi | right | 23:06 |
| corvus | setting it to 30m or 1h seems quite generous. | 23:06 |
| corvus | and probably sufficient. | 23:06 |
| pots | clarkb fungi: you might want to make it longer than the longest CI job is expected to take | 23:07 |
| corvus | i'm assuming activity from gerrit counts? | 23:07 |
| fungi | tcp keepalive which was on by default with sshd.backend=MINA is ineffective with NIO2 according to the docs | 23:07 |
| clarkb | corvus: the docs don't really say | 23:08 |
| *** lbragstad has quit IRC | 23:08 | |
| clarkb | it doesn't define what idle is | 23:08 |
| *** Swimingly has joined #openstack-infra | 23:09 | |
| *** Swimingly has quit IRC | 23:09 | |
| *** Swimingly has joined #openstack-infra | 23:09 | |
| fungi | anyway, i expect that if zuul did ssh keepalives (not just tcp keepalives) then the connection wouldn't idle out, but yeah the docs don't really define "idle" in this context | 23:09 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: Remove zuul._projects https://review.openstack.org/535518 | 23:10 |
| corvus | clarkb: i'm seeing some suggestion that it's send/receive... trying to find some solid confirmation | 23:12 |
| corvus | also, what's the deal with nio2 and mina? they're looking pretty indistinguishable to an outsider | 23:12 |
| corvus | https://mina.apache.org/sshd-project/apidocs/org/apache/sshd/common/io/nio2/package-summary.html | 23:13 |
| clarkb | nio is the non blocking api of mina looks like | 23:13 |
| clarkb | or no | 23:13 |
| clarkb | nio is java non blocking io api | 23:14 |
| clarkb | mina is written on top of at least v1 | 23:14 |
| clarkb | v2 is in java7 | 23:14 |
| clarkb | ya so mina sits on top of nio I think | 23:14 |
| corvus | gerrit's choice of "NIO2 vs MINA" is confusing to me | 23:15 |
| fungi | confusingly gerritlike | 23:15 |
| openstackgerrit | Merged openstack-infra/zuul master: Remove feature/zuulv3 references from README https://review.openstack.org/535481 | 23:15 |
| corvus | well, at any rate, the first comment in https://issues.apache.org/jira/browse/SSHD-715 makes me think it's worth a try. | 23:15 |
| corvus | those two properties are what gerrit sets when you set idleTimeout | 23:16 |
| corvus | getProperties().put(IDLE_TIMEOUT, String.valueOf(SECONDS.toMillis(idleTimeoutSeconds))); | 23:16 |
| corvus | getProperties().put(NIO2_READ_TIMEOUT, String.valueOf(SECONDS.toMillis(idleTimeoutSeconds))); | 23:16 |
| *** dsariel has joined #openstack-infra | 23:17 | |
| clarkb | "NIO2_READ_TIMEOUT controls any single packet timeout." I'm not sure what that is trying to say but the IDLE_TIMEOUT sentence seems to indicate it is packets received and sent | 23:17 |
| corvus | ya | 23:18 |
| corvus | my guess is the single packet means partial data received over the ssh protocol | 23:18 |
| *** dhajare|BRB has quit IRC | 23:19 | |
| corvus | so, if nothing is sent or received in idleTimeout, it's closed. and also, if it receives half a packet, then nothing for idleTimeout, it's also closed. | 23:19 |
| corvus | (since gerrit sets those to the same value) | 23:19 |
| corvus | normally, i think you'd be able to set nio2_read_timeout much lower | 23:20 |
| clarkb | ya | 23:22 |
| *** edmondsw has joined #openstack-infra | 23:23 | |
| mordred | uhm. ... I'm seeing some weird devstack failures in gate jobs ... | 23:26 |
| mordred | of note, look at this line and the one after it: | 23:27 |
| mordred | http://logs.openstack.org/76/535476/3/check/openstacksdk-functional-devstack/23804a7/job-output.txt.gz#_2018-01-18_23_17_43_770322 | 23:27 |
| *** mattoliverau has joined #openstack-infra | 23:27 | |
| mordred | it seems strange that the apt mirror config there has mirror settings from two different clouds | 23:27 |
| clarkb | mordred: thats because of cloud archive I think | 23:28 |
| clarkb | we must not be setting cloud archive properly | 23:28 |
| *** edmondsw has quit IRC | 23:29 | |
| mordred | clarkb: k. I'll dig in and look... I'm basically seeing that failure on every devstack job currently, which seems bad | 23:29 |
| *** jamesmcarthur has quit IRC | 23:30 | |
| clarkb | that is because that mirror apperas to not be there anymore | 23:30 |
| clarkb | I'll look at the mirror if you look at the job config | 23:31 |
| *** jamesmcarthur has joined #openstack-infra | 23:31 | |
| mordred | clarkb: kk. looking at job config | 23:31 |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: added update attendee endpoint https://review.openstack.org/535521 | 23:32 |
| mordred | clarkb: oh - wel, yah - we have a TODO "remove when configure-mirrors is fixed" | 23:34 |
| mordred | clarkb: and a export NODEPOOL_UCA_MIRROR=http://mirror.dfw.rax.openstack.org/ubuntu-cloud-archive | 23:34 |
| clarkb | well thats exceptionally curious | 23:35 |
| clarkb | oh I'm a derp hold on | 23:35 |
| clarkb | mirror.dfw.openstack.org <- is apparently a valid dns record and it has made me confused | 23:36 |
| mordred | hah | 23:37 |
| mordred | clarkb: ok - so - from what I can see, there is a role in openstack-zuul-jobs called 'mirror-info' that writes out /etc/ci/mirror-info.sh | 23:39 |
| clarkb | but ya I can't get into the host going to try console now | 23:39 |
| mordred | that role seems to correctly write out the file - and we have tests for it and stuff too | 23:40 |
| mordred | so I *think* we just need to potentially not do the devstack workaround | 23:40 |
| * mordred is going to try a patch | 23:40 | |
| clarkb | rackspcae being slow from here, still not logged in even... | 23:43 |
| openstackgerrit | Ian Wienand proposed openstack-infra/zuul master: Remove zuul._projects https://review.openstack.org/535518 | 23:46 |
| clarkb | ok finally confirmed the kernel hung task timeout | 23:47 |
| clarkb | going to hard reboot the server now | 23:47 |
| mordred | clarkb: https://review.openstack.org/535522 Remove hack for ubuntu cloud archive | 23:47 |
| clarkb | server is rebooting | 23:47 |
| mordred | clarkb: also - the hack is in the pre-playbook of the new devstack job, which explains why we aren't seeing a sea of red | 23:49 |
| clarkb | and that will be self testing | 23:49 |
| *** caphrim007 has joined #openstack-infra | 23:49 | |
| corvus | mordred: yep, i rememeber that, and i think it is fixed. | 23:50 |
| clarkb | dfw rax mirror is back up now | 23:52 |
| *** smatzek has joined #openstack-infra | 23:57 | |
| *** smatzek has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
