| *** salv-orlando has quit IRC | 00:01 | |
| *** salv-orlando has joined #openstack-infra | 00:02 | |
| *** aeng has quit IRC | 00:03 | |
| *** pahuang has quit IRC | 00:04 | |
| *** salv-orlando has quit IRC | 00:06 | |
| *** gouthamr has joined #openstack-infra | 00:08 | |
| *** HeOS has quit IRC | 00:08 | |
| *** HeOS has joined #openstack-infra | 00:09 | |
| *** hongbin has quit IRC | 00:10 | |
| *** pahuang has joined #openstack-infra | 00:17 | |
| *** aeng has joined #openstack-infra | 00:20 | |
| *** caphrim007_ has joined #openstack-infra | 00:22 | |
| *** ramishra has joined #openstack-infra | 00:22 | |
| *** caphrim007 has quit IRC | 00:25 | |
| *** caphrim007_ has quit IRC | 00:26 | |
| *** caphrim007 has joined #openstack-infra | 00:26 | |
| EmilienM | how can I specify some bash ENV in a job layout? | 00:28 |
|---|---|---|
| EmilienM | in a job definition I mean | 00:28 |
| EmilienM | ah I could use "vars" | 00:29 |
| *** caphrim007 has quit IRC | 00:31 | |
| *** markvoelker has quit IRC | 00:37 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config master: Don't repropose changes that already exist. https://review.openstack.org/519839 | 00:40 |
| jlvillal | clarkb: ^^^ | 00:40 |
| jlvillal | clarkb: You might find that interesting/useful. Trying to make it so that we don't keep re-proposing requirements changes when they are already there. | 00:41 |
| jlvillal | ianw: You too ^^^ | 00:41 |
| jlvillal | :) | 00:41 |
| ianw | will look, i think we were discussing this yesterday | 00:41 |
| jlvillal | ianw: yeah, that is why I decided to look into it. Hopefully somebody has a more elegant solution :) | 00:42 |
| jlvillal | At the moment the requirements changes come in so fast it is making difficult to merge them as the time getting through the gate can be longer than the next patch set proposal | 00:44 |
| clarkb | jlvillal: ideally we would do it without the rebase so we can avoid rebase conflicts | 00:44 |
| jlvillal | clarkb: Well my idea on the rebase was to get it to the current code base so can check if it really has changed. | 00:44 |
| jlvillal | clarkb: Hopefully someone has a better idea how to do that. | 00:45 |
| clarkb | w can make our changes locally against $branch then pull the existing change and compare diffs | 00:45 |
| clarkb | if diffs match dont push else push | 00:45 |
| jlvillal | clarkb: How do you know what to compare? | 00:45 |
| clarkb | you'd compare the two patch shas | 00:47 |
| clarkb | iirc there if a git command to generate that hash | 00:47 |
| * jlvillal is confused | 00:48 | |
| clarkb | https://git-scm.com/docs/git-patch-id | 00:48 |
| jlvillal | Do you mean compare the hash of the patch contents of each? | 00:48 |
| clarkb | ya | 00:48 |
| jlvillal | Ah, okay. I'm less confused now :) | 00:49 |
| clarkb | then you never have to worry about rebase failures | 00:49 |
| clarkb | this is what gerrit compared before | 00:49 |
| jlvillal | clarkb: Okay. I'll take a look at it some more tomorrow. | 00:49 |
| jlvillal | clarkb: But if you want to improve it tonight, feel free :) | 00:49 |
| *** priteau has joined #openstack-infra | 00:49 | |
| jlvillal | clarkb: One thing is that we don't want the commit message in there. At least the dates/times... | 00:50 |
| clarkb | ya that command operates on patch diffs | 00:50 |
| jlvillal | Okay. I'll look tomorrow. Time to go walk home in the dark. | 00:51 |
| jlvillal | Got to love Oregon winter! :) | 00:51 |
| *** tosky has quit IRC | 00:51 | |
| *** priteau has quit IRC | 00:54 | |
| dhinesh | clarkb: https://jenkins.io/security/advisory/2016-05-11/ , did not seem to work, i added it as JAVA_ARGS to jenkins to permit all plugins | 01:01 |
| *** kiennt26 has joined #openstack-infra | 01:01 | |
| sc` | ask.o.o looks sad. i'd think it was my browser were it not for the 500 page | 01:08 |
| *** rlandy|rover|afk is now known as rlandy|rover | 01:09 | |
| openstackgerrit | Rabi Mishra proposed openstack-infra/project-config master: Add heat-tempest-plugin project https://review.openstack.org/519567 | 01:10 |
| *** sdague has quit IRC | 01:12 | |
| *** cshastri has joined #openstack-infra | 01:13 | |
| *** cuongnv has joined #openstack-infra | 01:18 | |
| *** rhallisey has quit IRC | 01:22 | |
| *** gyee_ has quit IRC | 01:23 | |
| Jeffrey4l | is there any docker mirror service provided in the zuul env? which will reduce the bandwidth. | 01:24 |
| *** erlon has quit IRC | 01:24 | |
| *** liujiong has joined #openstack-infra | 01:26 | |
| inc0 | ^ we want to get rid of tarballs and move to dockerhub images for gates | 01:28 |
| inc0 | should be fine assumig we'll get some proxy | 01:28 |
| *** cuongnv has quit IRC | 01:28 | |
| inc0 | I remember we talked about using apache http proxy available in nodepools | 01:29 |
| pabelanger | we do reverse proxy cache docker hub today | 01:29 |
| pabelanger | http://mirror.dfw.rax.openstack.org:8081/registry-1.docker/ for example | 01:29 |
| inc0 | cool pabelanger | 01:30 |
| pabelanger | look at NODEPOOL_DOCKER_PROXY variable | 01:30 |
| pabelanger | I think, that was it | 01:30 |
| *** kiennt26 has quit IRC | 01:30 | |
| *** cuongnv has joined #openstack-infra | 01:30 | |
| *** dingyichen has joined #openstack-infra | 01:31 | |
| *** kiennt26 has joined #openstack-infra | 01:31 | |
| Jeffrey4l | cool. i will try this. thx. | 01:34 |
| ianw | clarkb: did git-review unit tests work for you? | 01:34 |
| *** pahuang has quit IRC | 01:34 | |
| clarkb | ianw: no there is a separate fix for that iirc | 01:35 |
| ianw | clarkb: ok ... https://review.openstack.org/#/c/519850/ ... hard to blame you if you just don't want to think about it :) | 01:36 |
| *** caphrim007 has joined #openstack-infra | 01:37 | |
| *** ihrachys has quit IRC | 01:37 | |
| *** markvoelker has joined #openstack-infra | 01:37 | |
| *** caphrim007 has quit IRC | 01:42 | |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: DNM: Test for skip list https://review.openstack.org/519878 | 01:46 |
| *** Apoorva_ has quit IRC | 01:46 | |
| *** pahuang has joined #openstack-infra | 01:48 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 01:49 |
| *** caphrim007 has joined #openstack-infra | 01:50 | |
| *** yamahata has quit IRC | 01:50 | |
| openstackgerrit | Ian Wienand proposed openstack-infra/git-review master: Fix output printing with python3 https://review.openstack.org/519850 | 01:54 |
| mriedem | clarkb: i'm trying to figure out what this is http://logs.openstack.org/58/519758/6/check/legacy-tempest-dsvm-neutron-full/8dd27b1/job-output.txt.gz#_2017-11-15_00_33_31_611200 | 01:57 |
| mriedem | does zuulv3 create the source repos on the workspace before stack.sh runs? | 01:58 |
| mriedem | so i can do something like cd to /opt/stack/new/keystone; git checkout newton-eol | 01:58 |
| *** links has joined #openstack-infra | 02:03 | |
| *** salv-orlando has joined #openstack-infra | 02:03 | |
| *** salv-orlando has quit IRC | 02:08 | |
| *** Douhet has quit IRC | 02:11 | |
| pabelanger | clarkb: I think zuul is swapping again | 02:12 |
| pabelanger | http://cacti.openstack.org/cacti/graph_view.php?action=tree&tree_id=1&leaf_id=557&page=2 | 02:12 |
| pabelanger | not sure if a side affect of killing off the playbook task | 02:13 |
| pabelanger | or just, a reconfigure | 02:14 |
| pabelanger | 2017-11-15 02:07:39,050 DEBUG zuul.ConfigLoader: Created layout id 63029a56b0774024961bfbbdf8fb2ba9 | 02:14 |
| pabelanger | is that message currently | 02:14 |
| *** liujiong has quit IRC | 02:14 | |
| *** liujiong has joined #openstack-infra | 02:15 | |
| pabelanger | https://review.openstack.org/519861/ | 02:15 |
| pabelanger | seems to be the change zuul is trying to dynamic reload | 02:15 |
| *** Douhet has joined #openstack-infra | 02:15 | |
| pabelanger | infra-root: I believe we need to consider restarting zuulv3.o.o again, looking at cacti, we appear to have a very slow memory leak | 02:17 |
| pabelanger | and now past 15GB of ram and into swapping | 02:18 |
| pabelanger | http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=63979 | 02:18 |
| pabelanger | I'm going to see if I can dump the queues | 02:25 |
| pabelanger | doesn't look like :( | 02:28 |
| pabelanger | k, I'm going to stop and start zuulv3. Which means we'll lose changes in the pipelines | 02:29 |
| pabelanger | okay, stopped / started | 02:31 |
| pabelanger | going to take aminute for zuul to reload all the configs | 02:31 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config master: Don't repropose changes that already exist. https://review.openstack.org/519839 | 02:32 |
| jlvillal | clarkb ^^^ I think I implemented your suggestion there | 02:32 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config master: Don't repropose requirements changes that already exist. https://review.openstack.org/519839 | 02:33 |
| *** dhinesh has quit IRC | 02:34 | |
| *** aeng has quit IRC | 02:36 | |
| pabelanger | ugh | 02:36 |
| pabelanger | zuul failed to start | 02:36 |
| pabelanger | due to configuration problems | 02:36 |
| *** aeng has joined #openstack-infra | 02:36 | |
| pabelanger | http://paste.openstack.org/show/626314/ | 02:37 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config master: Don't repropose requirements changes that already exist. https://review.openstack.org/519839 | 02:40 |
| *** hrubi_ has quit IRC | 02:40 | |
| *** iyamahat__ has joined #openstack-infra | 02:42 | |
| pabelanger | k, see the issue | 02:42 |
| EmilienM | sounds like zuul doesn't work indeed | 02:42 |
| EmilienM | ok you're on it, good :D | 02:43 |
| pabelanger | will have to force merge fix | 02:43 |
| *** hrubi has joined #openstack-infra | 02:45 | |
| *** iyamahat_ has quit IRC | 02:45 | |
| pabelanger | okay, patch forced merged | 02:47 |
| pabelanger | trying to start zuulv3 again | 02:47 |
| pabelanger | https://review.openstack.org/519949/ FYI | 02:47 |
| pabelanger | okay, started again | 02:49 |
| *** priteau has joined #openstack-infra | 02:50 | |
| *** Sukhdev has quit IRC | 02:51 | |
| pabelanger | https://review.openstack.org/513915/ appears to be the commit that broke zuulv3 configuration | 02:51 |
| *** gouthamr has quit IRC | 02:52 | |
| jlvillal | pabelanger, That commit landed on 21-Oct ??? | 02:52 |
| *** mhayden has quit IRC | 02:52 | |
| *** rosmaita has quit IRC | 02:52 | |
| pabelanger | jlvillal: yup, we must not have restarted zuulv3 since then | 02:53 |
| pabelanger | okay, zuul looks happy again | 02:53 |
| pabelanger | let me send out status message | 02:53 |
| jlvillal | pabelanger, Wow, hard to believe :) Thanks | 02:54 |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config master: Don't re-propose requirements changes that already exist. https://review.openstack.org/519839 | 02:54 |
| clarkb | pabelanger: I'm juggling the kids but I dont see an event from gerrit for the force merge? is there a change? | 02:54 |
| *** priteau has quit IRC | 02:55 | |
| pabelanger | #status notice Due to an unexpected outage with Zuul (1 hour), you'll need to recheck any jobs that were in progress. Sorry for the inconvenience. | 02:55 |
| openstackstatus | pabelanger: sending notice | 02:55 |
| clarkb | oh I see it niw | 02:55 |
| pabelanger | clarkb: 519949 was the force-merge | 02:56 |
| clarkb | tha ks | 02:56 |
| clarkb | I've got to go back to parenting, sorry Im not more helo | 02:56 |
| -openstackstatus- NOTICE: Due to an unexpected outage with Zuul (1 hour), you'll need to recheck any jobs that were in progress. Sorry for the inconvenience. | 02:56 | |
| pabelanger | clarkb: np | 02:57 |
| *** rlandy|rover has quit IRC | 02:57 | |
| openstackstatus | pabelanger: finished sending notice | 02:58 |
| tonyb | ^^^ that explains the zuul home page ;P | 02:59 |
| pabelanger | #status log Had to stop zuul-scheduler due memory issue, zuul pushed over 15GB of ram and started swapping. https://review.openstack.org/513915/ then prevented zuul from starting, which needed us to then land https://review.openstack.org/519949/ | 02:59 |
| openstackstatus | pabelanger: finished logging | 02:59 |
| *** mhayden has joined #openstack-infra | 03:00 | |
| pabelanger | tonyb: yah, a little excitement | 03:02 |
| tonyb | eeek | 03:03 |
| *** salv-orlando has joined #openstack-infra | 03:04 | |
| *** salv-orlando has quit IRC | 03:09 | |
| *** dave-mccowan has quit IRC | 03:09 | |
| prometheanfire | indeed | 03:12 |
| *** gildub has joined #openstack-infra | 03:14 | |
| *** liujiong has quit IRC | 03:14 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-openafs master: Start openafs-client after dkms has been run https://review.openstack.org/519956 | 03:15 |
| *** liujiong has joined #openstack-infra | 03:17 | |
| *** gildub has quit IRC | 03:23 | |
| *** gildub has joined #openstack-infra | 03:25 | |
| openstackgerrit | Rui Chen proposed openstack-infra/nodepool feature/zuulv3: Fix nodepool cmd TypeError when no arguemnts https://review.openstack.org/519582 | 03:30 |
| *** iyamahat_ has joined #openstack-infra | 03:30 | |
| openstackgerrit | Rui Chen proposed openstack-infra/nodepool feature/zuulv3: Apply floating ip for node according to configuration https://review.openstack.org/518875 | 03:31 |
| *** iyamahat__ has quit IRC | 03:34 | |
| *** iyamahat_ has quit IRC | 03:34 | |
| *** rcernin has quit IRC | 03:34 | |
| *** rcernin has joined #openstack-infra | 03:36 | |
| pabelanger | ianw: clarkb: mind a +3 https://review.openstack.org/519956/ that _should_ fix openafs-client on xenial, trusty appears to be happy currently | 03:41 |
| *** felipemonteiro has joined #openstack-infra | 03:41 | |
| *** udesale has joined #openstack-infra | 03:41 | |
| ianw | looking ... | 03:42 |
| *** slaweq has joined #openstack-infra | 03:43 | |
| *** slaweq has quit IRC | 03:48 | |
| pabelanger | ianw: thanks | 03:49 |
| *** liujiong has quit IRC | 03:52 | |
| openstackgerrit | Merged openstack-infra/git-review master: Actually output the warning https://review.openstack.org/443474 | 03:53 |
| *** liujiong has joined #openstack-infra | 03:53 | |
| *** kiennt26 has quit IRC | 03:57 | |
| pabelanger | cool | 03:57 |
| pabelanger | http://logs.openstack.org/d6/d6d643add8569510b685b1663e7afb060b01bf14/periodic/build-wheel-mirror-ubuntu-trusty/c5d2961/job-output.txt.gz was successful | 03:57 |
| *** yamahata has joined #openstack-infra | 03:57 | |
| pabelanger | I've +3 the puppet-openafs change for xenial | 03:57 |
| pabelanger | and will enqueue-ref the periodic job one more time | 03:57 |
| openstackgerrit | Merged openstack-infra/git-review master: show the config value result after fetching it in verbose mode https://review.openstack.org/513543 | 03:58 |
| *** kiennt26 has joined #openstack-infra | 04:01 | |
| *** yamamoto has joined #openstack-infra | 04:04 | |
| *** bhavik has joined #openstack-infra | 04:05 | |
| *** salv-orlando has joined #openstack-infra | 04:05 | |
| *** Apoorva has joined #openstack-infra | 04:08 | |
| *** salv-orlando has quit IRC | 04:11 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config master: Don't re-propose requirements changes that already exist. https://review.openstack.org/519839 | 04:11 |
| *** bhavik has quit IRC | 04:11 | |
| *** ykarel has joined #openstack-infra | 04:13 | |
| openstackgerrit | Merged openstack-infra/puppet-openafs master: Start openafs-client after dkms has been run https://review.openstack.org/519956 | 04:14 |
| ianw | jlvillal: let's merge that tomorrow my time and we can watch it? | 04:16 |
| jlvillal | ianw, Sounds good to me. | 04:16 |
| *** shu-mutou has joined #openstack-infra | 04:16 | |
| ianw | ok, on the todo list :) | 04:16 |
| *** kiennt26 has quit IRC | 04:16 | |
| jlvillal | :) | 04:16 |
| pabelanger | okay, periodic jobs enqueued again | 04:16 |
| *** hongbin has joined #openstack-infra | 04:17 | |
| *** felipemonteiro has quit IRC | 04:19 | |
| *** dbecker has quit IRC | 04:24 | |
| *** psachin has joined #openstack-infra | 04:28 | |
| *** annp has joined #openstack-infra | 04:29 | |
| *** gildub has quit IRC | 04:29 | |
| *** gongysh has joined #openstack-infra | 04:30 | |
| *** andreas_s has joined #openstack-infra | 04:31 | |
| openstackgerrit | Merged openstack-infra/git-review master: Support git 2.15 and newer https://review.openstack.org/517831 | 04:32 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-openafs master: Install openafs-modules-dkms before openafs-client https://review.openstack.org/519964 | 04:32 |
| pabelanger | ianw: slowly getting there ^ | 04:33 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-openafs master: Install openafs-modules-dkms before openafs-client / openafs-krb5 https://review.openstack.org/519964 | 04:35 |
| ianw | heh, i make a poor human puppet parser :) | 04:35 |
| *** andreas_s has quit IRC | 04:35 | |
| pabelanger | the joy of puppet | 04:36 |
| pabelanger | :) | 04:36 |
| *** dbecker has joined #openstack-infra | 04:37 | |
| *** felipemonteiro has joined #openstack-infra | 04:39 | |
| *** namnh has joined #openstack-infra | 04:41 | |
| *** Apoorva_ has joined #openstack-infra | 04:46 | |
| *** Apoorva has quit IRC | 04:49 | |
| *** priteau has joined #openstack-infra | 04:51 | |
| *** hongbin has quit IRC | 04:53 | |
| *** gildub has joined #openstack-infra | 04:55 | |
| *** priteau has quit IRC | 04:55 | |
| *** pgadiya has joined #openstack-infra | 05:00 | |
| *** claudiub|2 has joined #openstack-infra | 05:01 | |
| *** salv-orlando has joined #openstack-infra | 05:07 | |
| openstackgerrit | Felipe Monteiro proposed openstack-infra/project-config master: Add openstack-tox-cover check to Patrole https://review.openstack.org/519966 | 05:10 |
| *** salv-orlando has quit IRC | 05:12 | |
| EmilienM | pabelanger: hey, are you still around? | 05:13 |
| openstackgerrit | Merged openstack-infra/puppet-openafs master: Install openafs-modules-dkms before openafs-client / openafs-krb5 https://review.openstack.org/519964 | 05:13 |
| EmilienM | pabelanger: I'm doing https://review.openstack.org/#/c/514793 and it fails in zuul "error", but I'm having hard time to see what I missed | 05:13 |
| pabelanger | EmilienM: for a little longer | 05:13 |
| EmilienM | it can wait tomorrow | 05:14 |
| pabelanger | periodic-tripleo-ci-centos-7-ovb-fakeha-caserver | 05:15 |
| pabelanger | where did you define that job? | 05:15 |
| EmilienM | pabelanger: I removed all periodic jobs | 05:15 |
| EmilienM | see tripleo-ci-centos-7-containers-multinode | 05:15 |
| EmilienM | why it failed in check? | 05:15 |
| pabelanger | EmilienM: oh, maybe i am looking at old failure | 05:16 |
| EmilienM | see 514793 in zuul | 05:16 |
| pabelanger | let me see if I can get error from executor | 05:18 |
| EmilienM | thx | 05:19 |
| *** aeng has quit IRC | 05:19 | |
| pabelanger | EmilienM: just left comment | 05:19 |
| pabelanger | now looking in executors | 05:19 |
| EmilienM | pabelanger: oh do I need this playbook? ok | 05:20 |
| EmilienM | I thought it pulled from infra repo | 05:20 |
| pabelanger | 2017-11-14 21:24:36,150 DEBUG zuul.AnsibleJob: [build: 4a10809d89814bf3b699cdef2c807021] Sending result: {"error_detail": "Unable to find playbook /var/lib/zuul/builds/4a10809d89814bf3b699cdef2c807021/work/src/git.openstack.org/openstack-infra/tripleo-ci/playbooks/legacy/nodepool-provider/pre.yaml", "result": "ERROR"} | 05:20 |
| EmilienM | ok I see now | 05:20 |
| EmilienM | thx | 05:20 |
| pabelanger | yah, zuul should comment once al jobs run | 05:20 |
| EmilienM | ack | 05:20 |
| EmilienM | thx | 05:21 |
| EmilienM | I'll ammend | 05:21 |
| *** jaosorior has quit IRC | 05:24 | |
| *** janki has joined #openstack-infra | 05:33 | |
| *** janki has quit IRC | 05:40 | |
| *** Apoorva_ has quit IRC | 05:41 | |
| pabelanger | wow, somebody just uploaded a ton of new .zuul.yaml files | 05:58 |
| pabelanger | and were swapping again | 05:59 |
| pabelanger | ugh | 05:59 |
| pabelanger | EmilienM: who is zhongshengping ? | 06:00 |
| EmilienM | pabelanger: puppet-openstack core | 06:00 |
| EmilienM | I never know how to reach him on IRC | 06:00 |
| EmilienM | pabelanger: he uploaded a bunch of patches, see on #puppet-openstack | 06:00 |
| EmilienM | for zuul v3 transition, can't blame him :) | 06:01 |
| pabelanger | yah | 06:01 |
| pabelanger | so, IIRC, jeblair suggested we keep it to about 5 new .zuul.yaml files at a time | 06:01 |
| *** felipemonteiro has quit IRC | 06:01 | |
| pabelanger | pretty sure zuulv3 is out of memory again | 06:01 |
| pabelanger | and patches do line up with before, when we started swapping again | 06:02 |
| pabelanger | I'm going to stop / start zuul again | 06:04 |
| EmilienM | :( | 06:04 |
| Jeffrey4l | if i can contact that guy. what should he do to solve the issue? | 06:04 |
| pabelanger | https://review.openstack.org/519924/ | 06:04 |
| pabelanger | Jeffrey4l: see comments on ^ | 06:05 |
| EmilienM | maybe send him an email | 06:05 |
| pabelanger | basically, we need to limit the new .zuul.yaml files until we address the issue | 06:05 |
| pabelanger | I seem to recall jeblair asking odyssey4me to process 5 or so new patches at a time | 06:05 |
| tonyb | ianw: Any chance you can look at: https://review.openstack.org/#/c/519758 ? | 06:06 |
| Jeffrey4l | pabelanger, so let him push patch slowly? | 06:06 |
| EmilienM | pabelanger: is it fine to copy/paste playbooks/legacy/nodepool-provider/pre.yaml from openstack-zuul-jobs to tripleo-ci repo? | 06:06 |
| pabelanger | Jeffrey4l: right, slowly. it will be a little painful, but will help limit zuul running out of memory | 06:07 |
| pabelanger | EmilienM: maybe? what does the playbook do | 06:07 |
| Jeffrey4l | roger. let me try. | 06:07 |
| EmilienM | pabelanger: https://review.openstack.org/#/c/514793/10/zuul.d/base.yaml | 06:07 |
| EmilienM | it's tripleo-ci-dsvm | 06:07 |
| pabelanger | EmilienM: yah, if you need it, you'll have to copy it | 06:07 |
| EmilienM | so I have 2 options, iiuc: let the job layout in openstack-zuul-jobs | 06:07 |
| EmilienM | ok ack | 06:07 |
| pabelanger | otherwise, remove run command | 06:08 |
| EmilienM | sounds weird to copy/paste but ok | 06:08 |
| pabelanger | well, you should delete it from openstack-zuul-jobs | 06:08 |
| EmilienM | pabelanger: well, if I remove, I'll miss some tasks | 06:08 |
| EmilienM | gotcha | 06:08 |
| *** salv-orlando has joined #openstack-infra | 06:08 | |
| pabelanger | zuulv3 starting up again | 06:08 |
| pabelanger | #status log zuulv3 stopped / starteed again. It appears an influx of commits like: https://review.openstack.org/519924/ is causing zuul to burn memory quickly. | 06:09 |
| openstackstatus | pabelanger: finished logging | 06:09 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 06:10 |
| EmilienM | pabelanger: can you send an email to him? | 06:10 |
| pabelanger | infra-root: ^ another zuulv3 stop / start. Seems related to zuul processing a lot of .zuul.yaml at one time | 06:10 |
| pabelanger | EmilienM: yes, doing it now | 06:10 |
| EmilienM | ok | 06:10 |
| EmilienM | add me, mnaser and mwhahaha in cc | 06:10 |
| *** janki has joined #openstack-infra | 06:12 | |
| mnaser | Ouch | 06:12 |
| EmilienM | mnaser: do you remember how to reach him on IRC? I don't. | 06:12 |
| *** salv-orlando has quit IRC | 06:12 | |
| mnaser | EmilienM: i dont think I’ve seen zhong on irc | 06:13 |
| Jeffrey4l | i am trying to contact him. | 06:13 |
| mnaser | Or if he has one I’m not sure what it is :( | 06:13 |
| Jeffrey4l | almost success ;) wait for a moment. | 06:14 |
| pabelanger | Jeffrey4l: thanks, I've also sent an email | 06:14 |
| Jeffrey4l | np | 06:14 |
| mnaser | is there anything I can do to help in the meantime? | 06:17 |
| Jeffrey4l | contacted with him. and remind him do not push slowly. | 06:17 |
| pabelanger | mnaser: try to limit patches with new .zuul.yaml to a minimal, 5 seems to be the number to start with | 06:17 |
| pabelanger | each time zuul dynamic reconfigures, it required a sum of memory | 06:18 |
| mnaser | pabelanger: alrighty. I assume that you sent zhong an email and we’ll keep that in mind when doing a large number of cross repo changes | 06:18 |
| Jeffrey4l | remind hime push slowly ( typo..) | 06:19 |
| pabelanger | mnaser: I did, if he jumps on IRC, please let him know too | 06:19 |
| mnaser | most other puppet openstack cores are reachable over IRC so I’ll keep an eye out | 06:19 |
| pabelanger | thanks | 06:19 |
| mnaser | Also thanks Jeffrey4l for reaching out | 06:20 |
| Jeffrey4l | np. | 06:20 |
| pabelanger | ++ | 06:20 |
| mnaser | It’s 1:20 am so I’m off, sorry for the trouble. Later folks | 06:20 |
| Jeffrey4l | good night mnaser | 06:21 |
| *** kjackal has joined #openstack-infra | 06:22 | |
| *** daidv has joined #openstack-infra | 06:26 | |
| *** makowals has joined #openstack-infra | 06:27 | |
| *** makowals_ has quit IRC | 06:28 | |
| *** makowals has quit IRC | 06:29 | |
| *** makowals has joined #openstack-infra | 06:30 | |
| *** namnh has quit IRC | 06:33 | |
| *** armax has quit IRC | 06:33 | |
| *** armax has joined #openstack-infra | 06:33 | |
| *** armax has quit IRC | 06:34 | |
| *** armax has joined #openstack-infra | 06:34 | |
| *** armax has quit IRC | 06:34 | |
| *** threestrands has quit IRC | 06:34 | |
| *** armax has joined #openstack-infra | 06:35 | |
| *** armax has quit IRC | 06:35 | |
| *** armax has joined #openstack-infra | 06:36 | |
| *** armax has quit IRC | 06:36 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-openafs master: Revert "Install openafs-modules-dkms before openafs-client / openafs-krb5" https://review.openstack.org/519980 | 06:36 |
| openstackgerrit | Paul Belanger proposed openstack-infra/puppet-openafs master: Restart openafs-client after dkms package is installed https://review.openstack.org/519981 | 06:36 |
| ianw | tonyb: so just going through the logic of the directory not existing ... | 06:38 |
| tonyb | ianw: okay | 06:38 |
| *** salv-orlando has joined #openstack-infra | 06:38 | |
| *** nsingh has joined #openstack-infra | 06:40 | |
| *** jonher has quit IRC | 06:42 | |
| *** tobiash has quit IRC | 06:43 | |
| *** niraj_singh has quit IRC | 06:43 | |
| *** jonher has joined #openstack-infra | 06:44 | |
| *** deepak_ has joined #openstack-infra | 06:44 | |
| *** tobiash has joined #openstack-infra | 06:44 | |
| yamamoto | can this be an infra issue? https://bugs.launchpad.net/networking-midonet/+bug/1728602 | 06:48 |
| openstack | Launchpad bug 1728602 in networking-midonet "centos job failure with epel-release install" [Critical,In progress] - Assigned to YAMAMOTO Takashi (yamamoto) | 06:48 |
| *** deepak_ has quit IRC | 06:50 | |
| *** slaweq has joined #openstack-infra | 06:51 | |
| *** namnh has joined #openstack-infra | 06:51 | |
| openstackgerrit | Merged openstack-infra/nodepool feature/zuulv3: Be consistent with the ZK data model https://review.openstack.org/519706 | 06:52 |
| *** ramishra has quit IRC | 06:53 | |
| *** slaweq has quit IRC | 06:55 | |
| *** ramishra has joined #openstack-infra | 06:56 | |
| *** jaosorior has joined #openstack-infra | 06:57 | |
| ianw | tonyb: i've dropped some comments ... | 07:02 |
| ianw | yamamoto: i know fedora and copr had some big outages today. is it persistent? it might be a transient error | 07:03 |
| *** slaweq has joined #openstack-infra | 07:07 | |
| openstackgerrit | Merged openstack-infra/puppet-openafs master: Revert "Install openafs-modules-dkms before openafs-client / openafs-krb5" https://review.openstack.org/519980 | 07:07 |
| *** andreas_s has joined #openstack-infra | 07:07 | |
| *** dtantsur|afk is now known as dtantsur | 07:11 | |
| openstackgerrit | Merged openstack-infra/puppet-openafs master: Restart openafs-client after dkms package is installed https://review.openstack.org/519981 | 07:12 |
| *** spectr has joined #openstack-infra | 07:13 | |
| *** kiennt26 has joined #openstack-infra | 07:14 | |
| yamamoto | ianw: i'm seeing it occasionally since the last month | 07:14 |
| *** slaweq has quit IRC | 07:15 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Remove wheel-mirror semaphore https://review.openstack.org/519988 | 07:19 |
| *** rcernin has quit IRC | 07:20 | |
| *** gildub has quit IRC | 07:22 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove horizon legacy jobs (django variants, selenium-headless) https://review.openstack.org/519046 | 07:24 |
| *** armaan has joined #openstack-infra | 07:28 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Set strategy: free for build-wheel-mirror jobs https://review.openstack.org/519994 | 07:28 |
| *** cshastri has quit IRC | 07:32 | |
| *** ykarel is now known as ykarel|lunch | 07:33 | |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/openstack-zuul-jobs master: legacy-cross-networking-midonet-python35: Require requirements https://review.openstack.org/519995 | 07:35 |
| *** priteau has joined #openstack-infra | 07:36 | |
| *** cshastri has joined #openstack-infra | 07:38 | |
| *** spectr has quit IRC | 07:38 | |
| *** salv-orlando has quit IRC | 07:39 | |
| *** salv-orlando has joined #openstack-infra | 07:39 | |
| *** yamamoto_ has joined #openstack-infra | 07:41 | |
| ianw | yamamoto: we should have local epel mirrors. but a lot has changed, and it seems there at least they're not getting picked up. i'll put it on my todo list to have a closer look | 07:41 |
| *** salv-orlando has quit IRC | 07:43 | |
| *** yamamoto has quit IRC | 07:44 | |
| *** e0ne has joined #openstack-infra | 07:44 | |
| yamamoto_ | ianw: thank you | 07:44 |
| *** spiffxp has joined #openstack-infra | 07:46 | |
| *** vipul has joined #openstack-infra | 07:47 | |
| *** slaweq has joined #openstack-infra | 07:47 | |
| *** ccamacho has joined #openstack-infra | 07:48 | |
| *** pcaruana has joined #openstack-infra | 07:51 | |
| *** slaweq has quit IRC | 07:52 | |
| tonyb | ianw: Thanks | 07:52 |
| *** adriant has quit IRC | 07:58 | |
| *** Keitaro has quit IRC | 07:58 | |
| *** danpawlik has quit IRC | 07:59 | |
| *** mmedvede has quit IRC | 07:59 | |
| *** mandre has quit IRC | 07:59 | |
| *** smatzek has joined #openstack-infra | 07:59 | |
| *** hemna has quit IRC | 07:59 | |
| *** purp has quit IRC | 07:59 | |
| *** danpawlik has joined #openstack-infra | 07:59 | |
| *** alexchadin has joined #openstack-infra | 08:00 | |
| *** priteau has quit IRC | 08:00 | |
| *** priteau has joined #openstack-infra | 08:01 | |
| *** kazsh has quit IRC | 08:02 | |
| *** smatzek has quit IRC | 08:03 | |
| *** hughhalf has quit IRC | 08:04 | |
| *** hemna has joined #openstack-infra | 08:04 | |
| *** purp has joined #openstack-infra | 08:04 | |
| openstackgerrit | Merged openstack-infra/project-config master: Switch Puppet tarball job to use Zuul role https://review.openstack.org/519490 | 08:04 |
| *** hughhalf has joined #openstack-infra | 08:04 | |
| *** kazsh has joined #openstack-infra | 08:05 | |
| *** Keitaro has joined #openstack-infra | 08:05 | |
| *** mandre has joined #openstack-infra | 08:05 | |
| *** slaweq has joined #openstack-infra | 08:05 | |
| *** priteau has quit IRC | 08:06 | |
| *** mmedvede has joined #openstack-infra | 08:06 | |
| *** inc0 has quit IRC | 08:09 | |
| *** slaweq has quit IRC | 08:10 | |
| *** priteau has joined #openstack-infra | 08:11 | |
| *** gongysh has quit IRC | 08:12 | |
| *** adriant has joined #openstack-infra | 08:12 | |
| *** gongysh has joined #openstack-infra | 08:14 | |
| *** priteau has quit IRC | 08:15 | |
| *** tesseract has joined #openstack-infra | 08:17 | |
| *** ykarel|lunch is now known as ykarel | 08:19 | |
| *** zhurong has joined #openstack-infra | 08:24 | |
| *** alexchadin has quit IRC | 08:27 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: test unbound config and log https://review.openstack.org/520007 | 08:27 |
| *** alexchadin has joined #openstack-infra | 08:27 | |
| *** e0ne has quit IRC | 08:34 | |
| pabelanger | WOOT | 08:35 |
| pabelanger | now I can sleep | 08:35 |
| pabelanger | http://logs.openstack.org/d6/d6d643add8569510b685b1663e7afb060b01bf14/periodic/release-wheel-mirror/632fa14/job-output.txt.gz | 08:35 |
| pabelanger | successfuly build wheels for ubuntu-trusty and ubuntu-xenial and release the volumes | 08:35 |
| pabelanger | infra-root: AJaeger: ^ | 08:35 |
| pabelanger | when I wake up, I'll focus on centos-7 | 08:36 |
| AJaeger | pabelanger: woot! Congrats! | 08:36 |
| ianw | pabelanger: thanks for chasing that one down! | 08:36 |
| ianw | you might as well stay up now and watch australia win their way into the world cup. games starts in 20 minutes | 08:37 |
| pabelanger | Ha | 08:37 |
| AJaeger | ianw: Cheer them on ;) | 08:37 |
| AJaeger | ianw: do you want to review pabelanger's https://review.openstack.org/519988 change for the wheel build? | 08:37 |
| pabelanger | https://review.openstack.org/519988/ and https://review.openstack.org/519994/ are a few optimzations too | 08:37 |
| pabelanger | AJaeger: thanks! | 08:38 |
| pabelanger | now I pass out | 08:38 |
| pabelanger | see you in a few hours | 08:38 |
| *** martinkopec has joined #openstack-infra | 08:39 | |
| *** amoralej|off is now known as amoralej | 08:40 | |
| *** salv-orlando has joined #openstack-infra | 08:40 | |
| *** gongysh has quit IRC | 08:43 | |
| *** jbadiapa_ is now known as jbadiapa | 08:43 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove wheel-mirror semaphore https://review.openstack.org/519988 | 08:43 |
| *** salv-orlando has quit IRC | 08:44 | |
| *** martinkopec has quit IRC | 08:44 | |
| *** martinkopec has joined #openstack-infra | 08:45 | |
| *** dingyichen has quit IRC | 08:45 | |
| openstackgerrit | Merged openstack-infra/project-config master: Set strategy: free for build-wheel-mirror jobs https://review.openstack.org/519994 | 08:46 |
| *** jpena|off is now known as jpena | 08:52 | |
| *** priteau has joined #openstack-infra | 08:55 | |
| *** salv-orlando has joined #openstack-infra | 08:55 | |
| *** hrubi has quit IRC | 08:56 | |
| *** jpich has joined #openstack-infra | 08:57 | |
| *** jascott1 has quit IRC | 09:00 | |
| *** hrubi has joined #openstack-infra | 09:00 | |
| *** jascott1 has joined #openstack-infra | 09:00 | |
| *** florianf has joined #openstack-infra | 09:02 | |
| *** hrubi has quit IRC | 09:02 | |
| *** zhongshengping has joined #openstack-infra | 09:02 | |
| *** priteau has quit IRC | 09:02 | |
| *** jascott1 has quit IRC | 09:05 | |
| *** priteau has joined #openstack-infra | 09:05 | |
| *** efoley has joined #openstack-infra | 09:05 | |
| *** hrubi has joined #openstack-infra | 09:07 | |
| *** shardy has joined #openstack-infra | 09:07 | |
| *** zenpwner has joined #openstack-infra | 09:08 | |
| *** priteau has quit IRC | 09:10 | |
| *** andreas_s has quit IRC | 09:10 | |
| *** andreas_s has joined #openstack-infra | 09:14 | |
| *** gongysh has joined #openstack-infra | 09:16 | |
| *** lucas-afk is now known as lucasagomes | 09:17 | |
| *** alexchadin has quit IRC | 09:20 | |
| *** alexchadin has joined #openstack-infra | 09:21 | |
| *** ralonsoh has joined #openstack-infra | 09:23 | |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Updated Cache policies https://review.openstack.org/518420 | 09:24 |
| *** zhongshengping has quit IRC | 09:28 | |
| *** yamamoto_ has quit IRC | 09:28 | |
| *** andreas_s has quit IRC | 09:29 | |
| *** alexchadin has quit IRC | 09:31 | |
| *** alexchadin has joined #openstack-infra | 09:32 | |
| *** dsariel has joined #openstack-infra | 09:32 | |
| *** derekh has joined #openstack-infra | 09:34 | |
| *** andreas_s has joined #openstack-infra | 09:34 | |
| openstackgerrit | sebastian marcet proposed openstack-infra/openstackid-resources master: Marketplace API https://review.openstack.org/498102 | 09:35 |
| openstackgerrit | Merged openstack-infra/openstackid-resources master: Marketplace API https://review.openstack.org/498102 | 09:37 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Use prepared instackenv.json if exists https://review.openstack.org/520023 | 09:37 |
| *** Hal has joined #openstack-infra | 09:39 | |
| *** e0ne has joined #openstack-infra | 09:42 | |
| *** gildub has joined #openstack-infra | 09:43 | |
| *** hemna_ has quit IRC | 09:43 | |
| *** yamamoto has joined #openstack-infra | 09:46 | |
| *** andreas_s has quit IRC | 09:48 | |
| *** eumel8 has joined #openstack-infra | 09:50 | |
| eumel8 | morning | 09:51 |
| *** andreas_s has joined #openstack-infra | 09:53 | |
| *** Liced has joined #openstack-infra | 09:56 | |
| *** hemna_ has joined #openstack-infra | 09:58 | |
| *** jpena is now known as jpena|mtg | 09:59 | |
| *** liujiong has quit IRC | 10:02 | |
| Liced | AJaeger: Hi. There was few merges yesterday and before for networking-bgpvpn and the folder in zanata is still empty | 10:02 |
| *** eroux has joined #openstack-infra | 10:02 | |
| *** andreas_s has quit IRC | 10:02 | |
| *** strigazi has quit IRC | 10:05 | |
| *** strigazi has joined #openstack-infra | 10:06 | |
| *** efoley has quit IRC | 10:07 | |
| *** efoley_ has joined #openstack-infra | 10:07 | |
| AJaeger | Liced: let's check logs... | 10:08 |
| *** makowals has quit IRC | 10:08 | |
| *** gildub has quit IRC | 10:08 | |
| AJaeger | Liced: still no logs for that one ;(((( | 10:11 |
| Liced | Ajaeger : :( | 10:12 |
| openstackgerrit | Raissa Sarmento proposed openstack-infra/project-config master: Add check-requirements to manila-tempest-plugin https://review.openstack.org/520034 | 10:12 |
| *** annp has quit IRC | 10:13 | |
| AJaeger | infra-root, I'm missing post logs from networking-bgvpn, none of the changes merged yesterday has any post logs. Could you check what's up there, please? | 10:13 |
| *** zenpwner has quit IRC | 10:15 | |
| *** namnh has quit IRC | 10:17 | |
| *** cuongnv has quit IRC | 10:17 | |
| *** efoley_ has quit IRC | 10:17 | |
| *** cuongnv has joined #openstack-infra | 10:18 | |
| AJaeger | infra-root, looking at https://docs.openstack.org/networking-bgpvpn/latest/ , it was last updated October 9th - so I wonder whether we have some bug in project-config/projects.yaml or in zuul that results in no running post jobs at all for the repo? | 10:20 |
| *** mat128 has joined #openstack-infra | 10:20 | |
| *** cuongnv has quit IRC | 10:21 | |
| *** yamamoto has quit IRC | 10:21 | |
| AJaeger | Liced: looking at the import job, there'S some work to do: http://logs.openstack.org/periodic/git.openstack.org/openstack/networking-bgpvpn/master/propose-translation-update/bedf696/job-output.txt.gz#_2017-11-14_07_09_01_872764 | 10:23 |
| *** daidv has quit IRC | 10:23 | |
| *** efoley has joined #openstack-infra | 10:25 | |
| *** kiennt26 has quit IRC | 10:25 | |
| *** tosky has joined #openstack-infra | 10:26 | |
| AJaeger | Liced: that's the only translated repo that needs extra requirements for install ;( | 10:29 |
| *** andreas_s has joined #openstack-infra | 10:31 | |
| *** dizquierdo has joined #openstack-infra | 10:33 | |
| *** andreas_s has quit IRC | 10:35 | |
| *** andreas_s has joined #openstack-infra | 10:36 | |
| *** efoley has quit IRC | 10:38 | |
| Liced | AJaeger: Thank you. I will check this. | 10:38 |
| *** kiennt26 has joined #openstack-infra | 10:41 | |
| *** daidv has joined #openstack-infra | 10:41 | |
| *** raissa_ has quit IRC | 10:42 | |
| *** hrubi has quit IRC | 10:43 | |
| *** andreas_s has quit IRC | 10:45 | |
| *** hrubi has joined #openstack-infra | 10:47 | |
| *** hrubi has quit IRC | 10:49 | |
| *** kiennt26 has quit IRC | 10:51 | |
| *** daidv has quit IRC | 10:52 | |
| *** hrubi has joined #openstack-infra | 10:52 | |
| *** raissa_ has joined #openstack-infra | 10:53 | |
| *** raissa_ is now known as raissa | 10:57 | |
| *** andreas_s has joined #openstack-infra | 11:01 | |
| *** armaan has quit IRC | 11:04 | |
| *** armaan has joined #openstack-infra | 11:04 | |
| *** yee37929 has quit IRC | 11:05 | |
| *** yee379 has joined #openstack-infra | 11:05 | |
| *** rfolco has joined #openstack-infra | 11:08 | |
| *** kiennt26 has joined #openstack-infra | 11:09 | |
| *** daidv has joined #openstack-infra | 11:09 | |
| *** andreas_s has quit IRC | 11:09 | |
| *** shu-mutou is now known as shu-mutou-AWAY | 11:11 | |
| *** kiennt26 has quit IRC | 11:12 | |
| *** kiennt26 has joined #openstack-infra | 11:12 | |
| *** daidv has quit IRC | 11:12 | |
| *** mrunge has quit IRC | 11:15 | |
| *** gongysh has quit IRC | 11:15 | |
| *** wolverineav has joined #openstack-infra | 11:16 | |
| *** kiennt26 has quit IRC | 11:20 | |
| *** yamamoto has joined #openstack-infra | 11:22 | |
| *** cshastri has quit IRC | 11:22 | |
| *** sdague has joined #openstack-infra | 11:23 | |
| *** daidv has joined #openstack-infra | 11:27 | |
| *** yamamoto has quit IRC | 11:29 | |
| *** daidv has quit IRC | 11:31 | |
| *** yamamoto has joined #openstack-infra | 11:31 | |
| *** LindaWang1 has joined #openstack-infra | 11:32 | |
| *** andreas_s has joined #openstack-infra | 11:34 | |
| *** stakeda has quit IRC | 11:37 | |
| *** mrunge has joined #openstack-infra | 11:37 | |
| *** andreas_s has quit IRC | 11:38 | |
| *** andreas_s has joined #openstack-infra | 11:38 | |
| *** efoley has joined #openstack-infra | 11:43 | |
| openstackgerrit | Merged openstack/diskimage-builder master: elements: Respect devpts mount options https://review.openstack.org/518749 | 11:44 |
| *** daidv has joined #openstack-infra | 11:45 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Collect unbound DNS server configs and logs https://review.openstack.org/520007 | 11:59 |
| openstackgerrit | Davanum Srinivas (dims) proposed openstack-infra/project-config master: Remove legacy Castellan jobs https://review.openstack.org/515774 | 12:02 |
| dims | AJaeger : let's get this in please ^ | 12:03 |
| *** yamamoto has quit IRC | 12:03 | |
| *** yamamoto has joined #openstack-infra | 12:04 | |
| tbarron | I see a centos7 issue in multiple jobs wherein nova compute fails with "Nova requires QEMU version 2.1.0 or greater." | 12:04 |
| tbarron | Example: http://logs.openstack.org/47/510547/7/check/manila-tempest-minimal-dsvm-lvm-centos-7/f402868/logs/screen-n-cpu.txt.gz#_Nov_15_07_39_20_615055 | 12:04 |
| dims | AJaeger : the legacy-castellan-dsvm-functional is very flaky and the review for castellan-functional-devstack is stuck . so let's nuke the legacy first please | 12:04 |
| tbarron | Known issue? ^^^ | 12:04 |
| *** coolsvap has joined #openstack-infra | 12:04 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: debug unbound queries https://review.openstack.org/520055 | 12:05 |
| *** efoley has quit IRC | 12:06 | |
| *** efoley has joined #openstack-infra | 12:06 | |
| *** andreas_s has quit IRC | 12:06 | |
| *** slaweq has joined #openstack-infra | 12:06 | |
| *** lucasagomes is now known as lucas-hungry | 12:06 | |
| *** pbourke has quit IRC | 12:07 | |
| *** ianychoi has quit IRC | 12:07 | |
| *** pbourke has joined #openstack-infra | 12:08 | |
| hwoarang | Hello! Any thoughts on this https://review.openstack.org/#/c/505657/ bindep patch? it's been sitting there for quite a while :) | 12:10 |
| *** janki is now known as janki|afk | 12:10 | |
| *** andreas_s has joined #openstack-infra | 12:11 | |
| *** slaweq has quit IRC | 12:11 | |
| AJaeger | dims: +2 | 12:18 |
| dims | thanks AJaeger ! | 12:19 |
| *** andreas_s has quit IRC | 12:20 | |
| *** wolverineav has quit IRC | 12:20 | |
| *** priteau has joined #openstack-infra | 12:21 | |
| *** sree has joined #openstack-infra | 12:23 | |
| *** alexchadin has quit IRC | 12:24 | |
| *** andreas_s has joined #openstack-infra | 12:25 | |
| *** janki|afk is now known as janki | 12:27 | |
| *** gongysh has joined #openstack-infra | 12:33 | |
| *** smatzek has joined #openstack-infra | 12:33 | |
| *** andreas_s has quit IRC | 12:35 | |
| *** armaan has quit IRC | 12:39 | |
| *** rhallisey has joined #openstack-infra | 12:40 | |
| *** andreas_s has joined #openstack-infra | 12:40 | |
| *** rhallisey_ has joined #openstack-infra | 12:40 | |
| *** kiennt26 has joined #openstack-infra | 12:42 | |
| *** efoley has quit IRC | 12:42 | |
| *** inc0 has joined #openstack-infra | 12:42 | |
| *** rhallisey has quit IRC | 12:44 | |
| *** sree has quit IRC | 12:47 | |
| *** sree has joined #openstack-infra | 12:49 | |
| *** andreas_s has quit IRC | 12:49 | |
| *** bobh has joined #openstack-infra | 12:52 | |
| *** jascott1 has joined #openstack-infra | 12:53 | |
| *** andreas_s has joined #openstack-infra | 12:54 | |
| *** wolverineav has joined #openstack-infra | 12:55 | |
| *** efoley has joined #openstack-infra | 12:57 | |
| *** jascott1 has quit IRC | 12:57 | |
| *** kiennt26 has quit IRC | 12:59 | |
| *** kiennt26 has joined #openstack-infra | 13:00 | |
| *** dprince has joined #openstack-infra | 13:02 | |
| *** jpena|mtg is now known as jpena|lunch | 13:03 | |
| *** andreas_s has quit IRC | 13:03 | |
| *** dizquierdo has quit IRC | 13:05 | |
| *** ianychoi has joined #openstack-infra | 13:05 | |
| *** andreas_s has joined #openstack-infra | 13:06 | |
| *** lucas-hungry is now known as lucasagomes | 13:07 | |
| *** gongysh has quit IRC | 13:08 | |
| EmilienM | AJaeger: hey, could I use some help on https://review.openstack.org/#/c/514793/ please - I'm trying to understand why "run" playbook isn't executed on new jobs | 13:08 |
| *** smatzek has quit IRC | 13:09 | |
| *** smatzek has joined #openstack-infra | 13:10 | |
| *** makowals has joined #openstack-infra | 13:11 | |
| *** dprince has quit IRC | 13:12 | |
| *** dprince has joined #openstack-infra | 13:12 | |
| *** trown|outtypewww is now known as trown | 13:14 | |
| *** sree has quit IRC | 13:16 | |
| *** mat128 has quit IRC | 13:17 | |
| *** sree has joined #openstack-infra | 13:17 | |
| openstackgerrit | Cedric Savignan proposed openstack-infra/project-config master: Adds missing required-projects to netwokring-bgppvn https://review.openstack.org/520071 | 13:18 |
| openstackgerrit | Cedric Savignan proposed openstack-infra/project-config master: Adds missing required-projects to networking-bgppvn https://review.openstack.org/520071 | 13:19 |
| *** dizquierdo has joined #openstack-infra | 13:20 | |
| AJaeger | EmilienM: which job? | 13:21 |
| EmilienM | AJaeger: tripleo-ci-centos-7-ovb-ha-oooq for example | 13:22 |
| EmilienM | AJaeger: I tried to refactorize playbooks | 13:22 |
| *** links has quit IRC | 13:22 | |
| *** sree has quit IRC | 13:22 | |
| EmilienM | AJaeger: from https://review.openstack.org/#/c/514792/6/playbooks/legacy/tripleo-ci-centos-7-ovb-convergence-oooq/run.yaml into https://review.openstack.org/#/c/514793/11/playbooks/tripleo-ci/run.yaml | 13:22 |
| openstackgerrit | Cedric Savignan proposed openstack-infra/project-config master: Adds missing required-projects to networking-bgppvn https://review.openstack.org/520071 | 13:23 |
| *** amoralej is now known as amoralej|lunch | 13:23 | |
| *** links has joined #openstack-infra | 13:24 | |
| AJaeger | EmilienM: I really can't help on that level. I just notice that https://review.openstack.org/#/c/514793/11/playbooks/tripleo-ci/run.yaml has hosts: primary - but this is a single node job, so is primary really correct here? | 13:25 |
| *** sree has joined #openstack-infra | 13:25 | |
| EmilienM | AJaeger: yeah, no problem I'll try to look | 13:25 |
| *** edmondsw has joined #openstack-infra | 13:28 | |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy Castellan jobs https://review.openstack.org/515774 | 13:28 |
| *** markvoelker has quit IRC | 13:29 | |
| *** markvoelker has joined #openstack-infra | 13:29 | |
| *** bobh has quit IRC | 13:30 | |
| *** rlandy has joined #openstack-infra | 13:32 | |
| *** kiennt26 has quit IRC | 13:35 | |
| *** tosky has quit IRC | 13:37 | |
| *** armaan has joined #openstack-infra | 13:38 | |
| EmilienM | AJaeger: I'm confused about https://review.openstack.org/#/c/519826/2/zuul.d/layout.yaml | 13:40 |
| EmilienM | AJaeger: I thought the project layout could go in-tree | 13:40 |
| EmilienM | why should they stay on project-config ? | 13:40 |
| *** alexchadin has joined #openstack-infra | 13:43 | |
| EmilienM | but ok I'll do it | 13:43 |
| *** smatzek_ has joined #openstack-infra | 13:45 | |
| *** gouthamr has joined #openstack-infra | 13:45 | |
| *** smatzek has quit IRC | 13:45 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: tripleo-ui: use native nodejs6 jobs and cleanup https://review.openstack.org/519828 | 13:46 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/openstack-zuul-jobs master: Remove tripleo-ui legacy layout https://review.openstack.org/519829 | 13:48 |
| *** dizquierdo has quit IRC | 13:50 | |
| *** tung_doan has joined #openstack-infra | 13:53 | |
| *** links has quit IRC | 13:53 | |
| *** eroux has quit IRC | 13:53 | |
| *** jlibosva has joined #openstack-infra | 13:54 | |
| AJaeger | EmilienM: did you read the link I gave? | 13:55 |
| EmilienM | I did | 13:55 |
| *** andreas_s has quit IRC | 13:55 | |
| EmilienM | AJaeger: I updated ^ | 13:55 |
| AJaeger | We don't want everything moved into projects, some things have to stay in project-config - and some we prefer ;) | 13:55 |
| *** udesale has quit IRC | 13:56 | |
| EmilienM | ok | 13:56 |
| AJaeger | EmilienM: checking quickly | 13:56 |
| EmilienM | AJaeger: I'm looking at the inventory now, It seems like I did something wrong in the refacto | 13:56 |
| *** jpena|lunch is now known as jpena | 13:56 | |
| AJaeger | EmilienM: your project-config chagne is fine - that's the only one I could review now... | 13:57 |
| EmilienM | ack | 13:57 |
| EmilienM | AJaeger: I'm working on https://review.openstack.org/#/c/514793/ | 13:57 |
| *** udesale has joined #openstack-infra | 13:58 | |
| AJaeger | EmilienM: you might want to just do a single job - have that working and then the next one. | 13:58 |
| AJaeger | But your call ;) | 13:58 |
| EmilienM | AJaeger: yeah, I'm refactoring tripleo-ci jobs, to have a base | 13:59 |
| EmilienM | AJaeger: I'm really close I think | 13:59 |
| *** efoley has quit IRC | 13:59 | |
| *** efoley has joined #openstack-infra | 13:59 | |
| *** andreas_s has joined #openstack-infra | 14:00 | |
| *** slaweq has joined #openstack-infra | 14:00 | |
| jlibosva | dhellmann: smcginnis hi, could you please have a look at 11.0.2 release tag for Neutron? It already has +1 from release liaisons and the PTL https://review.openstack.org/#/c/519431/ | 14:01 |
| EmilienM | AJaeger: I would use your help on https://review.openstack.org/#/c/514793/11/playbooks/tripleo-ci/run.yaml - can you see my second question? | 14:01 |
| *** dave-mccowan has joined #openstack-infra | 14:01 | |
| *** eharney has joined #openstack-infra | 14:02 | |
| Zara | hi! any chance we could get some eyes on: https://review.openstack.org/#/c/518626/ ? for context, storyboard has a js draft build that offers a sneak preview of the webclient with any proposed changes, so that reviewers can try it out without spinning up their own local storyboard instance | 14:03 |
| Zara | the job for that currently fails so that patch should get it working again | 14:03 |
| *** kgiusti has joined #openstack-infra | 14:03 | |
| EmilienM | AJaeger: got this error: http://logs.openstack.org/93/514793/11/check/tripleo-ci-centos-7-containers-multinode/1379a2b/job-output.txt.gz#_2017-11-15_06_26_27_907079 | 14:04 |
| *** tung_doan has quit IRC | 14:04 | |
| Zara | so it will make it easier to review webclient patches and should help us get bugs fixed faster | 14:04 |
| *** jaypipes has joined #openstack-infra | 14:04 | |
| *** rfolco is now known as rfolco_brb | 14:07 | |
| *** camunoz has joined #openstack-infra | 14:07 | |
| *** slaweq_ has joined #openstack-infra | 14:08 | |
| *** pgadiya has quit IRC | 14:08 | |
| *** Liced has quit IRC | 14:08 | |
| *** andreas_s has quit IRC | 14:09 | |
| *** Goneri has joined #openstack-infra | 14:09 | |
| AJaeger | jlibosva: better use #openstack-release for release requests | 14:10 |
| jlibosva | AJaeger: thanks, going to ask there | 14:11 |
| AJaeger | EmilienM: sorry, in meetings now... | 14:11 |
| EmilienM | AJaeger: no prob I'm trying to debug myself | 14:12 |
| *** slaweq_ has quit IRC | 14:12 | |
| *** dave-mccowan has quit IRC | 14:13 | |
| *** rfolco_brb is now known as rfolco | 14:13 | |
| *** dizquierdo has joined #openstack-infra | 14:14 | |
| *** andreas_s has joined #openstack-infra | 14:14 | |
| *** makowals has quit IRC | 14:15 | |
| *** andreas_s has quit IRC | 14:15 | |
| *** andreas_s has joined #openstack-infra | 14:15 | |
| *** pblaho has quit IRC | 14:19 | |
| *** krtaylor has joined #openstack-infra | 14:19 | |
| *** ramishra has quit IRC | 14:19 | |
| *** amoralej|lunch is now known as amoralej | 14:20 | |
| *** dave-mccowan has joined #openstack-infra | 14:20 | |
| openstackgerrit | Andreas Jaeger proposed openstack-infra/openstack-zuul-jobs master: Remove legacy Castellan jobs https://review.openstack.org/515776 | 14:21 |
| *** rlandy is now known as rlandy|rover | 14:23 | |
| AJaeger | EmilienM: don't recheck openstack-zuul-jobs changes if they depend on project-config - until the project-config is merged. project-config is special and you will get a -1 for these normally | 14:23 |
| *** ramishra has joined #openstack-infra | 14:23 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 14:24 |
| *** smatzek_ has quit IRC | 14:24 | |
| *** tosky has joined #openstack-infra | 14:24 | |
| EmilienM | AJaeger: ok | 14:24 |
| *** dave-mcc_ has joined #openstack-infra | 14:24 | |
| DuncanT | Hi all. I've been off openstack dev for about a year, but I'm dipping my toe back in. Where's the right place to ask questions about devstack these days? | 14:25 |
| *** dave-mccowan has quit IRC | 14:25 | |
| frickler | DuncanT: most likely #openstack-qa | 14:26 |
| DuncanT | frickler: Thanks | 14:26 |
| *** jlibosva has left #openstack-infra | 14:27 | |
| *** slaweq has quit IRC | 14:28 | |
| *** udesale has quit IRC | 14:29 | |
| *** slaweq has joined #openstack-infra | 14:29 | |
| *** sree has quit IRC | 14:35 | |
| *** sree has joined #openstack-infra | 14:35 | |
| *** mpaolino has joined #openstack-infra | 14:36 | |
| *** armaan has quit IRC | 14:37 | |
| smcginnis | DuncanT! | 14:40 |
| DuncanT | smcginnis: Don't get too excited, I'm poking monasca ;-) | 14:40 |
| smcginnis | DuncanT: Bah! :) | 14:41 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 14:41 |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Document current_user_id in a release note https://review.openstack.org/520093 | 14:43 |
| *** sree has quit IRC | 14:43 | |
| *** slaweq has quit IRC | 14:44 | |
| *** esberglu has quit IRC | 14:44 | |
| *** slaweq has joined #openstack-infra | 14:45 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/shade master: Document current_user_id in a release note https://review.openstack.org/520093 | 14:45 |
| *** udesale has joined #openstack-infra | 14:45 | |
| *** esberglu has joined #openstack-infra | 14:45 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 14:48 |
| *** andreas_s has quit IRC | 14:49 | |
| *** esberglu has quit IRC | 14:49 | |
| *** andreas_s has joined #openstack-infra | 14:50 | |
| *** yee379 has quit IRC | 14:51 | |
| *** yee379 has joined #openstack-infra | 14:51 | |
| *** trown is now known as trown|brb | 14:53 | |
| *** mat128 has joined #openstack-infra | 14:55 | |
| openstackgerrit | Matt Riedemann proposed openstack-infra/devstack-gate master: Checkout from -eol tag if stable branch does not exist https://review.openstack.org/520095 | 14:55 |
| mriedem | ianw: tonyb: ^ | 14:55 |
| *** slaweq has quit IRC | 14:56 | |
| *** trown|brb is now known as trown | 14:57 | |
| *** eumel8 has quit IRC | 14:58 | |
| *** alexchadin has quit IRC | 14:59 | |
| *** coolsvap has quit IRC | 15:00 | |
| *** esberglu has joined #openstack-infra | 15:00 | |
| *** armax has joined #openstack-infra | 15:01 | |
| *** armaan has joined #openstack-infra | 15:01 | |
| *** abhi89 has joined #openstack-infra | 15:02 | |
| edmondsw | I'm seeing several cinder changes that got +W yesterday and say that zuul started gate jobs but zuul never came back and voted | 15:02 |
| *** rhallisey_ is now known as rhallisey | 15:02 | |
| edmondsw | e.g. https://review.openstack.org/#/c/476750/ | 15:02 |
| edmondsw | e.g. https://review.openstack.org/#/c/517725/ | 15:03 |
| *** ihrachys has joined #openstack-infra | 15:03 | |
| *** efoley_ has joined #openstack-infra | 15:03 | |
| *** armaan has quit IRC | 15:03 | |
| edmondsw | should we just reverify, or.. ? | 15:03 |
| *** armaan has joined #openstack-infra | 15:03 | |
| *** efoley has quit IRC | 15:04 | |
| *** ramishra has quit IRC | 15:05 | |
| smcginnis | edmondsw: We can just recheck those. Must have been lost in the restart. | 15:05 |
| *** yamamoto has quit IRC | 15:05 | |
| edmondsw | smcginnis ok will do | 15:05 |
| *** iyamahat has joined #openstack-infra | 15:07 | |
| *** iyamahat_ has joined #openstack-infra | 15:08 | |
| *** abhi89 has quit IRC | 15:10 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 15:10 |
| *** LindaWang1 has quit IRC | 15:11 | |
| *** dhill_ has quit IRC | 15:11 | |
| *** iyamahat has quit IRC | 15:11 | |
| *** links has joined #openstack-infra | 15:11 | |
| *** andreas_s has quit IRC | 15:12 | |
| *** dhill_ has joined #openstack-infra | 15:12 | |
| *** gouthamr has quit IRC | 15:12 | |
| *** janki has quit IRC | 15:13 | |
| *** yamamoto has joined #openstack-infra | 15:16 | |
| *** gouthamr has joined #openstack-infra | 15:16 | |
| *** hongbin has joined #openstack-infra | 15:16 | |
| sshnaidm | pabelanger, re: resolving failures, as I see from logs, unbound just redirects requests from the host to opendns and google dns, is it right? | 15:19 |
| *** rosmaita has joined #openstack-infra | 15:20 | |
| *** yamamoto has quit IRC | 15:21 | |
| openstackgerrit | Pavlo Shchelokovskyy proposed openstack-infra/shade master: Fix the devstack role for base functional job https://review.openstack.org/520105 | 15:22 |
| openstackgerrit | Merged openstack-infra/shade master: Add ability to work in other auth contexts https://review.openstack.org/519394 | 15:25 |
| openstackgerrit | Merged openstack-infra/shade master: Add helper property to get the current user id https://review.openstack.org/519511 | 15:25 |
| openstackgerrit | Merged openstack-infra/shade master: Remove reference to context-managers from release note https://review.openstack.org/519784 | 15:27 |
| fungi | sshnaidm: it should also cache responses locally so that subsequent requests for the same records get served directly out of cache rather than forwarded | 15:28 |
| fungi | sshnaidm: but otherwise, correct (it's set to forward to opendns and googledns) | 15:28 |
| openstackgerrit | Merged openstack-infra/project-config master: tripleo-ui: use native nodejs6 jobs and cleanup https://review.openstack.org/519828 | 15:28 |
| *** efoley_ has quit IRC | 15:30 | |
| *** pblaho has joined #openstack-infra | 15:30 | |
| *** andreas_s has joined #openstack-infra | 15:31 | |
| *** yamamoto has joined #openstack-infra | 15:31 | |
| *** efoley has joined #openstack-infra | 15:34 | |
| *** tosky has quit IRC | 15:35 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 15:35 |
| *** yamamoto has quit IRC | 15:36 | |
| *** udesale has quit IRC | 15:37 | |
| *** dave-mcc_ has quit IRC | 15:38 | |
| *** pblaho has quit IRC | 15:39 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: debug unbound queries https://review.openstack.org/520055 | 15:39 |
| pas-ha | hi infra, yet another question on zuul3 jobs - the base devstack job (defined in devstack) lists swift and cinder as 'required_projects'. how can I disable those in a inherited job? or this is what 'devstack_services' is for? | 15:39 |
| mordred | pas-ha: devstack_services | 15:41 |
| pas-ha | ok, thanks | 15:41 |
| mordred | pas-ha: required-projects just controls what git repos zuul will prepare things for - so you'll get swift and cinder git repos from zuul - but otherwise that should not break things for you | 15:41 |
| *** xarses_ has joined #openstack-infra | 15:42 | |
| pas-ha | yes, kind of though so | 15:42 |
| pas-ha | another question - is there any role/playbook already that does things similar to what the end of devstack-vm-gate.sh is doing? which is running tempest | 15:43 |
| *** yamamoto has joined #openstack-infra | 15:46 | |
| mordred | pas-ha: we could also do something like: remote: https://review.openstack.org/520115 Extract a devstack-minimal base job | 15:46 |
| *** iyamahat_ has quit IRC | 15:47 | |
| mordred | pas-ha: yes indeed! there is a devstack-tempest job in the tempest repo | 15:47 |
| mordred | pas-ha: andreaf has been working on that | 15:47 |
| pas-ha | oh, cool, will check it out too | 15:48 |
| *** yamamoto has quit IRC | 15:50 | |
| *** andreas_s has quit IRC | 15:53 | |
| *** jascott1 has joined #openstack-infra | 15:56 | |
| *** dhill_ has quit IRC | 15:56 | |
| *** dave-mccowan has joined #openstack-infra | 15:59 | |
| *** ykarel has quit IRC | 15:59 | |
| pabelanger | moring | 16:00 |
| *** jascott1 has quit IRC | 16:00 | |
| pabelanger | I'm restarting ze07.o.o to bring finger daemon backonline | 16:00 |
| *** yamamoto has joined #openstack-infra | 16:01 | |
| *** coolsvap has joined #openstack-infra | 16:01 | |
| *** dhill_ has joined #openstack-infra | 16:02 | |
| *** dave-mccowan has quit IRC | 16:02 | |
| fungi | has anyone figured out yet what causes it to die? | 16:02 |
| *** iyamahat has joined #openstack-infra | 16:02 | |
| *** wolverineav has quit IRC | 16:03 | |
| *** HeOS has quit IRC | 16:04 | |
| *** yamamoto has quit IRC | 16:06 | |
| pabelanger | fungi: Shrews has a patch up to debug more, but waiting for jeblair to review: https://review.openstack.org/517437/ | 16:06 |
| *** caphrim007 has quit IRC | 16:07 | |
| *** efoley has quit IRC | 16:07 | |
| *** efoley has joined #openstack-infra | 16:07 | |
| *** efoley_ has joined #openstack-infra | 16:09 | |
| *** efoley has quit IRC | 16:09 | |
| Shrews | pabelanger: fungi: to be fair, it's not really to "debug more", but more of a "throw something against the wall and see if it sticks" type effort | 16:09 |
| *** dizquierdo has quit IRC | 16:10 | |
| pabelanger | ze07.o.o started again | 16:11 |
| pabelanger | sshnaidm: that is correct. | 16:12 |
| *** smatzek has joined #openstack-infra | 16:12 | |
| fungi | Shrews: informal debugging is still debugging ;) | 16:12 |
| pabelanger | sshnaidm: are you seeing failures only on centos or specific cloud? | 16:12 |
| sshnaidm | pabelanger, I don't know yet, just enabled it.. I think it's better to run jobs with unbound debug to catch the issues | 16:13 |
| pabelanger | k | 16:14 |
| *** smatzek_ has joined #openstack-infra | 16:14 | |
| fungi | looks like we're still running tight on space for the logs.o.o volume with a 4-week retention. i think we're going to be over 90% blocks and around 85% inodes used | 16:16 |
| *** yamamoto has joined #openstack-infra | 16:16 | |
| fungi | we'll _probably_ make it to the expiry run this weekend | 16:17 |
| *** smatzek has quit IRC | 16:17 | |
| fungi | but still concerning because it seems like the past few weeks have been relatively low-activity | 16:17 |
| pabelanger | more failures and ARA is eating up inodes again? | 16:18 |
| AJaeger | do we really have 4 weeks retention, fungi? | 16:18 |
| *** markvoelker_ has joined #openstack-infra | 16:18 | |
| fungi | so as things ramp back to normal volumes we're likely to blow it up again | 16:18 |
| *** zhurong has quit IRC | 16:18 | |
| AJaeger | fungi, let me find something... | 16:18 |
| fungi | AJaeger: yeah, i manually scaled us back to two weeks, but that was more than two weeks ago now | 16:18 |
| clarkb | fungi: I think that same volume is why logstash is perpetually behind | 16:18 |
| AJaeger | http://logs.openstack.org/periodic/aodh-propose-translation-update/af8238b/ | 16:18 |
| AJaeger | 28th of September ^ | 16:19 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: WIP - Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 16:19 |
| clarkb | fungi: we switched to matching with regex on the logstash job submission and its matching more files and that mens more work for logstash/elasticsearch and we aren't keeping up | 16:19 |
| AJaeger | http://logs.openstack.org/periodic/aodh-propose-translation-update/953698d/ 10th of October | 16:19 |
| AJaeger | So, that does not look like 4 weeks to me on these | 16:19 |
| fungi | AJaeger: oh, interesting. i wonder why those aren't deleting | 16:19 |
| AJaeger | do we exclude periodic by accident? | 16:20 |
| *** markvoelker has quit IRC | 16:20 | |
| *** yamamoto has quit IRC | 16:20 | |
| AJaeger | infra-root, did you see my comments from earlier about networking-bgpvn post jobs? I don't find them at all ;( | 16:21 |
| openstackgerrit | Merged openstack-infra/shade master: Document current_user_id in a release note https://review.openstack.org/520093 | 16:21 |
| fungi | AJaeger: it doesn't look like we should be skipping deletion for those files at least: https://git.openstack.org/cgit/openstack-infra/puppet-openstackci/tree/files/log_archive_maintenance.sh | 16:21 |
| *** HeOS has joined #openstack-infra | 16:22 | |
| fungi | AJaeger: oh, october 10... so that's expected... we run weekly and delete files last modified >=30 days prior | 16:23 |
| *** dizquierdo has joined #openstack-infra | 16:23 | |
| fungi | those should be deleted this weekend | 16:24 |
| fungi | so technically our retention is somewhere between 30-37 days | 16:24 |
| *** tosky has joined #openstack-infra | 16:25 | |
| *** andreas_s has joined #openstack-infra | 16:25 | |
| *** martinkopec has quit IRC | 16:25 | |
| fungi | and the deletion takes a few days to complete, i wouldn't even find some as old as 40 days while a deletion pass is underway particularly surprising | 16:25 |
| *** smatzek_ is now known as smatzek | 16:27 | |
| fungi | though seeing that one from september 28 is perhaps unexpected | 16:27 |
| dhellmann | did something about the etherpad server change recently? it's no longer working for me in safari, and I'm trying to narrow down what might have changed to cause that | 16:28 |
| *** HeOS has quit IRC | 16:28 | |
| *** yamamoto has joined #openstack-infra | 16:29 | |
| fungi | AJaeger: it would have been at least 45 days old when the deletion pass kicked off on sunday, so could suggest something is not working as expected here. i'll try to fire it up manually | 16:29 |
| *** yamamoto has quit IRC | 16:29 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/system-config master: Add errors file to be indexed by logstash https://review.openstack.org/520133 | 16:29 |
| *** dizquierdo has quit IRC | 16:29 | |
| sshnaidm | pabelanger, ^ | 16:29 |
| AJaeger | fungi: ok | 16:29 |
| fungi | dhellmann: in the past we've seen some browser extensions make etherpad unusable for people, but safari doesn't do extensions right? as far as the etherpad server, it was last restarted 11 days ago, but hasn't been upgraded for... months at least i think? | 16:30 |
| fungi | when do you last recall it working for you under safari? | 16:31 |
| dhellmann | safari does do extensions, but I haven't installed or updated any lately as far as I know. This was working for me yesterday. | 16:31 |
| *** wolverineav has joined #openstack-infra | 16:31 | |
| dhellmann | looking at the network access, it seems to be hanging on pad.js download | 16:31 |
| dhellmann | or processing? | 16:31 |
| fungi | is it just one particular pad maybe? or any pad you try? | 16:31 |
| dhellmann | oh, good idea, let me try that | 16:32 |
| fungi | sometimes pads can get into a broken state where they exhibit the behavior you describe | 16:32 |
| dhellmann | https://etherpad.openstack.org/p/SYD-forum-ops-guide-transition loads but https://etherpad.openstack.org/p/LTS-proposal is not | 16:32 |
| dhellmann | chrome doesn't have a problem loading the 2nd one | 16:32 |
| fungi | i'm looking at the LTS-proposal pad right now... it's got quite a lot of people connected | 16:33 |
| dhellmann | hmm, let me see if I can figure out how to clear the cache for that page | 16:33 |
| fungi | and more than a couple hundred lines | 16:33 |
| fungi | possible it's a performance or payload size issue under some browsers | 16:33 |
| dhellmann | that could be | 16:33 |
| dhellmann | that pad was working yesterday, earlier in the discussion | 16:34 |
| fungi | the other one you reported working in safari only has two people at this time, including me, and only a few dozen lines of content | 16:34 |
| *** kjackal has quit IRC | 16:38 | |
| fungi | AJaeger: one theory is that the log_archive_maintenance.sh pass is taking more than a week to complete, so the subsequent week's cronjob exits immediately (because of the flock) | 16:38 |
| *** andreas_s has quit IRC | 16:39 | |
| *** mrunge_ has joined #openstack-infra | 16:39 | |
| *** mrunge has quit IRC | 16:39 | |
| fungi | i didn't see one running now, which i found sort of surprising | 16:39 |
| pabelanger | sshnaidm: how large are the files? | 16:40 |
| pabelanger | clarkb: any thoughts on https://review.openstack.org/520133 | 16:40 |
| fungi | AJaeger: but it's possible it took 8 or 9 days to run to completion and so finished earlier this week | 16:40 |
| dhellmann | fungi : yeah, I'm going to chalk this up to a browser limitation of some sort and just use chrome for that pad for now. | 16:41 |
| *** efoley_ has quit IRC | 16:41 | |
| *** efoley has joined #openstack-infra | 16:42 | |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: Add MTU adjustment parameters to RDO Cloud settings https://review.openstack.org/519486 | 16:43 |
| openstackgerrit | Merged openstack-infra/shade master: Fix the devstack role for base functional job https://review.openstack.org/520105 | 16:44 |
| *** jpich has quit IRC | 16:44 | |
| *** Apoorva has joined #openstack-infra | 16:45 | |
| *** yamahata has quit IRC | 16:46 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade master: DNM: Testing to see if we even need this https://review.openstack.org/520140 | 16:46 |
| *** makowals has joined #openstack-infra | 16:49 | |
| *** lucasagomes is now known as lucas-afk | 16:52 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Refactor build-wheel-mirror-base again https://review.openstack.org/520147 | 16:55 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Bring build-wheel-mirror-centos-7 online https://review.openstack.org/520148 | 16:55 |
| pabelanger | fungi: AJaeger: clarkb: a few more build-wheel-mirror patches. 1 is refactor for clarkb, other is enabling centos-7 wheels | 16:56 |
| *** jascott1 has joined #openstack-infra | 16:59 | |
| *** dprince has quit IRC | 16:59 | |
| *** links has quit IRC | 17:00 | |
| *** eumel8 has joined #openstack-infra | 17:00 | |
| *** dave-mccowan has joined #openstack-infra | 17:00 | |
| EmilienM | pabelanger, dmsimard, AJaeger: I made good progress on https://review.openstack.org/#/c/514793, it's now running in zuul - I would love your feedback so if it pass, we can move forward and I can continue the work | 17:01 |
| *** hashar has joined #openstack-infra | 17:01 | |
| *** psachin has quit IRC | 17:02 | |
| *** tesseract has quit IRC | 17:03 | |
| *** lukebrowning_ has joined #openstack-infra | 17:05 | |
| *** hemna_ has quit IRC | 17:05 | |
| *** lukebrowning_ has quit IRC | 17:05 | |
| *** lukebrowning_ has joined #openstack-infra | 17:06 | |
| *** hemna_ has joined #openstack-infra | 17:06 | |
| mwhahaha | so i have a zuul-legacy-project-template.yaml question. If i want to make a set of jobs non-voting for master but still voting for pike, what's the best way to express that | 17:07 |
| *** lukebrowning has quit IRC | 17:08 | |
| mwhahaha | specifically, i'd like to switch scenario001 to non-voting for master but leaving it voting for pike https://github.com/openstack-infra/openstack-zuul-jobs/blob/master/zuul.d/zuul-legacy-project-templates.yaml#L441 | 17:08 |
| sshnaidm | pabelanger, in overcloud and subnode it's usually empty, in undercloud is about 23K | 17:09 |
| clarkb | pabelanger: sshnaidm 520133 is configuring thw wrong file, its now that regex in the ansible role | 17:10 |
| clarkb | byt also logstash isnt currently keeping up, I'd kinda like to see us stabilize it first then add stuff | 17:10 |
| sshnaidm | clarkb, and where to configure it? | 17:12 |
| *** d0ugal has quit IRC | 17:12 | |
| sshnaidm | clarkb, at least I'll prepare the patch | 17:12 |
| pabelanger | mwhahaha: you'll need to create a job variant for pike, and set voting: true. You likely can do this in pipeline stanza | 17:13 |
| mwhahaha | pabelanger: hmm ok, i think i might just switch it to non-voting for both for now instead | 17:14 |
| *** masber has quit IRC | 17:15 | |
| clarkb | sshnaidm: https://git.openstack.org/cgit/openstack-infra/project-config/tree/roles/submit-logstash-jobs/defaults/main.yaml | 17:15 |
| fungi | mwhahaha: you might also consider making it run in the experimental pipeline for those branches rather than non-voting, if it's not expected to pass reliably currently and you're iterating on getting it to work. that way you can run it on demand, but it doesn't get run for changes where you're not paying attention to the results anyway | 17:17 |
| *** mat128 has quit IRC | 17:17 | |
| mwhahaha | fungi: eh we want to keep it visible as it does have a significant amount of coverage. The problem is the ~20-25% failure rate in the gate due to a suspected performance regression. so rather than continuing to hold everything up while we work on figuring it out, we're going to switch to to NV so it doesn't block folks | 17:19 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: Send graphite stats from ARA https://review.openstack.org/480121 | 17:20 |
| *** dangers_away has quit IRC | 17:22 | |
| openstackgerrit | Alex Schultz proposed openstack-infra/project-config master: Switch tripleo-ci scenario001 to non-voting https://review.openstack.org/520155 | 17:24 |
| *** iyamahat has quit IRC | 17:26 | |
| pabelanger | config-core; I'd love some reviews on https://review.openstack.org/#/q/topic:build-wheel-mirror I'd like to finish off build-wheel-mirror jobs for today | 17:27 |
| sshnaidm | clarkb, and what is full path for this line? https://github.com/openstack-infra/project-config/blob/3271b4c0d5be4c5b16814d594554cf44f71744b6/roles/submit-logstash-jobs/defaults/main.yaml#L69 | 17:28 |
| *** makowals has quit IRC | 17:28 | |
| *** hashar is now known as hasharAway | 17:28 | |
| sshnaidm | clarkb, what is "root" for "var/log/extra/logstash\.txt" ? | 17:28 |
| *** dangers has joined #openstack-infra | 17:29 | |
| *** florianf has quit IRC | 17:29 | |
| *** yamamoto has joined #openstack-infra | 17:29 | |
| *** caphrim007 has joined #openstack-infra | 17:30 | |
| *** dizquierdo has joined #openstack-infra | 17:30 | |
| *** caphrim007 has quit IRC | 17:30 | |
| *** claudiub|2 has quit IRC | 17:30 | |
| *** caphrim007 has joined #openstack-infra | 17:30 | |
| *** caphrim007 has quit IRC | 17:31 | |
| *** caphrim007 has joined #openstack-infra | 17:31 | |
| AJaeger | mwhahaha: move the job in repo and have different config per branch. You backport the jobs to each branch and can configure them this way | 17:33 |
| mwhahaha | AJaeger: yea that assumes we're in a state to do that (we're not) | 17:33 |
| *** gyee_ has joined #openstack-infra | 17:34 | |
| *** yamamoto has quit IRC | 17:35 | |
| *** dangers is now known as dangers_away | 17:36 | |
| openstackgerrit | David Shrewsbury proposed openstack-infra/shade master: Remove unnecessary roles reference https://review.openstack.org/520140 | 17:36 |
| *** jascott1 has quit IRC | 17:40 | |
| *** jascott1 has joined #openstack-infra | 17:40 | |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: DNM, update logging for undercloud containers https://review.openstack.org/518972 | 17:42 |
| *** Hal has quit IRC | 17:43 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/git-review master: Handle http queries below / https://review.openstack.org/478325 | 17:44 |
| clarkb | fungi: ianw mordred ^ fyi wmf pinged me about this issue, so if that doesn't look too objectionable would be nice to have in the git review release we will be making soon | 17:44 |
| *** iyamahat has joined #openstack-infra | 17:45 | |
| *** jascott1 has quit IRC | 17:45 | |
| dirk | why are the periodic jobs running multiple times a day now? | 17:45 |
| clarkb | sshnaidm: I think it is the central log location for the job | 17:45 |
| dirk | we get "generate-constraints" run on the requirements repo at least twice a day | 17:45 |
| clarkb | sshnaidm: we copy logs from each of the job nodes to the executor and then from there copy it to the log server. The relative location is the logs dirfor the build on the executor | 17:46 |
| pabelanger | dirk: that is me testing things | 17:46 |
| pabelanger | trying to get build-wheel-mirror jobs working | 17:46 |
| pabelanger | and requires we enqueue them by hand | 17:46 |
| *** dtantsur is now known as dtantsur|afk | 17:46 | |
| openstackgerrit | Jose Luis Franco proposed openstack-infra/tripleo-ci master: WIP: Upgrade UC and OC using tripleo-upgrade role https://review.openstack.org/515643 | 17:47 |
| dirk | pabelanger: ah, so you're triggering all periodic jobs? or is the build-wheel thing a post of the generate-constraints job? | 17:47 |
| *** shardy has quit IRC | 17:48 | |
| *** mpaolino has quit IRC | 17:51 | |
| pabelanger | dirk: I am manually triggering them | 17:54 |
| AJaeger | pabelanger: should we +A https://review.openstack.org/519839 - that will avoid some of the changes to go out | 17:54 |
| *** rhallisey has quit IRC | 17:58 | |
| pabelanger | AJaeger: sure, have some time to watch | 17:58 |
| *** derekh has quit IRC | 17:59 | |
| *** ralonsoh has quit IRC | 18:01 | |
| AJaeger | thanks | 18:02 |
| *** yamahata has joined #openstack-infra | 18:02 | |
| *** bobh has joined #openstack-infra | 18:05 | |
| *** Swami has joined #openstack-infra | 18:06 | |
| mnaser | pabelanger: EmilienM mwhahaha im going to be approving the backported zuul v3 changes in chunks of 5 | 18:06 |
| mnaser | to avoid letting zuul v3 oom | 18:07 |
| AJaeger | mnaser: with long breaks between ;) | 18:07 |
| mnaser | AJaeger: yes i'll make sure to let things settle between then and keep an eye on cacti.o.o | 18:08 |
| openstackgerrit | Merged openstack-infra/project-config master: Don't re-propose requirements changes that already exist. https://review.openstack.org/519839 | 18:08 |
| *** efoley has quit IRC | 18:08 | |
| mnaser | http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=63979&rra_id=all | 18:08 |
| mnaser | i guess this is what i'll keep an eye out on | 18:08 |
| openstackgerrit | Clark Boylan proposed openstack-infra/git-review master: Handle http queries below / https://review.openstack.org/478325 | 18:08 |
| clarkb | fungi: ^ sorry new tests were added that failed with after my rebase (because I didn't update them). I think this patchset addresses that | 18:09 |
| AJaeger | config-core, could you review https://review.openstack.org/519328 to remove a job from project-config and https://review.openstack.org/517829 to fix grafana for zuul, please? | 18:09 |
| *** dprince has joined #openstack-infra | 18:11 | |
| *** pcaruana has quit IRC | 18:11 | |
| pabelanger | mnaser: thanks | 18:12 |
| openstackgerrit | Ronelle Landy proposed openstack-infra/tripleo-ci master: Be more prescriptive in log collection https://review.openstack.org/511526 | 18:13 |
| *** mat128 has joined #openstack-infra | 18:13 | |
| *** sdague has quit IRC | 18:13 | |
| mnaser | forgive me, but do we have to keep noop-jobs if all jobs are removed in project-config? (but the project has jobs in-repo) | 18:14 |
| openstackgerrit | Merged openstack-infra/project-config master: Refactor build-wheel-mirror-base again https://review.openstack.org/520147 | 18:15 |
| *** jascott1 has joined #openstack-infra | 18:15 | |
| AJaeger | mnaser: we should remove noop-jobs if there are jobs in both gate and check somewhere | 18:16 |
| mnaser | AJaeger: ok gotcha | 18:16 |
| *** jpena is now known as jpena|off | 18:16 | |
| *** samueldmq has quit IRC | 18:19 | |
| *** samueldmq has joined #openstack-infra | 18:19 | |
| *** rosmaita has quit IRC | 18:20 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove tripleo-ui legacy layout https://review.openstack.org/519829 | 18:21 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove legacy Castellan jobs https://review.openstack.org/515776 | 18:21 |
| *** rhallisey has joined #openstack-infra | 18:23 | |
| sshnaidm | clarkb, in this case there is an error in current config, it should be undercloud/var/log... not var/log/... | 18:24 |
| sshnaidm | clarkb, I'll fix it in my patch too | 18:24 |
| clarkb | sshnaidm: also it is a regex | 18:26 |
| *** trown is now known as trown|lunch | 18:26 | |
| clarkb | sshnaidm: I'm not sure if it uses re.match or re.serach in python but that may be fine | 18:26 |
| sshnaidm | clarkb, if it uses re.search, then maybe I can add only one line instead of 6 | 18:27 |
| clarkb | sshnaidm: the code is there in the role it calls | 18:27 |
| openstackgerrit | Merged openstack-infra/project-config master: Bring build-wheel-mirror-centos-7 online https://review.openstack.org/520148 | 18:28 |
| openstackgerrit | Merged openstack-infra/project-config master: Switch tripleo-ci scenario001 to non-voting https://review.openstack.org/520155 | 18:28 |
| openstackgerrit | Merged openstack-infra/project-config master: Add check-requirements to manila-tempest-plugin https://review.openstack.org/520034 | 18:28 |
| openstackgerrit | Merged openstack-infra/project-config master: Remove legacy python-manilaclient jobs https://review.openstack.org/519328 | 18:28 |
| clarkb | sshnaidm: https://git.openstack.org/cgit/openstack-infra/project-config/tree/roles/submit-log-processor-jobs/library/submit_log_processor_jobs.py#n34 it uses search | 18:28 |
| mnaser | https://review.openstack.org/#/c/519534/ -- should this be marked WIP as the other packages were (i believe looking over meeting notes it was mentioned that an email will be sent out first before this is done) | 18:29 |
| AJaeger | mnaser: yes - but it's on top of https://review.openstack.org/#/c/519533/ which is WIP. So, I didn't bother ;) | 18:31 |
| *** aspiers has quit IRC | 18:31 | |
| mnaser | aha, i see :p | 18:31 |
| AJaeger | ianw: but might be nice if you WIP both - and then I'll remove my WIP ;) ^ | 18:31 |
| *** yamamoto has joined #openstack-infra | 18:31 | |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/project-config master: Add errors file to be indexed by logstash https://review.openstack.org/520171 | 18:33 |
| sshnaidm | clarkb, pabelanger is it ok? ^^ | 18:33 |
| *** mrunge_ is now known as mrunge | 18:34 | |
| *** tosky has quit IRC | 18:34 | |
| jlvillal | pabelanger: Regarding https://review.openstack.org/519839 Did you watch the job to make sure it is working? | 18:35 |
| jlvillal | pabelanger: And thanks for merging it. I tested things locally, but see what happens in production... | 18:35 |
| *** e0ne has quit IRC | 18:36 | |
| jlvillal | pabelanger: And also, is the new code running already? | 18:36 |
| *** yamamoto has quit IRC | 18:37 | |
| *** dsariel has quit IRC | 18:37 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove horizon legacy jobs (django variants, selenium-headless) https://review.openstack.org/519047 | 18:38 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: legacy-cross-networking-midonet-python35: Require requirements https://review.openstack.org/519995 | 18:38 |
| *** aspiers has joined #openstack-infra | 18:39 | |
| *** coolsvap has quit IRC | 18:40 | |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove migrated legacy qinling jobs https://review.openstack.org/519272 | 18:41 |
| openstackgerrit | Merged openstack-infra/openstack-zuul-jobs master: Remove Barbican legacy jobs https://review.openstack.org/510414 | 18:41 |
| openstackgerrit | Merged openstack-infra/project-config master: Add openstack-tox-cover check to Patrole https://review.openstack.org/519966 | 18:41 |
| pabelanger | jlvillal: yah, keeping an eye on it now, was going to ask requirements to approve something | 18:42 |
| jlvillal | pabelanger: Thanks! I hope it works :) | 18:42 |
| *** makowals has joined #openstack-infra | 18:42 | |
| pabelanger | jlvillal: I was assuming it was locally tested :) | 18:42 |
| jlvillal | hrh | 18:43 |
| jlvillal | heh | 18:43 |
| *** dhinesh has joined #openstack-infra | 18:44 | |
| AJaeger | mordred: can we remove zuul-migrate now? see https://review.openstack.org/516028 | 18:46 |
| *** jascott1 has quit IRC | 18:47 | |
| *** jascott1 has joined #openstack-infra | 18:48 | |
| *** kjackal has joined #openstack-infra | 18:48 | |
| *** dsariel has joined #openstack-infra | 18:49 | |
| pabelanger | AJaeger: think so, we talked at summit about deleting zuul-migrate python script, since it is broken. However, there was discussions about a rewrite and attempt to support it to aid people converting from JJB | 18:52 |
| AJaeger | pabelanger, clarkb : for openstack-infra/publications, we need to fix the index file generation, could you review https://review.openstack.org/516009 , please? | 18:53 |
| AJaeger | pabelanger: yeah, that might be helpfull for a conversion - but it's not usefull in its current form. | 18:54 |
| openstackgerrit | Matt Riedemann proposed openstack-infra/devstack-gate master: Checkout from -eol tag if stable branch does not exist https://review.openstack.org/520095 | 18:59 |
| *** andreww has joined #openstack-infra | 18:59 | |
| *** andreww has quit IRC | 19:00 | |
| *** dizquierdo has quit IRC | 19:00 | |
| *** iyamahat has quit IRC | 19:00 | |
| *** xarses_ has quit IRC | 19:01 | |
| *** iyamahat has joined #openstack-infra | 19:01 | |
| *** andreww has joined #openstack-infra | 19:02 | |
| mtreinish | fungi, clarkb: can you check the subunit workers to make sure the python script is up to date? | 19:03 |
| *** tosky has joined #openstack-infra | 19:03 | |
| mtreinish | I'm seeing the unaltered names on the failed list: http://status.openstack.org/openstack-health/#/ | 19:04 |
| mtreinish | oops, that was supposed to be the screen shot, not the o-h link | 19:04 |
| mtreinish | although I guess it works the same :) | 19:04 |
| mtreinish | https://i.imgur.com/70YvTbF.png | 19:04 |
| mtreinish | https://review.openstack.org/#/c/508258/ is supposed to fix that (and it was after in the past) | 19:05 |
| *** rosmaita has joined #openstack-infra | 19:12 | |
| openstackgerrit | Merged openstack-infra/shade master: Remove unnecessary roles reference https://review.openstack.org/520140 | 19:16 |
| clarkb | fungi: re log storage could be also side effect of adding the ansible json output too? | 19:17 |
| clarkb | mtreinish: on worker02 /usr/local/bin/subunit-gearman-worker.py was updated october 16 (when that change merged looks like) and process there has been running since october 31 | 19:18 |
| clarkb | mtreinish: so it should be running the code /me checks if 01 got rebuilt | 19:18 |
| clarkb | no 01 in dns | 19:18 |
| *** d0ugal has joined #openstack-infra | 19:20 | |
| *** trown|lunch is now known as trown | 19:21 | |
| openstackgerrit | Manuel Buil proposed openstack/diskimage-builder master: Add other OS to the documentation https://review.openstack.org/518968 | 19:24 |
| openstackgerrit | Merged openstack-infra/project-config master: snaps: add snap-gnocchi definitions https://review.openstack.org/500964 | 19:24 |
| *** rwsu has quit IRC | 19:24 | |
| fungi | clarkb: mtreinish: right, 01 is pending getting the puppet module working on xenial, i need to retry launching that and seeing where i left off | 19:25 |
| *** caphrim007_ has joined #openstack-infra | 19:26 | |
| *** tobiash has quit IRC | 19:27 | |
| *** tobiash has joined #openstack-infra | 19:27 | |
| *** Sukhdev has joined #openstack-infra | 19:28 | |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/infra-manual master: Add a section on secrets to the migration guide https://review.openstack.org/520181 | 19:29 |
| mtreinish | clarkb: hmm, then I wonder why the names aren't getting updated | 19:29 |
| *** caphrim007 has quit IRC | 19:29 | |
| *** yamamoto has joined #openstack-infra | 19:33 | |
| *** yamamoto has quit IRC | 19:38 | |
| *** e0ne has joined #openstack-infra | 19:39 | |
| frickler | mtreinish: hmm, the names are updated on the project details page like http://status.openstack.org/openstack-health/#/g/project/openstack~2Fheat-agents | 19:43 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Install parallel from EPEL for build-wheel-mirror https://review.openstack.org/520185 | 19:45 |
| mtreinish | frickler: hmm, yeah I see the fixed names everywhere else. I wonder if there is something specifically with that api call acting up | 19:45 |
| inc0 | pabelanger: I have problem using docker with mirror.rax.../docker-registry1 (or whatever url is) | 19:45 |
| inc0 | problem is, it complains about /docker-registry | 19:45 |
| *** dprince has quit IRC | 19:45 | |
| pabelanger | inc0: how did you set it up? | 19:45 |
| inc0 | can we set squid to listen on port rather than url? | 19:46 |
| pabelanger | we are not using squid | 19:46 |
| inc0 | I try both https://review.openstack.org/#/c/519939/ | 19:46 |
| inc0 | well,apache then:) whatever you are using | 19:46 |
| inc0 | https://review.openstack.org/#/c/519939/9/tests/templates/globals-default.j2 | 19:47 |
| inc0 | I also tried docker pull mirror.dfw.rax.openstack.org:8081/registry-1.docker/kolla/ubuntu-base-source | 19:47 |
| inc0 | which is what happens underneath I assume | 19:47 |
| inc0 | it's REST api so url is relevant | 19:48 |
| pabelanger | inc0: try doing what we did for the commit message: https://review.openstack.org/453811/ | 19:48 |
| pabelanger | also, make sure you have trailing slash | 19:48 |
| pabelanger | you can then enable debug on docker and see what is happening | 19:48 |
| mtreinish | frickler: ah, yeah it looks like a bug, the api call for that table doesn't actually get the name from the db: http://health.openstack.org/tests/recent/fail | 19:48 |
| pabelanger | inc0: this is working in tripleo jobs too, so you can also look at there setup | 19:49 |
| inc0 | ok thanks I'll keep poking around | 19:50 |
| mtreinish | frickler: yep, it's just pulling it from the url instead of looking at the db: https://github.com/openstack/openstack-health/blob/master/app/views/home.html#L124 | 19:50 |
| pabelanger | what version of docker too? I had some issues with newer versions, but never debugged why | 19:51 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 19:51 |
| EmilienM | AJaeger: thanks, you gave me the idea of refactoring the playbooks, on https://review.openstack.org/#/c/514793/ - see results | 19:51 |
| pabelanger | 46.254.11.128 - - [15/Nov/2017:16:53:03 +0000] "GET /registry-1.docker/v2/tripleomaster/centos-binary-rabbitmq/blobs/sha256:8580a13c71440e274305d7734180437821501104f70fc31fef45cfd2e058ff26 HTTP/1.1" 307 1130 "-" "docker/1.12.6 go/go1.8.3 kernel/3.10.0-693.5.2.el7.x86_64 os/linux arch/amd64 UpstreamClient(docker-sdk-python/2.4.2)" | 19:53 |
| pabelanger | inc0: that is an example if it working ^ | 19:53 |
| pabelanger | inc0: link me a log with a failure and I can see what is happening on the reverse proxy | 19:54 |
| *** salv-orl_ has joined #openstack-infra | 19:55 | |
| *** salv-orlando has quit IRC | 19:55 | |
| inc0 | pabelanger: http://paste.openstack.org/show/626438/ | 19:56 |
| inc0 | that's what I get from local | 19:56 |
| inc0 | maybe it's permission issue, do you block non-nodepool traffic? | 19:56 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/openstack-zuul-jobs master: Move TripleO jobs/templates to openstack-infra/tripleo-ci https://review.openstack.org/514792 | 19:56 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Install parallel from EPEL for build-wheel-mirror https://review.openstack.org/520185 | 19:57 |
| pabelanger | inc0: not permission | 19:57 |
| pabelanger | let me check apache logs | 19:57 |
| inc0 | thanks | 19:57 |
| SamYaple | inc0: pabelanger that image doesnt exist. https://hub.docker.com/r/kolla/ubuntu-base-source/ | 19:58 |
| pabelanger | 192.55.66.149 - - [15/Nov/2017:19:49:49 +0000] "GET /registry-1.docker/v2/ HTTP/1.1" 401 455 "-" "docker/1.13.1 go/go1.7.5 git-commit/092cba3 kernel/4.4.0-98-generic os/linux arch/amd64 UpstreamClient(Docker-Client/1.13.1 \\(linux\\))" | 19:58 |
| pabelanger | yah | 19:59 |
| inc0 | my god | 19:59 |
| pabelanger | SamYaple: was just about to say that | 19:59 |
| SamYaple | inc0: its ok we still love you | 19:59 |
| *** amoralej is now known as amoralej|off | 19:59 | |
| inc0 | somebody take keyboard from me before I hurt myself >< | 20:00 |
| SamYaple | **hugs** | 20:00 |
| pabelanger | at least docker gave the right error message: Error response from daemon: repository kolla/ubuntu-base-source not found: does not exist or no pull access :) | 20:01 |
| *** docaedo has joined #openstack-infra | 20:01 | |
| SamYaple | i would have been hunting like inc0 but i noticed the name was wrong and double checked. ubuntu-source-base | 20:02 |
| fungi | "unexpected error condition: all systems normal" | 20:02 |
| *** caphrim007_ has quit IRC | 20:02 | |
| SamYaple | fungi: now that is a valid one. ive done that before | 20:02 |
| SamYaple | because we had a constant alarm on one site, but if we lost contact for a time the crappy software would report all clear | 20:02 |
| *** caphrim007 has joined #openstack-infra | 20:02 | |
| fungi | heh | 20:02 |
| SamYaple | so if it was all clear, there was a problem | 20:03 |
| * fungi suddenly had a not-so-fond flashback of "what's up gold" | 20:03 | |
| dmsimard | pabelanger: where is the executor work root when you use keep ? | 20:03 |
| fungi | er, i guess it's spelled "WhatsUp Gold" | 20:04 |
| SamYaple | fungi: this was nec microwave radios stuff with proprietary monitoring | 20:04 |
| SamYaple | so yay :) | 20:04 |
| fungi | even better! | 20:04 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Install parallel from EPEL for build-wheel-mirror https://review.openstack.org/520185 | 20:04 |
| pabelanger | dmsimard: /var/lib/zuul/builds/ | 20:05 |
| dmsimard | clarkb: saw a comment about log storage, just a reminder that https://review.openstack.org/#/q/topic:ara-sqlite-middleware is a thing (see: http://ara.readthedocs.io/en/latest/advanced.html ) | 20:05 |
| dmsimard | there might be an issue with the vhost config though, someone told me about certain paths not working but I haven't had the chance to look yet | 20:05 |
| dmsimard | pabelanger: thanks | 20:05 |
| *** iyamahat has quit IRC | 20:06 | |
| pabelanger | AJaeger: our ansible-linter jobs are working! | 20:06 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 20:06 |
| dmsimard | clarkb: I'm biased but I'd very much love getting ara reports even in successful jobs | 20:07 |
| *** iyamahat has joined #openstack-infra | 20:07 | |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: DNM, update logging for undercloud containers https://review.openstack.org/518972 | 20:07 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/infra-manual master: Add a section on secrets to the migration guide https://review.openstack.org/520181 | 20:09 |
| *** slaweq has joined #openstack-infra | 20:10 | |
| *** slaweq has quit IRC | 20:15 | |
| *** nsingh has quit IRC | 20:15 | |
| *** nsingh has joined #openstack-infra | 20:16 | |
| *** slaweq has joined #openstack-infra | 20:18 | |
| *** slaweq has quit IRC | 20:18 | |
| dmsimard | so I have a question about project creation | 20:19 |
| *** slaweq has joined #openstack-infra | 20:19 | |
| *** slaweq_ has joined #openstack-infra | 20:21 | |
| EmilienM | when I define a project layout in .zuul.yaml, can I modify on the fly the "irrelevant-files" of a job? (and it will override the original one?) | 20:22 |
| dmsimard | If I take this review: https://review.openstack.org/#/c/515316/ -- We're creating the project *and* adding it to the zuul configuration simultaneously... how does zuul not initially fail to load the project, because it hasn't been created in gerrit yet ? Is there some orchestration involved ? | 20:24 |
| *** slaweq has quit IRC | 20:25 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Make build-python-release job https://review.openstack.org/513925 | 20:25 |
| *** d0ugal_ has joined #openstack-infra | 20:25 | |
| ianw | jlvillal / pabelanger: let's keep an eye on the proposals after https://review.openstack.org/#/c/519839/ | 20:25 |
| pabelanger | ianw: yah, doing so now | 20:26 |
| *** d0ugal has quit IRC | 20:26 | |
| dmsimard | EmilienM: yes, you can override job-level parameters directly in the layout | 20:26 |
| pabelanger | ianw: https://review.openstack.org/520185/ could use a +3 to keep centos-7 wheel-builders moving foward | 20:27 |
| EmilienM | dmsimard: http://paste.openstack.org/show/TVFXICWgAA8aJNkabZFp/ will work ??? | 20:27 |
| openstackgerrit | Monty Taylor proposed openstack-infra/openstack-zuul-jobs master: Stop using tox to build python tarballs https://review.openstack.org/513927 | 20:28 |
| dmsimard | EmilienM: yes | 20:29 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Remove old python-sdist job https://review.openstack.org/513926 | 20:30 |
| EmilienM | this is freaking awesome | 20:30 |
| dmsimard | EmilienM: should be wary not to let things get out of hand, it gives you to power to make things dirty and all over the place :) | 20:31 |
| EmilienM | we're dirty | 20:31 |
| *** jbadiapa_ has joined #openstack-infra | 20:33 | |
| AJaeger | pabelanger: yeah! | 20:33 |
| mriedem | clarkb: fungi: mordred: this will unblock stable/newton for nova to get a cve out and eol that branch https://review.openstack.org/#/c/520095/ | 20:34 |
| mriedem | it's got a passing run, linked in the comments | 20:34 |
| *** yamamoto has joined #openstack-infra | 20:34 | |
| *** rwsu has joined #openstack-infra | 20:35 | |
| *** jbadiapa has quit IRC | 20:36 | |
| *** sdague has joined #openstack-infra | 20:36 | |
| openstackgerrit | Merged openstack-infra/project-config master: Install parallel from EPEL for build-wheel-mirror https://review.openstack.org/520185 | 20:37 |
| mordred | mriedem: that's fine by me for this situation - I believe we'll want to think about this more broadly as we move more things to the new v3-native devstack base job | 20:38 |
| fungi | thanks for the heads-up, mriedem! i was trying to keep tabs on that one | 20:38 |
| mriedem | mordred: alright, that's way over my head | 20:39 |
| *** yamamoto has quit IRC | 20:39 | |
| *** Sukhdev has quit IRC | 20:39 | |
| jlvillal | ianw, Regarding https://review.openstack.org/#/c/519839/ Is that code now running? | 20:40 |
| *** d0ugal_ has quit IRC | 20:40 | |
| *** d0ugal has joined #openstack-infra | 20:40 | |
| *** d0ugal has quit IRC | 20:40 | |
| *** d0ugal has joined #openstack-infra | 20:40 | |
| *** armaan has quit IRC | 20:40 | |
| ianw | jlvillal: i guess it comes out in the next build, or we could manually encourage one | 20:41 |
| jlvillal | ianw, Okay. I was curious because 4 minutes after it had merged a proposal went out. And I wasn't sure if new or old code was running. | 20:41 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config master: WIP: Add subunit2sql check database server https://review.openstack.org/520192 | 20:41 |
| mtreinish | clarkb, fungi, mordred: ^^^ I think I'm on the right path with that, but it clearly needs more work (and I still have to test it) | 20:42 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul feature/zuulv3: DNM no-op patch to test the current gate job https://review.openstack.org/520194 | 20:42 |
| *** Apoorva has quit IRC | 20:43 | |
| *** andreas_s has joined #openstack-infra | 20:43 | |
| *** Apoorva has joined #openstack-infra | 20:43 | |
| mtreinish | I figured we could just use: https://forge.puppet.com/puppetlabs/mysql since it seems to do everything we need (although I wasn't sure how to declare that dependency in system-config) | 20:43 |
| *** Apoorva has quit IRC | 20:43 | |
| pabelanger | jlvillal: ianw: http://zuulv3.openstack.org/static/stream.html?uuid=7421f013657f45f6aedeb8f5639332e2&logfile=console.log | 20:45 |
| pabelanger | that should be a propose.sh update | 20:45 |
| jlvillal | thanks | 20:45 |
| *** andreas_s has quit IRC | 20:47 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Move yum install for parallel into pre_tasks https://review.openstack.org/520195 | 20:47 |
| pabelanger | ianw: :( ^should have been pre_tasks | 20:48 |
| ianw | jlvillal: ahh, i'd missed "legacy-copy-project-config-scripts : Copy the scripts directory to the remote node" ... so yeah, it runs directly | 20:49 |
| jlvillal | ianw, Okay. I'm watching the run. that "generate-constraints" part takes a long time... | 20:50 |
| *** felipemonteiro has joined #openstack-infra | 20:52 | |
| *** d0ugal has quit IRC | 20:52 | |
| *** camunoz has quit IRC | 20:55 | |
| *** armaan has joined #openstack-infra | 20:55 | |
| openstackgerrit | Merged openstack-infra/shade master: Fix creating a server with specifying scheduler_hints https://review.openstack.org/519705 | 20:56 |
| ianw | jlvillal: that seems to have correctly proposed https://review.openstack.org/#/c/520174/2..3/upper-constraints.txt ? | 20:57 |
| jlvillal | ianw, Okay good. What I'm waiting for is the negative case. When it does not re-propose :) | 20:58 |
| jlvillal | ianw, But glad it does propose when it should. | 20:59 |
| ianw | though "1 file changed, 12 insertions(+), 12 deletions(-)" seems odd for the one char change | 20:59 |
| pabelanger | cool | 21:01 |
| jlvillal | ianw, https://review.openstack.org/#/c/520174/3/upper-constraints.txt | 21:01 |
| jlvillal | ianw, More than one character for the whole patch | 21:01 |
| jlvillal | ianw, One character between the two patch sets | 21:01 |
| ianw | oh, ok | 21:01 |
| openstackgerrit | Jeremy Stanley proposed openstack-infra/infra-manual master: Add a section on secrets to the migration guide https://review.openstack.org/520181 | 21:02 |
| * jlvillal walks back to work after lunch... | 21:02 | |
| openstackgerrit | Merged openstack-infra/project-config master: Move yum install for parallel into pre_tasks https://review.openstack.org/520195 | 21:03 |
| pabelanger | jlvillal: ianw: just enqueue a periodic job again, for testing. hopefully that gets no patches generated this time | 21:04 |
| pabelanger | will link stream in a moment | 21:05 |
| *** Goneri has quit IRC | 21:06 | |
| pabelanger | http://zuulv3.openstack.org/static/stream.html?uuid=56b32fc1847c4de589b5c6419abc1f05&logfile=console.log | 21:10 |
| *** smatzek has quit IRC | 21:12 | |
| ianw | fungi / clarkb: if you haven't had enough of "bang your head against the desk" stuff today, try the "general information" in https://github.com/Drekin/win-unicode-console (apropos https://review.openstack.org/#/c/519850) | 21:13 |
| ianw | what is generate constraints doing to take so long? | 21:14 |
| fungi | ianw: installing all of global-requirements.txt | 21:14 |
| fungi | including building some c extensions from source | 21:14 |
| ianw | i would have thought it was wheeled, but i guess that might be wip | 21:15 |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Fix syntax error with when statement for build-wheel-mirror https://review.openstack.org/520205 | 21:16 |
| *** sree has joined #openstack-infra | 21:16 | |
| fungi | ianw: i think it has to not use wheels because that might cause it to miss some transitive setup_requires? | 21:17 |
| jlvillal | pabelanger: thanks. I'm watching it | 21:18 |
| inc0 | pabelanger: registry mirror will always be {{ zuul_site_mirror_fqdn }}:8081/registry-1.docker/ ? | 21:20 |
| *** dhill_ has quit IRC | 21:20 | |
| pabelanger | inc0: should be, I think we layout /etc/ci/mirror.sh still, which has some variables | 21:20 |
| *** dhill_ has joined #openstack-infra | 21:21 | |
| pabelanger | you likely can source that | 21:21 |
| *** sree has quit IRC | 21:21 | |
| inc0 | well I'm laying down configs in ansible, so I need to base on facts | 21:21 |
| pabelanger | yah, we should convert them to facts too | 21:21 |
| *** jtomasek has quit IRC | 21:21 | |
| pabelanger | just haven't yet | 21:21 |
| pabelanger | should be able to use what you have for now, until we talk facts | 21:22 |
| pabelanger | will add that to next meeting | 21:22 |
| inc0 | thansk | 21:23 |
| jlvillal | pabelanger: Okay. It did not re-propose :) | 21:24 |
| pabelanger | jlvillal: hey! your hired | 21:25 |
| pabelanger | :) | 21:25 |
| jlvillal | pabelanger: heh :) | 21:25 |
| pabelanger | ianw: left comment, but was following http://docs.ansible.com/ansible/latest/playbooks_conditionals.html#the-when-statement | 21:25 |
| pabelanger | ianw: but should be able to test locally | 21:25 |
| jlvillal | ianw: ^^^ Basically it did not re-propose the change | 21:25 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Test something https://review.openstack.org/520208 | 21:28 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 21:29 |
| openstackgerrit | Merged openstack-infra/project-config master: Fix syntax error with when statement for build-wheel-mirror https://review.openstack.org/520205 | 21:30 |
| *** armaan has quit IRC | 21:31 | |
| *** threestrands has joined #openstack-infra | 21:32 | |
| ianw | jlvillal: excellent ... did you port to the thing AJaeger mentioned? | 21:33 |
| jlvillal | ianw: Uh...no | 21:33 |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Test something https://review.openstack.org/520208 | 21:33 |
| jlvillal | I'll go look... | 21:33 |
| *** hasharAway has quit IRC | 21:34 | |
| *** yamamoto has joined #openstack-infra | 21:36 | |
| *** slaweq_ has quit IRC | 21:40 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 21:40 |
| *** trown is now known as trown|outtypewww | 21:41 | |
| *** yamamoto has quit IRC | 21:41 | |
| *** slaweq has joined #openstack-infra | 21:41 | |
| openstackgerrit | David Moreau Simard proposed openstack-infra/openstack-zuul-jobs master: Test something https://review.openstack.org/520208 | 21:41 |
| *** slaweq_ has joined #openstack-infra | 21:42 | |
| *** slaweq has quit IRC | 21:43 | |
| *** wolverineav has quit IRC | 21:44 | |
| fungi | ianw: the issues that win-unicode-console lib aim to address look mostly solved in python 3.6 now, so perhaps given the nature of the python for windows ecosystem it could simply be easiest to tell windows users to run git-review under python >=3.6 now? | 21:45 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-log_processor master: Add separate gearman job type for check jobs https://review.openstack.org/520212 | 21:46 |
| openstackgerrit | Matthew Treinish proposed openstack-infra/puppet-subunit2sql master: Add config option for check data https://review.openstack.org/520213 | 21:46 |
| openstackgerrit | Sagi Shnaidman proposed openstack-infra/tripleo-ci master: DNM: debug unbound queries https://review.openstack.org/520055 | 21:46 |
| *** caphrim007_ has joined #openstack-infra | 21:47 | |
| *** rwsu has quit IRC | 21:47 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/system-config master: WIP: Add subunit2sql check database server and workers https://review.openstack.org/520192 | 21:48 |
| *** caphrim007 has quit IRC | 21:49 | |
| *** dsariel has quit IRC | 21:55 | |
| openstackgerrit | John L. Villalovos proposed openstack-infra/project-config master: Don't re-propose translation changes that already exist. https://review.openstack.org/520216 | 21:56 |
| *** jascott1 has quit IRC | 21:58 | |
| *** jascott1 has joined #openstack-infra | 21:59 | |
| *** jascott1 has quit IRC | 21:59 | |
| *** smatzek has joined #openstack-infra | 22:00 | |
| *** jascott1 has joined #openstack-infra | 22:02 | |
| *** jascott1 has quit IRC | 22:02 | |
| *** ccamacho has quit IRC | 22:02 | |
| *** jascott1 has joined #openstack-infra | 22:03 | |
| openstackgerrit | wes hayutin proposed openstack-infra/tripleo-ci master: DNM, update logging for undercloud containers https://review.openstack.org/518972 | 22:03 |
| *** smatzek has quit IRC | 22:04 | |
| *** makowals has quit IRC | 22:06 | |
| *** rwsu has joined #openstack-infra | 22:08 | |
| *** e0ne has quit IRC | 22:09 | |
| *** eumel8 has quit IRC | 22:09 | |
| *** e0ne has joined #openstack-infra | 22:09 | |
| *** e0ne has quit IRC | 22:10 | |
| *** e0ne has joined #openstack-infra | 22:11 | |
| *** e0ne has quit IRC | 22:11 | |
| *** slaweq has joined #openstack-infra | 22:11 | |
| *** e0ne has joined #openstack-infra | 22:12 | |
| *** e0ne has quit IRC | 22:13 | |
| *** rwsu has quit IRC | 22:14 | |
| *** e0ne has joined #openstack-infra | 22:14 | |
| *** rhallisey has quit IRC | 22:14 | |
| *** e0ne has quit IRC | 22:14 | |
| *** e0ne has joined #openstack-infra | 22:15 | |
| *** e0ne has quit IRC | 22:15 | |
| *** e0ne has joined #openstack-infra | 22:15 | |
| *** slaweq has quit IRC | 22:15 | |
| *** e0ne has quit IRC | 22:16 | |
| *** slaweq_ has quit IRC | 22:17 | |
| *** slaweq has joined #openstack-infra | 22:18 | |
| *** kgiusti has left #openstack-infra | 22:19 | |
| openstackgerrit | Paul Belanger proposed openstack-infra/project-config master: Just enable epel for build-wheel-mirror centos-7 job https://review.openstack.org/520228 | 22:22 |
| pabelanger | ianw: okay, ^we need EPEL for more things | 22:23 |
| *** rcernin has joined #openstack-infra | 22:23 | |
| pabelanger | and now I step away to eat | 22:24 |
| ianw | fungi: i'd be ok with that too ... it seems like you must have gone to some effort to get the python runtime on windows so just downloading another installer seems ok. honestly i can't even think how i'd get access to a windows machine to try it :/ | 22:24 |
| *** rwsu has joined #openstack-infra | 22:25 | |
| mordred | fungi, ianw: tobiash has windows hosts in his infrastructure and might be a decent person to ask | 22:27 |
| ianw | if anyone here would like to admit to running windows & wants to help test git-review with some pushing and pulling of some sandbox changes with unicode chars, let me know and i'll walk you through it :) | 22:32 |
| ianw | pabelanger: what wheel build of a requirements library needs an EPEL package? that feels like an issue. the afs stuff i can see with dkms etc | 22:33 |
| *** rcarrillocruz has quit IRC | 22:34 | |
| *** esberglu has quit IRC | 22:36 | |
| *** jascott1 has quit IRC | 22:36 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 22:36 |
| *** jascott1 has joined #openstack-infra | 22:37 | |
| *** esberglu has joined #openstack-infra | 22:37 | |
| *** yamamoto has joined #openstack-infra | 22:37 | |
| *** jbernard has quit IRC | 22:38 | |
| *** jbernard has joined #openstack-infra | 22:38 | |
| *** claudiub|2 has joined #openstack-infra | 22:38 | |
| dmsimard | It's funny to see references to the Denver train every now and then. I think it's going to remain a classic. | 22:39 |
| mtreinish | ianw: fwiw, I don't think you need a license to install windows anymore. So you probably could download an iso install it into a vm and test things | 22:40 |
| *** jascott1 has quit IRC | 22:40 | |
| mtreinish | ianw: yeah looks like you can just download it: https://www.microsoft.com/en-us/software-download/windows10ISO | 22:41 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: Move TripleO jobs in-tree https://review.openstack.org/514778 | 22:41 |
| *** rcarrillocruz has joined #openstack-infra | 22:41 | |
| *** esberglu has quit IRC | 22:41 | |
| *** yamamoto has quit IRC | 22:42 | |
| *** jascott1 has joined #openstack-infra | 22:43 | |
| ianw | mtreinish: "We encountered a problem processing your request. Please try again later." ... can't even give it away :) | 22:43 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/openstack-zuul-jobs master: Move TripleO jobs/templates to openstack-infra/tripleo-ci https://review.openstack.org/514792 | 22:43 |
| *** felipemonteiro has quit IRC | 22:44 | |
| mtreinish | ianw: hmm, it seemed to work for me. It started to download an iso before I killed it | 22:45 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: Move TripleO jobs in-tree https://review.openstack.org/514778 | 22:45 |
| mtreinish | because I really didn't want to download the iso :) | 22:45 |
| *** jascott1 has quit IRC | 22:45 | |
| *** wolverineav has joined #openstack-infra | 22:45 | |
| ianw | mtreinish: ok, that "en-us" in the url gets automagically converted to "en-au" for me, which is then all broken. but if i go via google, it works | 22:46 |
| *** edmondsw has quit IRC | 22:46 | |
| *** jascott1 has joined #openstack-infra | 22:48 | |
| *** bobh has quit IRC | 22:50 | |
| *** priteau has quit IRC | 22:50 | |
| *** d0ugal has joined #openstack-infra | 22:50 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/openstack-zuul-jobs master: Stop using tox to build python tarballs https://review.openstack.org/513927 | 22:53 |
| openstackgerrit | Monty Taylor proposed openstack-infra/openstack-zuul-jobs master: Change override-branch to override-checkout https://review.openstack.org/520236 | 22:53 |
| *** jascott1 has quit IRC | 22:53 | |
| *** dizquierdo has joined #openstack-infra | 22:53 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Make build-python-release job https://review.openstack.org/513925 | 22:54 |
| openstackgerrit | Monty Taylor proposed openstack-infra/zuul-jobs master: Remove old python-sdist job https://review.openstack.org/513926 | 22:54 |
| *** mat128 has quit IRC | 22:54 | |
| *** jascott1 has joined #openstack-infra | 22:55 | |
| *** jascott1_ has joined #openstack-infra | 22:58 | |
| *** jascott1 has quit IRC | 22:59 | |
| *** jascott1_ has quit IRC | 23:00 | |
| *** jascott1 has joined #openstack-infra | 23:00 | |
| *** rwsu has quit IRC | 23:01 | |
| *** dizquierdo has quit IRC | 23:01 | |
| *** pahuang has quit IRC | 23:02 | |
| ianw | omg, i just about had a heart attack as "cortana" started speaking to me at full volume out of nowhere | 23:04 |
| *** jascott1 has quit IRC | 23:05 | |
| mordred | ianw, pabelanger: I'm seeing some jobs seem to hang for a LONG time - is this a known issue? | 23:05 |
| *** aeng has joined #openstack-infra | 23:06 | |
| mordred | (sorry, I'm still reconnecting all of my brain synapses so not 100% sure of current states) | 23:06 |
| mtreinish | ianw: lol | 23:07 |
| ianw | mordred: not that i know of, sorry. hang during the job or pre/post? | 23:07 |
| clarkb | mordred: is job in inyernap? | 23:07 |
| clarkb | we've seen what we think may be network connectivity issues there | 23:08 |
| clarkb | but havent really dug in yet and I'm covered in carrots feeding two year olds so not mich help | 23:08 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 23:09 |
| clarkb | taking care of two two year olds during the day is 90% food prep | 23:09 |
| mordred | clarkb: yes - it is. | 23:09 |
| mordred | clarkb: fwiw - it seems that there are two copies of the post playbook running | 23:10 |
| mordred | http://paste.openstack.org/show/626451/ <-- | 23:10 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: Move TripleO jobs in-tree https://review.openstack.org/514778 | 23:11 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/openstack-zuul-jobs master: Move TripleO jobs/templates to openstack-infra/tripleo-ci https://review.openstack.org/514792 | 23:11 |
| mordred | that seems like a single stack though - am I just reading that wrong? | 23:11 |
| mordred | ianw: during post ... | 23:11 |
| *** priteau has joined #openstack-infra | 23:13 | |
| ianw | ok, not to bore you with windows issues. install "windows s". recover from cortana yelling at me. figure out run prompt ... "cmd.exe" ... this version doesn't run unsigned apps, click here to read more ... windows store opens ... for just $199, upgrade now to windows pro | 23:14 |
| fungi | wow | 23:14 |
| mordred | that's ... | 23:14 |
| fungi | if i had known i could have refrained from wiping windows off the us$160 burner laptop i used for sydney | 23:15 |
| *** wolverineav has quit IRC | 23:16 | |
| *** priteau has quit IRC | 23:17 | |
| inc0 | mordred: got a second? | 23:18 |
| *** dave-mccowan has quit IRC | 23:19 | |
| mordred | inc0: sure! it'll hopefully help me avoid trying to figure out why override-checkout isn't working | 23:20 |
| inc0 | yeah, it will | 23:20 |
| inc0 | nothing to do with override-checkout | 23:21 |
| inc0 | but docker build pipeline | 23:21 |
| inc0 | so we already have periodic jobs to build and publish to dockerhub | 23:21 |
| inc0 | but on PTG we were dicsussing build -> test deploy ansible -> test deploy k8s -> publish to dockerhub | 23:21 |
| inc0 | that would require some sort of passing an artifact from one gate to another | 23:22 |
| jlvillal | pabelanger: I'm not sure why a new patch set 4 was proposed for https://review.openstack.org/#/c/520174/ | 23:22 |
| jlvillal | pabelanger: I wonder if there is a bug :( | 23:22 |
| inc0 | one idea would be to have multinode job where one node builds, 2 more deploy, when 2 more succeeds, first one publish | 23:23 |
| inc0 | but that's hacky as hell | 23:23 |
| jlvillal | ianw: https://www.windowscentral.com/microsoft-extends-free-windows-10-s-upgrade-offer-through-march-2018 | 23:23 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 23:23 |
| inc0 | if we could tarball images and send it over to next gate, would be awesome...but also bandwidth hungly | 23:23 |
| inc0 | hungry | 23:23 |
| inc0 | so unless we could schedule everything on same nodepool, which I understand is not possible, it's gonna be hard...even if we figure out how to even transfer this tarball between gates | 23:24 |
| inc0 | sooo...ideas? | 23:25 |
| mordred | inc0: well ... if we used a local dockerhub instead of a tarball, that should be a least somewhat better because of layers, right? | 23:25 |
| inc0 | local registry you mean, yeah | 23:26 |
| mordred | yah | 23:26 |
| inc0 | but won't solve problem of transfering stuff between nodepools | 23:26 |
| mordred | yah - it would still be bandwidth hungry for sure | 23:26 |
| inc0 | I'd rather create temporary registry vm, or keep build vm alive | 23:27 |
| inc0 | and point to it from deploy jobs | 23:27 |
| *** gouthamr has quit IRC | 23:28 | |
| inc0 | but that'd require ability to 1. orchestrate multiple jobs (dear zuul, don't start this job until this one reaches point X), and schedule on same nodepool | 23:28 |
| mordred | yah - so we don't have a good way to do that atm - the two main structural options are what you mentioned above - you can do a multi-node job - those will all be guaranteed to be int he same cloud-region, so passing things between them is easy enough | 23:28 |
| mordred | or can do multiple jobs with copying a tarball or using a registry - but with most-likely transits across regions | 23:29 |
| inc0 | right, I think multinode job would be best, even if ugly | 23:29 |
| mordred | yah - at least for now - it'll at least do the thing you're looking for | 23:30 |
| clarkb | why would it need to be multinode in that case? | 23:30 |
| clarkb | you could just serialize on a single node | 23:30 |
| inc0 | clarkb: well, we want build node and deploy nodes to be different | 23:30 |
| mordred | clarkb: they have more than one job that would consume the images | 23:30 |
| inc0 | right | 23:30 |
| clarkb | I see | 23:30 |
| *** masber has joined #openstack-infra | 23:30 | |
| inc0 | I guess quick win would be to do aio deploy with at least kolla-ansible | 23:30 |
| inc0 | better than no deploy at all | 23:31 |
| mordred | so rather than a build job then 3 deploy jobs, do one 4-node build+deploy job | 23:31 |
| inc0 | right build and deploy 2 different openstacks | 23:31 |
| mordred | yah | 23:31 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/project-config master: Move TripleO jobs in-tree https://review.openstack.org/514778 | 23:31 |
| *** wolverineav has joined #openstack-infra | 23:31 | |
| * inc0 sees the fun of making this happen ahead, it's one of these sad funs | 23:32 | |
| mordred | :) | 23:32 |
| mordred | sad funs are the best funs, no? | 23:32 |
| mtreinish | ianw: did you download a windows 10 pro iso? Last time I did this (which was a while ago) that's what I downloaded | 23:32 |
| *** esberglu has joined #openstack-infra | 23:32 | |
| inc0 | well I'll start by adding aio ansible deploy | 23:33 |
| mtreinish | I should also say that I also wasn't really that successful in getting a python env working when I tried it (this was to test stestr stuff) | 23:33 |
| *** rlandy|rover is now known as rlandy|rover|bbl | 23:33 | |
| mtreinish | which is why when I found the windows ci thing for github I just gave up trying to test in manually in a vm | 23:33 |
| inc0 | we already have this gate, so it shouldn't be too hard | 23:33 |
| *** pahuang has joined #openstack-infra | 23:34 | |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 23:34 |
| *** aeng has quit IRC | 23:35 | |
| *** wolverineav has quit IRC | 23:36 | |
| ianw | mtreinish: yeah, i can select pro in the install. my attention span on this is waning, if it gets any harder i give up | 23:36 |
| openstackgerrit | Emilien Macchi proposed openstack-infra/tripleo-ci master: Import TripleO legacy zuul jobs https://review.openstack.org/514793 | 23:36 |
| *** ianychoi has quit IRC | 23:37 | |
| *** yamamoto has joined #openstack-infra | 23:38 | |
| *** caphrim007_ has quit IRC | 23:39 | |
| *** caphrim007 has joined #openstack-infra | 23:39 | |
| *** caphrim007 has quit IRC | 23:39 | |
| *** caphrim007 has joined #openstack-infra | 23:41 | |
| *** andreww has quit IRC | 23:43 | |
| *** yamamoto has quit IRC | 23:44 | |
| *** edmondsw has joined #openstack-infra | 23:44 | |
| *** aeng has joined #openstack-infra | 23:52 | |
| *** stakeda has joined #openstack-infra | 23:54 | |
| *** HeOS has joined #openstack-infra | 23:54 | |
| *** rfolco has quit IRC | 23:55 | |
| *** edmondsw has quit IRC | 23:55 | |
| *** edmondsw has joined #openstack-infra | 23:57 | |
| *** dingyichen has joined #openstack-infra | 23:57 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
