Tuesday, 2018-11-06

edmondsw#startmeeting PowerVM Driver Meeting15:01
openstackMeeting started Tue Nov  6 15:01:55 2018 UTC and is due to finish in 60 minutes.  The chair is edmondsw. Information about MeetBot at http://wiki.debian.org/MeetBot.15:01
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.15:01
*** openstack changes topic to " (Meeting topic: PowerVM Driver Meeting)"15:01
openstackThe meeting name has been set to 'powervm_driver_meeting'15:02
edmondsw#link agenda https://etherpad.openstack.org/p/powervm_driver_meeting_agenda15:02
edmondswefried mdrabe ping15:02
edmondswmay be a short meeting if nobody's around :)15:05
efriedo/ sorry15:06
edmondswnp15:06
edmondsw#topic In-Tree Driver15:06
*** openstack changes topic to "In-Tree Driver (Meeting topic: PowerVM Driver Meeting)"15:06
edmondswwe merged a couple things this week:15:06
edmondsw1) https://review.openstack.org/#/c/610174/15:06
edmondsw2) https://review.openstack.org/#/c/468560/15:06
edmondswI don't recall anything else to discuss IT... anyone else?15:07
edmondswalright, moving on15:08
edmondsw#topic Out-of-Tree Driver15:08
*** openstack changes topic to "Out-of-Tree Driver (Meeting topic: PowerVM Driver Meeting)"15:08
edmondswwe merged the corresponding tempfile change OOT similar to IT: https://review.openstack.org/#/c/613342/15:08
edmondswand https://review.openstack.org/#/c/614781/15:09
edmondswoh, and there was a corresponding IT change for that, which I should have mentioned above : https://review.openstack.org/#/c/614643/15:10
edmondswthe only other thing I have for the OOT driver is that there's been a customer issue with NovaLink GPFS volumes since we switched to the LOOP backstore15:11
edmondswfrom what I hear, seems to be a kernel issue, but the NovaLink guys are working on a workaround15:12
edmondswif they don't get that resolved, we may have to revert the change that we put in a while back to switch to loop: b86a9480c3c39f357b862c6a57ca6a2eb092558c15:12
edmondswanything else for OOT discussion?15:13
edmondswso quiet today15:14
edmondsw#topic Device Passthrough15:14
*** openstack changes topic to "Device Passthrough (Meeting topic: PowerVM Driver Meeting)"15:14
edmondswefried ^15:14
efriedNothing to report. I still owe updates to the specs and code.15:14
edmondswack15:15
edmondsw#topic PowerVM CI15:15
*** openstack changes topic to "PowerVM CI (Meeting topic: PowerVM Driver Meeting)"15:15
edmondswwe found the CI not voting again late last week and mmedvede helped fix it15:15
edmondswI believe it was a nodepool issue, same as last time. Not sure what's causing that15:16
edmondswmujahid doesn't seem to be around to give us his thoughts15:16
edmondswhttp://ci-watch.tintri.com/ doesn't seem to be running anymore, so mmedvede setup http://ciwatch.mmedvede.net15:17
edmondswwe appear to be voting there, though I'm seeing more red than I'd like15:18
edmondswI'll ask mujahid to take a look at that15:18
edmondsw#topic Open Discussion15:18
*** openstack changes topic to "Open Discussion (Meeting topic: PowerVM Driver Meeting)"15:18
edmondswefried and I will be at the summit in Berlin next week, so no meeting15:19
edmondswfloor is open for any other topics15:19
efriedFYI I've been working on perf improvements in the resource tracker. It's nothing powervm specific, but it's what's consuming my time the past couple of work days.15:20
edmondswack15:20
mmedvedewith regard to nodepool problem I helped to fix - it may happen again I am afraid. nodepool is stuck with too many open files error. I did not change ulimits15:21
edmondswmmedvede any idea what causes that?15:21
edmondswI'm with you, expecting this to keep happening unless we figure out what's causing it15:21
edmondswor put in some mitigation or something15:22
mmedvedeedmondsw: no, other than it is somehow not closing files15:22
efriededmondsw: We fixed a file descriptor leak recently...15:22
efriedoh, it was in tempfile usage. And it never made it to prod.15:22
mmedvedesimple mitigation would be periodic restart of nodepool every day15:22
edmondswmmedvede would we lose things with the restart?15:23
mmedvedenodepool keeps its state in database. Still slight possibility something would be lost, but it is usually safe to restart15:23
edmondswgood to know15:24
edmondswmaybe we can setup a cron job for the middle of the night or something15:24
edmondswI'll suggest that to mujahid15:24
edmondswI remembered one other thing I should mention... and mmedvede you'll also be interested in this15:24
edmondswa while back we hit an issue with greenlet 0.4.14 where they'd made a change that didn't work on Ubuntu 16.0415:25
edmondswthey've since fixed that in greenlet with a different change in 0.4.15, which I believe I already noted previously15:25
edmondswbut I was also working with Ubuntu to fix the gcc bug that was the real issue15:25
edmondswand I have now successfully tested a new gcc5 package on xenial-proposed with greenlet 0.4.1415:26
edmondswso once they promote that from proposed (which they've been waiting to do until I'd confirmed the fix), even 0.4.14 should be ok15:26
edmondsweom15:27
mmedvedethis is good news, thank you15:27
edmondswyep... shouldn't have to worry about other things besides greenlet breaking on Ubuntu 16.04 due to that gcc bug15:28
edmondswalright, I think that's it for today15:29
edmondsw#endmeeting15:29
*** openstack changes topic to "This channel is for PowerVM-related development and discussion. For general OpenStack support, please use #openstack."15:29
openstackMeeting ended Tue Nov  6 15:29:04 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:29
openstackMinutes:        http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2018/powervm_driver_meeting.2018-11-06-15.01.html15:29
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2018/powervm_driver_meeting.2018-11-06-15.01.txt15:29
openstackLog:            http://eavesdrop.openstack.org/meetings/powervm_driver_meeting/2018/powervm_driver_meeting.2018-11-06-15.01.log.html15:29

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!