15:00:15 <anteaya> #startmeeting third-party
15:00:16 <openstack> Meeting started Mon Apr 13 15:00:15 2015 UTC and is due to finish in 60 minutes.  The chair is anteaya. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:17 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:20 <openstack> The meeting name has been set to 'third_party'
15:00:23 <anteaya> hello
15:00:40 <anteaya> raise your hand if you are here for the third party meeting
15:00:51 <asselin> hi
15:00:56 <anteaya> hey asselin
15:01:00 <anteaya> #chair asselin
15:01:01 <openstack> Current chairs: anteaya asselin
15:01:24 <anteaya> asselin: I'm on a train, if my wifi gets knocked out can you continue and ensure we end on time?
15:01:53 <asselin> ok sure
15:01:56 <anteaya> thanks
15:02:10 <anteaya> while we see if anyone else is joing us today, how are you doing?
15:02:31 <asselin> I'm fine. It's monday :)
15:02:35 <anteaya> yay monday
15:03:04 <anteaya> what is the status of the tempest volume encryption situation?
15:03:11 <ameade> o/
15:03:16 <anteaya> hello ameade
15:03:25 <ameade> hey hey
15:03:32 <rhe00> hi
15:03:37 <anteaya> hi
15:03:47 <rhe00> hemna was working on that last week. it is quite broken
15:03:48 <asselin> anteaya, there are quite a few bugs out for volume encryption
15:03:56 <anteaya> hmmmm
15:04:02 <rhe00> I think only non multipath iscsi works
15:04:09 <anteaya> is thingee aware of the situation?
15:04:10 <asselin> we disabled all of them since they're broken.
15:04:14 <anteaya> :(
15:04:26 <asselin> the test passing doesn't mean anything anyways (false positive)
15:04:31 <anteaya> right
15:05:09 <asselin> I think there's 3-4 bugs open now.
15:05:17 <anteaya> anyone working on them?
15:05:57 <rhe00> hemna was working on it last week. not sure if that is his highest priority
15:06:02 <asselin> I saw Nha Pham phqnha pick one up...but hemnafk might be working on it too: https://bugs.launchpad.net/bugs/1439855
15:06:03 <openstack> Launchpad bug 1439855 in OpenStack Compute (nova) "encrypted iSCSI volume fails to attach, name too long" [Medium,Triaged] - Assigned to Nha Pham (phqnha)
15:06:26 <anteaya> are any of you able to help address any of the bugs in any way?
15:06:44 <rhe00> I am working on one in Cinder https://bugs.launchpad.net/cinder/+bug/1442302
15:06:45 <openstack> Launchpad bug 1442302 in Cinder "volume manager should set encrypted property in connection_info" [Undecided,In progress] - Assigned to Richard Hedlind (richard-hedlind)
15:06:49 <anteaya> help track down the exact source, for instance?
15:06:54 <anteaya> rhe00: yay
15:07:20 <rhe00> asselin: do you know if hemna is around?
15:07:30 <rhe00> maybe he could give us an update on his work
15:07:44 <anteaya> well that isn't really required
15:07:53 <asselin> rhe00, he should be in later. You can ping him in cinder channel.
15:07:53 <anteaya> as this is a third party meeting, not a cinder meeting
15:08:00 <rhe00> ok
15:08:09 <anteaya> mostly I just wanted to ensure cinder core is awaare
15:08:30 <anteaya> and that third party operators get involved in supporting a solution as much as possilbe
15:08:33 <anteaya> that is my goal
15:08:46 <asselin> anteaya, yes, there are formal bugs open in cinder, nova, & tempest.
15:08:47 <anteaya> so thank you, rhe00 for picking up that bug
15:08:48 <rhe00> asselin: has encrypted volumes test been disabled in tempest?
15:08:56 <anteaya> asselin: great
15:08:57 <asselin> rhe00, no
15:09:18 <anteaya> that would be thingee's decision I think
15:09:33 <anteaya> you can share your thoughts on that, but I would leave that to thingee to decide
15:09:44 <anteaya> so let's move on, shall we?
15:09:52 <asselin> sure
15:09:57 <rhe00> sure
15:10:04 <anteaya> does anyone have anything they would like to discuss about their system today?
15:10:15 <anteaya> thanks
15:10:27 <asselin> I found a workaround for the cinder "test_minimum_scenario" --> use neutron networking instead of nova-networking
15:10:43 <anteaya> asselin: do expand
15:11:30 <asselin> no idea why nova-networking fails. I was debugging it and for some reason the nova instance was not able to get an ip address, and hence the test failed
15:11:40 <anteaya> :(
15:11:58 <anteaya> do you feel it warrents a bug report?
15:12:03 <asselin> I debugged it a bit, and it's the path going from nova to the instance that's broken.
15:12:13 <wznoinsk> o/
15:12:22 <anteaya> hey wznoinsk
15:12:44 <asselin> tcpdump showed that an ip address was being assigned, but the packet couldn't make it's way in....
15:12:52 <anteaya> hmmmm
15:13:24 <anteaya> asselin: what action would you like to take on that?
15:13:42 <asselin> seems this is hitting certain ci systems, and not others....no idea why that would be the case.
15:14:00 <asselin> anteaya, none: I switched to neutron which is working.
15:14:11 <anteaya> hmmmm
15:14:28 <asselin> anteaya, anyway, seems that's the future solution, so all the better that one works.
15:14:35 <anteaya> true
15:14:43 <anteaya> what did you do to fix?
15:15:05 * asselin looks up the line
15:16:16 <asselin> #link http://git.openstack.org/cgit/openstack-infra/devstack-gate/tree/devstack-vm-gate.sh#n82
15:16:23 <asselin> export DEVSTACK_GATE_NEUTRON=1
15:16:53 <anteaya> where did you put that in your code?
15:17:19 <asselin> you can put that in your jenkins job
15:17:37 <anteaya> awesome
15:17:47 <anteaya> thank you for sharing that, asselin
15:17:52 <anteaya> any questions here?
15:18:02 <asselin> #link https://github.com/rasselin/os-ext-testing-data/blob/master/etc/jenkins_jobs/config/dsvm-cinder-driver.yaml.sample#L57
15:18:48 <anteaya> nice
15:19:17 <anteaya> any more for this topic?
15:19:42 <asselin> that's it, just wanted to share it with others in the same boat
15:19:53 <anteaya> thank you for doing so
15:20:12 <anteaya> not wanting to rush you, just no point in having dead air
15:20:24 <anteaya> anyone else with anything to discuss today?
15:20:31 <rhe00> I noticed on Friday that some CIs do not provide console logs in their results, which makes it really hard to know what tests they ran.
15:20:40 <anteaya> no kidding
15:20:49 <asselin> rhe00, ++
15:20:53 <ameade> i would just ping them individually
15:20:56 <anteaya> what project are those ci systems testing?
15:20:58 <rhe00> ok
15:21:06 <rhe00> Cinder
15:21:14 <rhe00> might test others as well, not sure
15:21:35 <anteaya> so I suggest you create a list of the systems you saw, and share the list with thingee
15:21:37 <asselin> rhe00, do you have a link?
15:21:47 <asselin> or what anteaya said
15:21:48 <rhe00> I was trying to chase down false positives, since I checked in an incomplete fix and pretty much all CIs should have failed.
15:21:50 <anteaya> they need to have console logs
15:22:00 <anteaya> rhe00: good test
15:22:07 <anteaya> rhe00: what did you discover?
15:22:11 <rhe00> asselin: one second, I'll find it
15:23:20 <rhe00> This is the link to my check in. https://review.openstack.org/#/c/172531/
15:23:36 <anteaya> #link https://review.openstack.org/#/c/172531/
15:24:02 <rhe00> the VMWare NSX was the one without console logs. Now it is not accesible at all.
15:24:35 <anteaya> rhe00: please bring this to thingee's attention
15:24:46 <anteaya> rhe00: thank you for sharing your findings
15:24:56 <rhe00> anteaya: I will
15:25:08 <anteaya> the ci operators are the best source of ensuring quality in the ci systems of a project
15:25:32 <anteaya> poor reliablity means that all cis for the project are painted with the same brush
15:26:05 <anteaya> by helping operators get good quality in their ci systems, all ci systems for that project are seen as more reliable
15:26:09 <anteaya> so thank you
15:26:44 <asselin> I still have this issue: there are still failing in my setup: test_cinder_volume_create_delete, test_cinder_volume_create_delete_retain
15:26:47 <rhe00> anteaya: I will follow up on this and hopefully help getting the situation improved
15:26:50 <asselin> anyone else?
15:26:52 <krtaylor> rhe00, try the monitoring dashboard that patrickeast set up, it will show comparisons for all cinder systems
15:27:00 <anteaya> rhe00: thank you
15:27:12 <anteaya> asselin: nice job, carry on
15:27:18 <asselin> related to heat
15:27:18 <rhe00> krtaylor: do you have the link for that? I used to have it, but not sure where I have it
15:27:22 <krtaylor> hm, which seems to be down at the moment   http://ec2-54-67-102-119.us-west-1.compute.amazonaws.com:5000
15:27:23 * anteaya reorganized the space
15:27:36 <krtaylor> rhe00, ^^^
15:28:15 <rhe00> krtaylor: got it. anteaya: carry on
15:28:28 <asselin> #link https://github.com/patrick-east/scoreboard
15:28:48 <asselin> rhe00, ^^ I fyou want to run your own
15:29:05 <rhe00> asselin: I might try that. thanks
15:29:11 <krtaylor> asselin, good point, we are internally too
15:29:20 <krtaylor> very nice simple tool
15:30:04 <ameade> nice
15:31:03 <anteaya> asselin: did you want to ask your question again?
15:31:16 <asselin> I still have this issue: there are still failing in my setup: test_cinder_volume_create_delete, test_cinder_volume_create_delete_retain
15:32:05 <asselin> (it would be great to have a scoreboard showing what tests each ci is running)
15:32:21 <anteaya> it would indeed
15:32:52 <asselin> those are related to heat. tests pass when run manually, but not as part of the ci job....?
15:33:02 <anteaya> how interesting
15:33:46 <anteaya> well so far it doesn't sound like anyone present is able to confirm your findings
15:33:50 <asselin> but I don't have stack traces available now.
15:33:58 <asselin> yeah, guess it's just me.
15:33:59 <krtaylor> asselin, is it related to the caching of all the heat images (regardless of whether they will be used or not)
15:34:42 <asselin> krtaylor, not sure...it seemed to be an issue with the heat client not able to access the heat service....??
15:35:33 <krtaylor> asselin, hm, that doesnt sound the same
15:35:42 <asselin> anyway we can move on....when I get stack traces I can ask again.
15:35:52 <anteaya> okay sounds good
15:36:06 <anteaya> does anyone have anything else they would like to discuss today?
15:36:20 <krtaylor> asselin, the problem we were seeing was failure when it tried to download the fedora cloud image -> https://github.com/openstack-dev/devstack/commit/89983b6dfe15e8e83f390e9870cc3ddfbf2b8243
15:37:01 <krtaylor> asselin, not sure if you are pinned to an old version maybe
15:37:20 <asselin> krtaylor, no i'm on master now....
15:37:38 <krtaylor> asselin, must be a different problem then
15:38:06 <asselin> krtaylor, yes, thanks anyway
15:38:50 <anteaya> any other topics needing to be discussed today?
15:39:35 <anteaya> if the answer is no, feel free to say no, we can have 20 minutes of our life back
15:39:50 <asselin> not from me
15:39:56 <anteaya> thanks asselin
15:40:01 <anteaya> anyone else?
15:40:06 <krtaylor> none for me, thanks anteaya
15:40:12 <rhe00> none here
15:40:15 <anteaya> okay
15:40:21 <anteaya> let's round it out then
15:40:27 <anteaya> asselin: would you do the honours?
15:40:38 <asselin> sure:
15:40:42 <asselin> #endmeeting