14:02:02 <SergeyLukjanov> #startmeeting sahara
14:02:03 <openstack> Meeting started Thu Dec  4 14:02:02 2014 UTC and is due to finish in 60 minutes.  The chair is SergeyLukjanov. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:02:04 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:02:04 <NikitaKonovalov> o/
14:02:07 <openstack> The meeting name has been set to 'sahara'
14:02:07 <huichun> hello
14:02:28 <kchen> o/
14:03:15 <SergeyLukjanov> zhidong, huichun, kchen, it's a better time for you folks, yeah?
14:03:45 <zhidong> SergeyLukjanov: yes, it's 10pm here. a little bit late. but it works.
14:03:56 <kchen> fine for me
14:03:57 <SergeyLukjanov> zhidong, okay
14:04:05 <SergeyLukjanov> zhidong, the alt time is 10pm in RU ;)
14:04:07 <weiting> Yes, it is good from us
14:04:07 <SergeyLukjanov> #link https://wiki.openstack.org/wiki/Meetings/SaharaAgenda#Agenda
14:04:31 <SergeyLukjanov> #topic sahara@horizon status (croberts, NikitaKonovalov)
14:04:53 <NikitaKonovalov> we have several patches on review
14:05:00 <crobertsrh> Query filtering patch is up for review, among others
14:05:33 <SergeyLukjanov> crobertsrh, yay!
14:05:39 <NikitaKonovalov> alazarev has also published a few improvements
14:05:39 <SergeyLukjanov> crobertsrh, do you have link?
14:05:55 <crobertsrh> #link https://review.openstack.org/#/c/136459/
14:06:01 <SergeyLukjanov> it's 6am for alazarev, so, most probably he's sleeping
14:06:51 <SergeyLukjanov> okay
14:06:54 <SergeyLukjanov> yhx
14:06:55 <SergeyLukjanov> thx*
14:06:56 <SergeyLukjanov> #topic News / updates
14:06:59 <SergeyLukjanov> folks, please
14:07:33 <SergeyLukjanov> zhidong, huichun, kchen, weiting, folks, could you please introduce yourself?
14:07:39 <zhidong> sure
14:07:46 <elmiko> i've been doing a bunch of reviews, and i'm working on the security guideline spec. also a little work on the swagger impl.
14:08:24 <sreshetnyak> i'm working on integration test for hive edp job
14:08:28 <zhidong> we are from Intel Software and Services Group, working on Big Data, in particular in the area of bigdata/cloud intersection.
14:08:43 <zhidong> we are all based on Shanghai China
14:08:46 <crobertsrh> Lots of reviews for me.  Other work items will be improvements to data_sources as args and possibly editing templates.
14:09:37 <tmckay> I've been working on UI filtering and making a way to use data_sources as general arguments
14:10:42 <SergeyLukjanov> btw, https://review.openstack.org/#/c/138842/ client bump approved, thx to mordred
14:10:42 <zhidong> currently we are wroking on CDH plugin, but not limited to this
14:11:33 <SergeyLukjanov> zhidong, cool, thx
14:12:08 <oikawa> I'm working on improving mapreduce compatibility and diskimage-builder with swiftfs build script.
14:13:53 <aignatov> oikawa: that’s swesome
14:14:07 <SergeyLukjanov> folks, please, don't forget to review specs
14:14:07 <mattf> oikawa, would you consider helping w/ the root of the problem, that the swiftfs code is semi-fork/orphaned from upstream hadoop and doesn't have its own release schedule ?
14:14:19 <SergeyLukjanov> especially core reviewers :)
14:15:06 <tmckay> nice to meet you folks, I have to change locations.
14:15:43 <SergeyLukjanov> #topic Kilo release schedule
14:15:49 <SergeyLukjanov> #link https://wiki.openstack.org/wiki/Kilo_Release_Schedule
14:15:58 <SergeyLukjanov> so, we have a Kilo-1 release at Dec 18
14:16:13 <SergeyLukjanov> so, don't forget about few days freeze before the dev milestones
14:16:21 <SergeyLukjanov> to verify and tag some working stuff ;)
14:16:47 <SergeyLukjanov> the kilo-2 is Feb 5 and kilo-3 is Mar 19
14:17:02 <SergeyLukjanov> don't forget that now we're fully following all freezes
14:17:27 <SergeyLukjanov> and there is a FeatureProposalFreeze (FPF) at Mar 15
14:17:35 <SergeyLukjanov> at Mar 5* sorry
14:17:50 <SergeyLukjanov> and FeatureFreeze, StringFreeze, DepFreeze at Mar 19 with kilo-3
14:18:11 * SergeyLukjanov already want to go the next summit
14:18:25 <elmiko> lol
14:18:29 <SergeyLukjanov> any questions on release schedule?
14:19:58 <mattf> nope
14:20:32 <SergeyLukjanov> ok
14:20:46 <SergeyLukjanov> #topic Bug / doc / spec days
14:21:21 <SergeyLukjanov> so, it looks like we need to have a bug triage day (many not triaged bugs or not assigned and etc.)
14:21:32 <SergeyLukjanov> doc needs some cleanups (as always)
14:21:56 <SergeyLukjanov> specs are mostly healthy now - https://review.openstack.org/#/q/sahara-specs+AND+status:open,n,z
14:22:01 <SergeyLukjanov> thoughts?
14:22:19 <elmiko> i thought we had talked about having a general doc cleanup as a task for kilo?
14:23:25 <SergeyLukjanov> elmiko, I think we've been talking a bit about it
14:24:04 <SergeyLukjanov> what's about scheduling bug triage after kilo-1?
14:24:17 <SergeyLukjanov> and next week after bug triage doc cleanup?
14:24:21 <elmiko> SergeyLukjanov: ok, so maybe make a bp for doc cleanup and then we can break down the docs into individual reviews?
14:24:35 <SergeyLukjanov> elmiko, yeah, that's a good idea
14:24:39 <oikawa> mattf: Yes. But hadoop release cycle isn't sahara release cycle. If binary uploaded at sahara-extra's github release page, CI doesn't use swiftfs new features. And the patch that depends swiftfs's new feature doesn't pass CI tests.
14:24:43 <mattf> SergeyLukjanov, rht folks are on vacation most of that week and until the 3rd, iirc
14:25:12 <SergeyLukjanov> mattf, oh, I forget about it
14:25:32 <SergeyLukjanov> mattf, and RU folks will be on vacation Jan 1-12 approximately
14:25:34 <mattf> oikawa, yeah, i think we need to put a process in place for how we build&distro our version of the swiftfs
14:25:54 <mattf> SergeyLukjanov, sooo, we have about 4 weeks of low attendance!
14:26:00 <SergeyLukjanov> mattf, yeah
14:26:28 <SergeyLukjanov> sounds like we could have one of this events before the kilo-1 and one after the month of NY vacations
14:26:54 <mattf> 22 dec is likely the last day we'll be doing much work until 5 jan
14:27:44 <mattf> oikawa, would you have run into your issue of a stale swiftfs if we made sure that it was rebuilt for each minor release, e.g. kilo-1 should build it ?
14:29:11 <mattf> SergeyLukjanov, is it a manual process right now to build and upload swiftfs to the mirantis file server?
14:29:40 <SergeyLukjanov> mattf, yeah
14:30:05 <SergeyLukjanov> I'm thinking about fast tracking the auto build and publishing it to the tarballs.o.o
14:30:13 <SergeyLukjanov> at least "latest version from master"
14:30:52 <mattf> fyi folks - https://review.openstack.org/#/c/121003/ - a longstanding ceilometer bug was squashed
14:31:14 <mattf> SergeyLukjanov, that'd be very nice actually
14:31:32 <mattf> oikawa, would having an autobuild served from tarballs.o.o address the root of your problem?
14:32:09 <mattf> oh, result of that ceilometer bug being fixed means we don't have to be careful w/ data field names anymore in sahara
14:32:18 <elmiko> nice
14:32:20 <oikawa> mattf, SergeyLukjanov: Yes !
14:32:27 <crobertsrh> sorry...my connection died for a few min
14:32:29 <SergeyLukjanov> mattf, yay re https://review.openstack.org/#/c/121003/
14:33:07 <SergeyLukjanov> mattf, oikawa, okay, I'll make a job for doing it asap
14:33:14 <SergeyLukjanov> asap == ~1 week I hope
14:33:16 * mattf doesn't mention that it took 13 months
14:33:38 <mattf> SergeyLukjanov, +1
14:33:53 <oikawa> SergeyLukjanov: thanks !
14:34:00 <SergeyLukjanov> I have a random question
14:34:11 <mattf> oikawa, can we abandon your CR?
14:34:29 <SergeyLukjanov> what's about removing word Hadoop from the "Hadoop version" column names in Horizon?
14:34:31 <mattf> ...maybe right after the tarball shows up on tarballs.o.o?
14:34:49 <SergeyLukjanov> it's really destructing folks how see Spark plugin with hadoop version
14:34:56 <mattf> or storm
14:35:03 <SergeyLukjanov> exactly
14:35:08 <crobertsrh> Yes....I had a bug filed awhile ago to get rid of "hadoop" in inappropriate places
14:35:14 <crobertsrh> I think that was UI only though
14:35:23 <mattf> i'd be happy removing it from the param names too
14:35:25 <SergeyLukjanov> we probably could add an alias in API to the hadoop_version fields
14:35:27 <oikawa> mattf: yes
14:35:30 <crobertsrh> somehow, column names must have been missed
14:35:48 <mattf> oikawa, great
14:36:11 <SergeyLukjanov> I think that we need to think about removing hadoop from names and other places
14:36:13 <SergeyLukjanov> offline
14:36:20 <SergeyLukjanov> probably to propose some spec
14:36:23 * mattf nods
14:36:44 <SergeyLukjanov> let's move on now and if we'll have time let's continue it on open discussion time
14:36:47 <SergeyLukjanov> #topic Update meeting time to make Asia folks able to attend meetings
14:37:03 <SergeyLukjanov> I think the current time (16:00 UTC) works ok as alternate time
14:37:09 <mattf> this time is accidentally very good for me
14:37:10 * mattf smiles
14:37:30 <SergeyLukjanov> and if there are no objections let's approve it as an alt time
14:37:38 <NikitaKonovalov> works for me
14:37:40 <SergeyLukjanov> mattf, :)
14:37:41 <mattf> 16UTC or 14UTC?
14:37:50 <SergeyLukjanov> oops, I mean 1400 UTC
14:38:23 <oikawa> this time is ok for me (23:00 at japan)
14:38:43 <zhidong> good to know we have a friend in Asia :)
14:41:00 <SergeyLukjanov> :)
14:41:16 <SergeyLukjanov> #agreed 14:00 UTC approved as alt time for meetings
14:41:24 <SergeyLukjanov> #topic CDH plugin integration test in Mirantis CI (zhidong)
14:41:27 <SergeyLukjanov> zhidong, please
14:42:16 <zhidong> currently the cdh plugin integration test case is non-voting
14:42:32 <zhidong> may I know why? and can we change it to voting?
14:42:56 <mattf> as an aside - ideally cloudera would be running that CI
14:43:17 <SergeyLukjanov> mattf, ideally yup
14:43:19 <mattf> my understanding is it wasn't stable until recently. SergeyLukjanov knows for sure
14:43:22 <zhidong> yes I understand it. I will convey this message to them.
14:43:31 <zhidong> but I guess it won't happen in short time
14:43:45 <SergeyLukjanov> mattf, but in the real worlds it's very difficult to make a CI and it's not because of software
14:43:56 <mattf> also, the cloudera plugin isn't enabled by default
14:44:07 <SergeyLukjanov> right now I think that CDH plugin is stable enough to enable it as voting
14:44:14 <mattf> SergeyLukjanov, /me nods
14:44:30 <zhidong> SergeyLukjanov, that would be great.
14:44:35 <SergeyLukjanov> sreshetnyak, any comments on CDH stability?
14:45:48 <kchen> btw, these two days I found CI tests became very slow. Is it expected?
14:46:08 <SergeyLukjanov> kchen, there is a bunch of external issues
14:46:18 <sreshetnyak> CDH plugin stable in few month
14:46:21 <SergeyLukjanov> kchen, it makes CI failing for a few times
14:47:05 <SergeyLukjanov> sreshetnyak, I think you means that it's already stable for a few month ;)
14:47:07 <kchen> ok I see
14:47:34 <sreshetnyak> SergeyLukjanov, yes
14:47:42 <SergeyLukjanov> kchen, anyway we're working on making it better ;)
14:48:03 <SergeyLukjanov> #agreed make CDH jobs voting
14:48:06 <SergeyLukjanov> zhidong, anything else on the topic?
14:48:13 <zhidong> no. thanks
14:48:33 <SergeyLukjanov> #topic Reference image hosting (sahara-files.mirantis.com?) (zhidong)
14:48:34 <mattf> https://sahara.mirantis.com/jenkins/view/sahara/job/gate-sahara-integration-cdh-centos/
14:48:39 <mattf> https://sahara.mirantis.com/jenkins/view/sahara/job/gate-sahara-integration-cdh-ubuntu/
14:48:40 <SergeyLukjanov> zhidong, one more topic for you :)
14:49:04 <zhidong> currently the reference images for vanilla and HDP plugins are hosted on sahara-files.mirantis.com
14:49:20 <SergeyLukjanov> mattf, looks the same as other plugins
14:49:27 <zhidong> I am wondering can we upload CDH reference images to mirantis website too?
14:49:28 <mattf> SergeyLukjanov, aye, it does
14:49:39 <SergeyLukjanov> zhidong, yup, for sure
14:49:44 <zhidong> how?
14:49:51 <SergeyLukjanov> zhidong, contact sreshetnyak
14:49:57 <zhidong> get it.
14:50:15 <SergeyLukjanov> btw I'm working on publishing latest built images on tarballs.openstack.org
14:50:31 <mattf> zhidong, does the cdh license allow for 3rd party distribution?
14:50:34 <SergeyLukjanov> but there are some issues with timeouts, internet speed and random bugs
14:50:54 <zhidong> hmm... I need to check it
14:51:02 <SergeyLukjanov> mattf, good point
14:51:04 <mattf> SergeyLukjanov, that's good too! are those images all auto-built from DIB?
14:51:19 <zhidong> I think it is not really a 'distribution'
14:51:27 <mattf> iirc, they're from DIB, but the base centos image is hand built
14:51:29 <zhidong> anyway I will double check it.
14:51:42 <SergeyLukjanov> mattf, yeah, I'm planning to just make a post-commit job to build and publish all images to tarballs.o.o
14:52:13 <mattf> SergeyLukjanov, that sounds good
14:52:24 <SergeyLukjanov> mattf, we already have a job in sahara-image-elements that is building vanilla images
14:52:31 <SergeyLukjanov> mattf, but there are some issues with it
14:52:41 <SergeyLukjanov> mattf, no time to debug atm
14:52:46 * mattf nods
14:53:05 <SergeyLukjanov> if someone could digg into the logs
14:53:06 <SergeyLukjanov> http://logs.openstack.org/76/138976/1/check/check-sahara-buildimages-vanilla/cb7122a/
14:54:26 <SergeyLukjanov> everyone trying to scroll down faster than logs loading? :)
14:54:44 <zhidong> SergeyLukjanov: yes
14:55:02 <zhidong> SergeyLukjanov: my browser just hangs
14:55:25 <SergeyLukjanov> yeah...
14:55:38 <mattf> Error: Cannot retrieve metalink for repository: epel. Please verify its path and try again
14:56:04 <SergeyLukjanov> the question is why it's not working on the gate ;)
14:56:15 <SergeyLukjanov> so, let's move on
14:56:42 <SergeyLukjanov> #action zhidong to check if it's ok to publish cdh images to sahara-files (license)
14:56:51 <zhidong> sure
14:56:59 <mattf> interesting, we're using 2 different urls to get epel
14:57:06 <SergeyLukjanov> any other topics to chat about?
14:57:41 <zhidong> nope.
14:57:46 <mattf> nada
14:57:47 <SergeyLukjanov> #topic Open discussion
14:57:48 <elmiko> nothing from me
14:58:10 <SergeyLukjanov> 2 mins left
14:58:38 <SergeyLukjanov> folks, please, make sure that alazarev will not try to start meeting at 1800 UTC :)
14:59:16 <elmiko> noted ;)
14:59:26 <tmckay> SergeyLukjanov, if we have the meeting at 1800 we can just read the minutes and discuss them :)
14:59:50 <SergeyLukjanov> :)
14:59:54 <SergeyLukjanov> thank you folks!
14:59:57 <mattf> ok, ciao folks
14:59:58 <SergeyLukjanov> #endmeeting