18:00:07 <tellesnobrega> #startmeeting sahara
18:00:11 <openstack> Meeting started Thu Jul 20 18:00:07 2017 UTC and is due to finish in 60 minutes.  The chair is tellesnobrega. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:12 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:00:15 <openstack> The meeting name has been set to 'sahara'
18:01:55 <jeremyfreudberg> o/
18:03:11 <tellesnobrega> waiting a couple more minutes
18:05:19 <tellesnobrega> #topic News/Updates
18:05:50 <jeremyfreudberg> right
18:05:55 <tellesnobrega> I'm working on ambari images now, and fixing some environment issues, other than that I'm dealing with some downstream stuff
18:06:40 <jeremyfreudberg> cool, i think you've reviewed all the patches i have up right now, those two SIE things, plus the dashboard thing. Other than that I will have another SIE patch very soon for adding the S3 hadoop jar
18:06:54 <tellesnobrega> awesome
18:06:58 <tellesnobrega> how is that integration going?
18:07:32 <jeremyfreudberg> uh, i'd feel more comfortable with landing everything else (jb, data store, client work, dashboard work) in queens
18:08:05 <jeremyfreudberg> but adding it to the images, and providing some reminder in doc about how to run job against s3 manually would be nice
18:08:16 <tellesnobrega> we can start merging small stuff now
18:08:29 <jeremyfreudberg> yep
18:08:45 <tellesnobrega> aiming to have a bigger window in queens
18:09:01 <tellesnobrega> please keep me up to date on that
18:09:04 <jeremyfreudberg> btw, will add the s3 jar to the new-style image gen as well, once you merge that
18:09:04 <jeremyfreudberg> yep
18:09:13 <jeremyfreudberg> will keep you up to date on everything
18:09:13 <tellesnobrega> cool
18:09:41 <tellesnobrega> thanks
18:09:50 <tellesnobrega> this is very important
18:10:42 <jeremyfreudberg> that it's it for the "news" topic I think
18:10:49 <tellesnobrega> cool
18:10:51 <tellesnobrega> lets move on
18:11:05 <tellesnobrega> #topic Open Discussion
18:11:13 <tellesnobrega> will skip PTG this time, there isn't updates on that
18:11:20 <jeremyfreudberg> yep
18:11:53 <jeremyfreudberg> first thing is, do we want to try to get cdh plugin upgrade done?
18:12:33 <jeremyfreudberg> shuyingya was working on 5.10 back in april: https://review.openstack.org/#/c/456546/ but now we are on to 5.12
18:12:58 <tellesnobrega> hmm, it was part of the plan yes, shuyingya was supposed to make that happen
18:13:40 <tellesnobrega> 5.12?
18:13:51 <jeremyfreudberg> 5.12 came out in june I think
18:14:08 <tellesnobrega> maybe we should try to merge 5.10 for pike and we upgrade to 5.12 in queens
18:14:16 <tellesnobrega> what do you think?
18:14:25 <tellesnobrega> it is very late to start a new patch now
18:14:27 <jeremyfreudberg> at this point, it's the same amount of work to get 5.10 ready vs 5.12 ready
18:15:03 <tellesnobrega> even considering shuyingya's patch?
18:15:21 <jeremyfreudberg> ok, it's slightly less work to 5.10 because the patch is there
18:15:30 <jeremyfreudberg> but it is mostly just changing some numbers around for 5.12
18:15:47 <jeremyfreudberg> not to mention we still have to do sahara-side change for either version
18:16:05 <tellesnobrega> ok
18:16:17 <tellesnobrega> it makes sense to do to 12
18:16:30 <jeremyfreudberg> hi esikachev
18:16:33 <esikachev> hi
18:16:35 <esikachev> sorry
18:16:43 <tellesnobrega> hey esikachev, no worries
18:17:04 <tellesnobrega> jeremyfreudberg, are you saying that you are willing to to do the work? /me hopes so
18:17:27 <tellesnobrega> I mean, do you have the time?
18:17:31 <jeremyfreudberg> tellesnobrega, I will try. and ping you if I end up too busy
18:17:44 <tellesnobrega> jeremyfreudberg, thanks, let me know, I can help out for sure
18:18:22 <jeremyfreudberg> luckily it easy enough -- shuyingya did some refactoring to cut down on the work
18:18:29 <jeremyfreudberg> well, not too easy, but easier
18:18:56 <tellesnobrega> cool
18:18:58 <jeremyfreudberg> that's all I have to say about that topic, we can move on to something else
18:19:10 <tellesnobrega> yeah, we have to formally thank him for the refactoring
18:19:15 <tellesnobrega> cool
18:19:29 <jeremyfreudberg> i have to step out for 1 minute, I will be back asap
18:19:33 <jeremyfreudberg> my boss is being my boss
18:19:36 <tellesnobrega> esikachev, do you have any topics in mind to discuss?
18:19:39 <esikachev> tellesnobrega: do you have any updates about hosts for sahara-ci?
18:19:41 <esikachev> yes)
18:19:45 <tellesnobrega> jeremyfreudberg, no worries
18:20:24 <tellesnobrega> esikachev, so about that, we don't have any resources yet
18:20:37 <esikachev> it's sad :(
18:20:47 <tellesnobrega> it is
18:21:14 <tellesnobrega> tosky is looking into adding some tests into the infra tests
18:21:32 <tellesnobrega> at least for vanilla
18:21:32 <esikachev> ok
18:21:47 <jeremyfreudberg> tellesnobrega, I am back
18:21:48 <tellesnobrega> in the mean time we are looking for resources so we can have a full CI
18:21:48 <esikachev> then I have not any topics
18:22:39 <tellesnobrega> cool
18:22:56 <esikachev> also, I don't have an answer from tsp
18:23:29 <tellesnobrega> that is also sad, let me know when you get some help
18:23:34 <tellesnobrega> some info
18:23:38 <esikachev> ok
18:24:21 <jeremyfreudberg> tellesnobrega, I can see if my team https://massopen.cloud might have some resources, but really no promises
18:24:23 <jeremyfreudberg> for ci i mean
18:24:39 <tellesnobrega> jeremyfreudberg, that would be great
18:24:53 <tellesnobrega> I don't take as a promise :)
18:25:52 <esikachev> we need around 64gb
18:25:58 <esikachev> it's min
18:26:29 <tellesnobrega> esikachev, how many machines would be ideal for the CI?
18:26:40 <jeremyfreudberg> 64gb ram per machine?
18:26:45 <esikachev> which machines?
18:26:45 <jeremyfreudberg> just so I can understand
18:26:48 <tellesnobrega> or just a very big 64gb machine?
18:26:57 <esikachev> it's for devstack
18:27:13 <esikachev> because we have a huge cdh clusters
18:27:25 <tellesnobrega> it is huge :(
18:27:30 <esikachev> "huge"
18:27:32 <esikachev> :)
18:27:34 <jeremyfreudberg> yep
18:28:03 <esikachev> i will think about restructurisation of mechanism of ci
18:28:20 <esikachev> maybe, we can use less resources
18:28:28 <tellesnobrega> that would be cool, if you could present us the desired architecture
18:28:42 <esikachev> i will try
18:28:48 <jeremyfreudberg> one more thing, on the topic of testing
18:29:01 <jeremyfreudberg> we still need to manually check ironic integration still works
18:29:04 <jeremyfreudberg> before pike relase
18:29:19 <tellesnobrega> jeremyfreudberg, true, i had forgot about that
18:29:28 <tellesnobrega> do you have plans on that?
18:29:56 <jeremyfreudberg> i don't really have plans
18:30:38 <jeremyfreudberg> certainly after FF
18:31:00 <esikachev> tellesnobrega: btw, if you will have a tasks for development in sahara, i can help
18:31:21 <tellesnobrega> jeremyfreudberg, yes. I will take a look into how to deploy fake ironic instances and see if I can test it
18:31:36 <jeremyfreudberg> tellesnobrega, yep, fake ironic was my thought too
18:31:45 <jeremyfreudberg> esikachev, any task?
18:31:51 <tellesnobrega> we can sync and try to get this going fast
18:32:03 <tellesnobrega> jeremyfreudberg, did you think cdh upgrade as well?
18:32:24 <jeremyfreudberg> you mean for esikachev to help with that?
18:32:30 <jeremyfreudberg> if he wants to
18:32:41 <tellesnobrega> yeah
18:32:52 <tellesnobrega> were you thinking into some other task?
18:32:55 <esikachev> sometimes i have a free time
18:33:01 <esikachev> i can
18:33:40 <jeremyfreudberg> resolving the concerns in https://review.openstack.org/#/c/333273/ ?
18:34:14 <jeremyfreudberg> I also filed these two bugs https://bugs.launchpad.net/sahara/+bug/1705335 and https://bugs.launchpad.net/sahara/+bug/1705037 lately
18:34:15 <openstack> Launchpad bug 1705335 in Sahara "[SIE] Cannot build image with oracle jdk" [Undecided,New]
18:34:16 <openstack> Launchpad bug 1705037 in Sahara "[UI] Caching can be improved" [Undecided,New]
18:34:22 <jeremyfreudberg> but yes, CDH is the priority
18:34:37 <jeremyfreudberg> (or rather, plugins upgrading in general)
18:35:06 <esikachev> but i have not any resources for devstack. only my macbook :)
18:35:16 <tellesnobrega> jeremyfreudberg, yes
18:35:19 <esikachev> but 16gb :)
18:35:48 <tellesnobrega> esikachev, if I could suggest I would recommend upgrade cdh to 5.10 and 5.12
18:36:01 <jeremyfreudberg> yep
18:36:05 <jeremyfreudberg> most of that work can be done without devstack
18:36:16 <esikachev> ok
18:36:19 <tellesnobrega> thanks
18:36:55 <jeremyfreudberg> esikachev, when I ask about new CI resources, I can also ask about donating some resources for your general use too
18:36:57 <jeremyfreudberg> but no promises
18:37:25 <esikachev> thanks, it will be good
18:38:26 <tellesnobrega> I have other topic to discuss, I hit an interesting issue yesterday, I was trying to run a job that its binary was "large", around 115MB
18:38:31 <esikachev> we can use 64gb labs, but we can get large queue of patches
18:39:53 <tellesnobrega> two problems that I saw, first is that it timed out to copy the jar file. This can be solved by changing the timeout
18:40:24 <jeremyfreudberg> tellesnobrega, is this internal db? or swift?
18:40:43 <tellesnobrega> but the bigger problem here is, sftp.write() from paramike takes a long time to write files when the files get bigger (not sure if linear)
18:40:46 <tellesnobrega> swift
18:41:06 <esikachev> yes, I suggested adding nightly jobs for testing this case
18:41:20 <jeremyfreudberg> tellesnobrega, i see
18:41:45 <tellesnobrega> looking around, the solution can be increasing the sftp window size
18:41:46 <esikachev> something like this https://review.openstack.org/#/c/367959/
18:42:32 <notmyname> tellesnobrega: swift will only return success after the data has been fsync'd, so larger objects will take longer to write than smaller objects (regardless of differences in network transfer)
18:43:05 <jeremyfreudberg> tellesnobrega, I think the real solution is actually something else
18:43:14 <tellesnobrega> notmyname, I don't think the problem was from swift
18:43:24 <jeremyfreudberg> so right now we have binary retriever inside of sahara itself
18:43:24 <jeremyfreudberg> https://github.com/openstack/sahara/blob/master/sahara/service/edp/binary_retrievers/internal_swift.py#L36
18:43:26 <tellesnobrega> the problem is when sahara writes the files to the remote isntance
18:43:36 <jeremyfreudberg> actually we should not do the retrieving inside of sahara
18:43:47 <jeremyfreudberg> we should instead retrieve it from inside the cluster itself
18:43:55 <jeremyfreudberg> but that is a bigger discussion
18:44:25 <tellesnobrega> jeremyfreudberg, maybe a bigger discussion (will add to the queens ptg etherpad)
18:45:08 <jeremyfreudberg> tellesnobrega, yep
18:45:17 <tellesnobrega> one thing that I tried was using sftp.put instead of write and the transfer goes a lot faster
18:45:30 <tellesnobrega> have to check with sahara, just tried a dummy example
18:46:07 <jeremyfreudberg> cool, let me know what happens
18:46:23 <tellesnobrega> I will create a bug on this
18:47:24 <jeremyfreudberg> great
18:48:42 <jeremyfreudberg> anything else to discuss?
18:48:55 <tellesnobrega> not from my side
18:48:59 <tellesnobrega> you? esikachev?
18:49:20 <esikachev> from me too
18:49:26 <jeremyfreudberg> that's it for mee
18:49:29 <jeremyfreudberg> *me
18:50:17 <tellesnobrega> ok, thanks guys for showing up, turned out the meeting was better than expected
18:50:27 <jeremyfreudberg> yep, very good meeting
18:50:31 <tellesnobrega> jeremyfreudberg, I will take a look into those bugs
18:50:37 <jeremyfreudberg> thanks
18:50:43 <tellesnobrega> keep an eye for the one I'm filing
18:50:59 <jeremyfreudberg> yep, i already added a small reference to it on the etherpa
18:51:00 <jeremyfreudberg> d
18:51:02 <jeremyfreudberg> so i don't forget
18:51:05 <tellesnobrega> thanks esikachev for helping out
18:51:10 <tellesnobrega> cool
18:51:11 <tellesnobrega> thanks
18:51:14 <jeremyfreudberg> thanks esikachev!
18:51:15 <esikachev> np)
18:51:40 <tellesnobrega> lets keep up the good work :)
18:51:47 <tellesnobrega> #endmeeting