15:01:08 <rhochmuth> #startmeeting monasca
15:01:08 <openstack> Meeting started Wed Jan 18 15:01:08 2017 UTC and is due to finish in 60 minutes.  The chair is rhochmuth. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:09 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:12 <openstack> The meeting name has been set to 'monasca'
15:01:21 <rhochmuth> o/
15:01:24 <rbak> o/
15:01:27 <kamil_> o/
15:01:33 <hosanai> o/
15:01:35 <bklei> o/
15:01:35 <witek> oo
15:01:52 <jgrassler> o/
15:01:52 <rhochmuth> https://etherpad.openstack.org/p/monasca-team-meeting-agenda
15:02:08 <rhochmuth> Agenda for Wednesday January 18 2016 (15:00 UTC)
15:02:08 <rhochmuth> 1.	Reviews:
15:02:08 <rhochmuth> 1.	https://review.openstack.org/#/c/420579/
15:02:08 <rhochmuth> 2.	https://review.openstack.org/#/c/418509/
15:02:15 <rhochmuth> Short agenda for today
15:02:21 <rhochmuth> let's get started
15:02:29 <rhochmuth> #topic https://review.openstack.org/#/c/420579/
15:02:36 <dhague> o/
15:02:55 <rhochmuth> Removes the monasca libs since they are not compatible with the new
15:02:55 <rhochmuth> kafka.
15:03:18 <rhochmuth> is anyone here to talk about this
15:03:23 <bklei> i guess i don't know what that affects?
15:03:31 <jgrassler> I guess that's me
15:03:31 <rhochmuth> i tried to track down joe prior to the meeting, but couldn't reach him
15:03:43 <bklei> will that break rally/tempest tests?
15:03:58 <jgrassler> (dirk added it to the agenda but looks like he isn't here)
15:04:13 <rhochmuth> do we still want to try and cover today
15:04:25 <rhochmuth> without joe, we won't have the best representation from monasca
15:04:41 <dirk> o/
15:04:45 <rhochmuth> as he know all the details and is familiar with the topic more so than me
15:04:52 <rhochmuth> but i can do my best
15:05:01 <bklei> it seems scary not knowing what it will break
15:06:20 <rhochmuth> i guess there is a request for us to remove our libraries so that oslo can move to the latest python-kafka libraries, which has significant issues
15:06:28 <rhochmuth> hasn't joe sent out email on this topic
15:06:35 <rhochmuth> i just discussed this again last week
15:06:47 <rhochmuth> the lates python-kafka libraries have got problems
15:06:53 <jgrassler> There was a fair amount of discussion on openstack-dev, yes.
15:07:06 <rhochmuth> additionally, they have moved to an async only model
15:07:10 <jgrassler> Performance issues with the latest python-kafka libraries.
15:07:16 <rhochmuth> which we can't use right now
15:08:12 <witek> http://lists.openstack.org/pipermail/openstack-dev/2017-January/110357.html
15:08:42 <rhochmuth> it looks like the monasca team is doing all the analysis here
15:10:45 <jgrassler> So the problem with Monasca's libraries being dropped from global-requirements is that they will be disregarded entirely from that point on, i.e. nobody will pay attention to whether the libraries the rest of OpenStack are compatible with Monasca.
15:11:17 <jgrassler> It's halfway to dropping out of OpenStack entirely, really...
15:11:45 <witek> jgrassler: agree
15:11:59 <jgrassler> Now most of Monasca doesn't share machines with other OpenStack services (barring the agent), but it's still not a good thing...
15:12:15 <rhochmuth> well, i just added a -1, but i'm confused why our libraries would be dropped
15:12:40 <jgrassler> Halfway, not all the way.
15:20:27 <jgrassler> Ok. So that's ruled out then.
15:21:03 <jgrassler> And the problems with python-kafka in its current version are unsurmountable?
15:21:29 <rhochmuth> well, i wouldn't call them unsurmountable
15:21:39 <rhochmuth> we would have to move to async
15:22:01 <rhochmuth> we are also very interested in the confluent kafka library
15:22:22 <rhochmuth> as it is reported to be faster and better supported
15:22:40 <rhochmuth> https://github.com/confluentinc/confluent-kafka-python
15:22:55 <jgrassler> Ok
15:23:04 <rhochmuth> but, the same issue about moving to async still need to be addressed
15:23:21 <rhochmuth> the main problem with async is the possiblty for message loss
15:23:45 <rhochmuth> messages are buffered in memory and then sent as a batch asynchronousely
15:24:26 <rhochmuth> if the api went down while the messages were still sitting in memory, then there is message loss
15:25:10 <rhochmuth> unfortunately, the http framework doesn't support response handlers or callbacks
15:25:48 <rhochmuth> we are looking at layering a sync api ontop of the async api
15:26:01 <rhochmuth> but it is more involved
15:26:01 <jgrassler> Yeah. That _is_ ugly.
15:26:52 <rhochmuth> if we figured out the async issue, then we would gladly move
15:26:54 <jgrassler> Might be an idea to put that reasoning on that review as well (at a guess the oslo folks are only aware of the problems this situation creates for them)
15:28:05 <rhochmuth> sure, we'll add comments to the review, but i thought the discussion in openstack-dev was sufficient
15:28:16 <jkeen> I explained the problems on the review and on the dev mailing list a couple times.  Do you think what is on there now is sufficient?
15:28:27 <jgrassler> I think there's just too much noise on the list :-/
15:28:32 <rhochmuth> hi joe
15:28:57 <rhochmuth> sounds like we should update the reviews with your findings/reasoning
15:29:23 <jkeen> I did put an update on there about our reasons for avoiding async last night.
15:29:37 <rhochmuth> thx jkeen
15:30:38 <rhochmuth> so, unless there is more to discuss on this topic, maybe we should move on
15:30:45 <rhochmuth> i realize this is a touchy subject
15:30:57 <rhochmuth> and oslo really want to move
15:31:07 <jgrassler> Yeah
15:31:11 <jgrassler> One more thing:
15:31:23 <jgrassler> I just had a quick chat with dirk (he's having connection trouble)
15:32:10 <jgrassler> He said that Monasca geting dropped is unlikely to break stuff as long as coexistence of Monasca and other OpenStack services on the same machine is not required.
15:32:20 <jgrassler> (Also, python-monasca is still in there)
15:32:51 <witek> monasca-agent has dependency on monasca-statsd
15:32:55 <rhochmuth> well, we want coexistence
15:33:15 <jgrassler> It puts us as distributor into a bit of a pickle, though because we cannot ship multiple versions of python-kafka in the same repository :-(
15:33:27 <rhochmuth> if the monasca libraires are removed, then the version of python-kafka used by monasca will be different than the one used by oslo
15:33:37 <rhochmuth> that isn't acceptable
15:35:34 <rhochmuth> it would have been nice if everything was in it's own virtualenv, but that isn't how openstack was structured
15:36:44 <jgrassler> No, not really.
15:37:01 <jgrassler> Maybe the oslo folks can be convinced to leave the old python-kafka in there for a bit longer.
15:37:34 <jgrassler> As far as I recall they were upgrading because they didn't want to ship a deprecated python-kafka version the second time in a row.
15:37:34 <rhochmuth> sorry, why do we have to convince the oslo folks to use the library longer?
15:38:01 <dirk> because sticking to old versions should be exception not the rule
15:38:13 <jgrassler> If the old library remains in there, there's no reason to remove Monasca from global-requirements
15:38:32 <rhochmuth> moving to new incompatible libraries with lot's of issues that break existing projects should also be a consideration
15:38:42 <jgrassler> And maybe "We can adapt but it will take time" will get them to give you time.
15:39:30 <rhochmuth> the old library is deprecated, but at least it worked
15:39:33 <jgrassler> Well, as you said, Monasca originally got python-kafka in there. I don't think it's used a lot in the rest of Openstack.
15:40:09 <jkeen> It might be worth pointing out that this is also the first version of the 1.x series that even works.  Up until 1.3 performance and stability was really bad.
15:41:58 <witek> jkeen: do you mean the new Producer or SimpleProducer?
15:42:18 <rhochmuth> jkeen had to leave the meeting
15:42:37 <rhochmuth> i think he is referring to Producer
15:43:16 <witek> thanks
15:43:24 <dirk> what are the options? can you talk to the oslo.messaging team once more?
15:43:48 <witek> I think we should prioritize migration to the new library (kafka-python or confluent-kafka)
15:43:57 <witek> but not in short-term
15:44:19 <jgrassler> Yeah, short-term didn't sound viable.
15:44:43 <rhochmuth> i'll discuss more with jkeen, but i didn't get the sense that it was going to be easy
15:44:52 <jgrassler> But in that case it's even more important to talk to the oslo.messaging folks (and hopefully get them convinced to give you enough time).
15:45:17 <rhochmuth> jgrassler: ok, we will follow-up
15:46:36 <rhochmuth> moving to next topic
15:46:48 <rhochmuth> #topic https://review.openstack.org/#/c/418509/
15:46:53 <rbak> That's mine
15:47:11 <rbak> I just wanted to get some eyes on this.  Either get it merged or get some more feedback.
15:47:30 <rhochmuth> ok, i'll take a look
15:47:46 <rbak> Thanks
15:47:51 <rhochmuth> seems like the biggest issues are around documentation and monasca-setup
15:48:14 <rhochmuth> code is fine, except for resolving what to do about that
15:48:36 <rbak> Yeah, just some confusion over whether we have common standards for the documentation or monasca-setup
15:48:57 <rbak> I think we resolved the docs question, but whether this needs a monasca-setup flag is still up for debate.
15:49:45 <bklei> it's a good question -- do all config options belong in monasca-setup?  if not, what's the criteria?
15:50:14 <rhochmuth> i'm not sure there is a criteria
15:50:17 <rhochmuth> :-)
15:50:29 <rbak> I didn't look like all config options were in monasca-setup when I looked, but I'm not sure how we judge what belongs.
15:50:47 <rhochmuth> in general, i think we were trying to get everythingn in monasca-setup
15:51:02 <rhochmuth> but, i haven't looked at that yet
15:51:22 <bklei> if that's the case, that's easy.  add new stuff, add to monasca-setup, try to backfill as we refactor?
15:51:48 <rhochmuth> i'll check with some of the folks here and try and get an answer or update to the review
15:52:06 <rbak> Sounds good.
15:53:23 <rhochmuth> i guess we have a few minutes
15:53:25 <rhochmuth> any other topics
15:53:29 <rhochmuth> in closing
15:54:43 <bklei> not from me
15:55:11 <rhochmuth> ok, i'll see you all next week
15:55:15 <rhochmuth> bye
15:55:15 <JamesGu> @rhochmuth... just an fyi.... we are starting to look into port monasca devstack to Suse OS
15:55:17 <kamil_> bye
15:55:31 <bklei> thx roland
15:55:42 <rhochmuth> thanks JamesGu. Nice to know.
15:55:53 <rhochmuth> #endmeeting