| *** Qiming has quit IRC | 00:02 | |
| *** shu-mutou-AFK is now known as shu-mutou | 00:12 | |
| *** dixiaoli_ has quit IRC | 01:00 | |
| *** zzxwill has joined #senlin | 01:04 | |
| *** Qiming has joined #senlin | 01:07 | |
| *** dixiaoli_ has joined #senlin | 01:10 | |
| *** dixiaoli_ has quit IRC | 01:14 | |
| *** Yanyanhu has joined #senlin | 02:09 | |
| openstackgerrit | Merged openstack/python-senlinclient: Updated from global requirements https://review.openstack.org/288036 | 02:14 |
|---|---|---|
| openstackgerrit | Merged openstack/senlin: Improve the text in install-guide https://review.openstack.org/289030 | 02:15 |
| openstackgerrit | Yanyan Hu proposed openstack/senlin: Move an item back to TODO list https://review.openstack.org/289116 | 02:24 |
| lixinhui | Yanyan Hu, it seems that the latest neutron/neutron-lbaas can not built up lbaas successfully... | 02:26 |
| lixinhui | could you let me know which version you are using? | 02:26 |
| Yanyanhu | ok, let me check it | 02:33 |
| Yanyanhu | hi, lixinhui, I think it is 7.0.0.0b2.dev50 | 02:35 |
| Yanyanhu | I installed it using devstack a long while ago | 02:35 |
| Yanyanhu | several month ago | 02:36 |
| lixinhui | got it, thanks, Yanyanhu | 02:37 |
| Yanyanhu | you're welcome | 02:38 |
| *** elynn has joined #senlin | 02:39 | |
| elynn | Morning | 02:39 |
| Yanyanhu | morning | 02:46 |
| *** dixiaoli has joined #senlin | 02:46 | |
| openstackgerrit | Yanyan Hu proposed openstack/senlin: Minor revise os.nova.server profile https://review.openstack.org/289119 | 02:52 |
| *** xuhaiwei has quit IRC | 02:54 | |
| openstackgerrit | Merged openstack/senlin: Move an item back to TODO list https://review.openstack.org/289116 | 03:04 |
| *** dixiaoli has left #senlin | 03:04 | |
| openstackgerrit | Yanyan Hu proposed openstack/senlin: Minor revise os.nova.server profile https://review.openstack.org/289119 | 03:19 |
| *** yuanying has quit IRC | 03:22 | |
| *** idonotknow_ has joined #senlin | 03:31 | |
| *** idonotknow__ has joined #senlin | 03:39 | |
| *** yuanying has joined #senlin | 03:40 | |
| *** idonotknow_ has quit IRC | 03:41 | |
| *** yuanying has quit IRC | 03:44 | |
| *** zzxwill has quit IRC | 03:50 | |
| *** elynn has quit IRC | 03:58 | |
| openstackgerrit | Merged openstack/senlin: Minor revise os.nova.server profile https://review.openstack.org/289119 | 04:01 |
| *** zzxwill has joined #senlin | 04:04 | |
| *** yuanying has joined #senlin | 04:08 | |
| *** zzxwill has quit IRC | 04:10 | |
| *** Yanyanhu has quit IRC | 04:13 | |
| *** Yanyanhu has joined #senlin | 04:13 | |
| *** elynn has joined #senlin | 04:19 | |
| *** elynn has quit IRC | 04:23 | |
| *** elynn has joined #senlin | 04:24 | |
| *** zzxwill has joined #senlin | 04:54 | |
| *** dixiaoli_ has joined #senlin | 05:30 | |
| *** dixiaoli_ has left #senlin | 05:30 | |
| lixinhui | Qiming, Yanyanhu, what threshold for incomming.rate is sensible in your mind to be as alarm | 05:51 |
| lixinhui | idonotknow_ what is your idea here | 05:52 |
| Yanyanhu | hi, lixinhui, actually that depends on the application type | 05:52 |
| lixinhui | all the suggeston are welcome | 05:52 |
| lixinhui | when redering to an application type | 05:52 |
| Yanyanhu | we just used a very small value, e.g. 100 bytes per second to make the test | 05:52 |
| Yanyanhu | if you just want to test the workflow, this is ok I think | 05:54 |
| lixinhui | Yanyanhu, when referring to the application type, are you mentionning the average rate for one application or the workload to afford | 05:54 |
| Yanyanhu | but in real cases, this threshold could vary cross different applications, like http server, or ftp server | 05:54 |
| lixinhui | yes | 05:54 |
| lixinhui | that is why to ask this question | 05:54 |
| Yanyanhu | e.g. for http server, I think this value won't be that large | 05:54 |
| Yanyanhu | I also have no experience about this issue :( | 05:55 |
| Qiming | it is always a co-design factor regarding your benchmark | 05:55 |
| lixinhui | I see, Qiming. | 05:55 |
| lixinhui | so I am asking what if you do when the workload is chosen to skeleton this threshold | 05:55 |
| lixinhui | get the average rate | 05:56 |
| lixinhui | or | 05:56 |
| lixinhui | max | 05:56 |
| lixinhui | half of max | 05:56 |
| Qiming | say if you have your benchmark send 10 requests per second, it makes no sense to do autoscaling above 20 requests per second | 05:56 |
| lixinhui | or some best practice here | 05:56 |
| lixinhui | once you mentioned | 05:56 |
| Qiming | I heard some opinions about 70% | 05:57 |
| lixinhui | you IBM presents some rate trigerred alarm to customers | 05:57 |
| lixinhui | could you find more details? | 05:57 |
| Qiming | which is already a signal of urgency | 05:57 |
| lixinhui | 70% waht? | 05:57 |
| lixinhui | the peak bandwidth | 05:57 |
| Yanyanhu | yea, I guess 70% is common choice | 05:57 |
| Qiming | 70% of the total capacity | 05:57 |
| Qiming | some users would prefer a lower threshold | 05:58 |
| Qiming | so ... you will need to know your capacity limit | 05:58 |
| Qiming | which is a per-application metric | 05:59 |
| lixinhui | okay | 05:59 |
| Qiming | you just push, push, ... stress test it, until it cannot produce more outputs | 05:59 |
| lixinhui | so maybe we can get the max rate range per member, then use 70% of the max as the threshold | 06:00 |
| Qiming | or the service becomes not so responsive | 06:00 |
| Qiming | if you have a LB, you can actually stressing the cluster as a whole | 06:00 |
| lixinhui | that is another problem | 06:00 |
| Qiming | but it would be good to get a per-node metric first | 06:00 |
| lixinhui | ceilometer seems only get incoming/out bytes in total, no | 06:01 |
| Qiming | the per-node metric can serve a base line | 06:01 |
| lixinhui | calcaution per memebr | 06:01 |
| lixinhui | we can only set a pre-set threahold | 06:01 |
| Qiming | so the metric is already aggregated at the LB side | 06:01 |
| lixinhui | based on planning on the side of pool | 06:01 |
| lixinhui | yes | 06:01 |
| Yanyanhu | hi, lixinhui, you can calculate the rate per member | 06:02 |
| Qiming | that is reasonable | 06:02 |
| Yanyanhu | just need to extend related support in lbaas | 06:02 |
| Yanyanhu | and also configure the pipeline | 06:02 |
| lixinhui | okay, Yanyanhu | 06:02 |
| Yanyanhu | the doc I gave you before about our extension on lbaas support in ceilometer includes related code and description :) | 06:02 |
| lixinhui | that is a good sharing | 06:02 |
| Yanyanhu | not very complicated in lbaas v1 | 06:02 |
| Yanyanhu | but not sure about lbaas v2 | 06:03 |
| lixinhui | That part included by the doc has been implemented by default ceilometer | 06:03 |
| lixinhui | just the per member part | 06:03 |
| lixinhui | okay | 06:03 |
| lixinhui | I think it is about to use len(pool['members'] | 06:04 |
| Yanyanhu | oh, about per member, you need first to confirm the metadata is set correctly | 06:04 |
| Yanyanhu | if so, you just need to confiure pipeline | 06:04 |
| Yanyanhu | @staticmethod | 06:05 |
| Yanyanhu | def _get_sample(pool, data): | 06:05 |
| Yanyanhu | res_metadata = { | 06:05 |
| Yanyanhu | 'metering': { | 06:05 |
| Yanyanhu | 'pool_id': pool['id'], | 06:05 |
| Yanyanhu | 'member_count': len(pool['members']) | 06:05 |
| Yanyanhu | } | 06:05 |
| Yanyanhu | } | 06:05 |
| Yanyanhu | return make_sample_from_pool( | 06:05 |
| Yanyanhu | pool, | 06:05 |
| Yanyanhu | name='network.services.lb.incoming.bytes', | 06:05 |
| Yanyanhu | type=sample.TYPE_CUMULATIVE, | 06:05 |
| Yanyanhu | unit='B', | 06:05 |
| Yanyanhu | volume=data.bytes_in, | 06:05 |
| Yanyanhu | resource_metadata=res_metadata | 06:05 |
| Yanyanhu | ) | 06:05 |
| Yanyanhu | some code like this | 06:05 |
| Yanyanhu | please notice the line 'member_count: len(pool['members'])' | 06:05 |
| idonotknow__ | I am testing this | 06:05 |
| lixinhui | cool | 06:05 |
| lixinhui | TThanks, Yanyanhu | 06:06 |
| lixinhui | Hope we can get them out | 06:06 |
| idonotknow__ | I can get the member_count through pdb... | 06:06 |
| lixinhui | soon | 06:06 |
| Yanyanhu | then add something like this in pipeline | 06:06 |
| Yanyanhu | idonotknow__, nice :) | 06:06 |
| Yanyanhu | - name: lb_sink_byte | 06:06 |
| Yanyanhu | transformers: | 06:06 |
| Yanyanhu | - name: "rate_of_change" | 06:06 |
| Yanyanhu | parameters: | 06:06 |
| Yanyanhu | source: | 06:06 |
| Yanyanhu | map_from: | 06:06 |
| Yanyanhu | name: "network.services.lb\\.(incoming|outgoing).bytes" | 06:06 |
| Yanyanhu | unit: "B" | 06:06 |
| Yanyanhu | target: | 06:06 |
| Yanyanhu | map_to: | 06:06 |
| Yanyanhu | name: "network.services.lb.\\1.bytes.rate" | 06:06 |
| Yanyanhu | unit: "B/s" | 06:06 |
| Yanyanhu | type: "gauge" | 06:06 |
| Yanyanhu | scale: "1.0 / (resource_metadata.metering.member_count or 1)" | 06:07 |
| Yanyanhu | publishers: | 06:07 |
| Yanyanhu | - notifier:// | 06:07 |
| Yanyanhu | this is example for byte, for connection, I think it's the same logic | 06:07 |
| idonotknow__ | so just do not consider about an inactive one in bytes case? | 06:08 |
| Yanyanhu | I guess so | 06:08 |
| Yanyanhu | but it really depends on your strategy since inactive ones actually have no contribution to the entire capacity of your system | 06:09 |
| idonotknow__ | it will trigger another alarm to recover or just delete | 06:10 |
| idonotknow__ | they should be separate from each other,cause I can not get its number as well as its status at the same time now | 06:12 |
| lixinhui | I think for scaling scenario, we do not consider the inactive member case inside | 06:12 |
| lixinhui | Recovering case is different from it | 06:13 |
| idonotknow__ | OK,I will just assume they are all active | 06:13 |
| lixinhui | that need more consideration and addtional work | 06:13 |
| lixinhui | yes | 06:13 |
| idonotknow__ | it works using that code...but it will meet a problem...when I delete a member from a pool,the incoming rate will be a negative number at one moment | 06:27 |
| idonotknow__ | {"counter_name": "network.services.lb.incoming.bytes.rate", "user_id": null, "message_signature": "16b49dc9dbc331985242de02071881c7fd8ca30ba935323eb2bab60979d03ea2", "timestamp": "2016-03-07T06:24:41.493311", "resource_id": "b2c8c946-f67c-4607-b1aa-131225b8ed22", "message_id": "46ac7ac6-e42d-11e5-a0e2-f80f41fd1e94", "source": "openstack", "counter_unit": "B/s", "counter_volume": -47499.39104240597, "project_id": "1b9b9920a8284f | 06:27 |
| idonotknow__ | just like this | 06:27 |
| Yanyanhu | hmm, didn't notice this problem before... | 06:29 |
| Yanyanhu | maybe some bugs in ceilometer's lbaas support | 06:30 |
| Yanyanhu | I remeber all these metrics are read from haproxy's socks | 06:30 |
| idonotknow__ | I do not know....it happens as soon as the pool's member changed | 06:31 |
| *** idonotknow_ has joined #senlin | 06:33 | |
| Yanyanhu | but I think that will be ok since once you remove a member from pool, it has no relationship with the loadbalancer you're evaluating | 06:35 |
| *** idonotknow__ has quit IRC | 06:35 | |
| Yanyanhu | it won't influence the bytes_in/out of lb pool any more | 06:36 |
| idonotknow_ | just once every time...it may be OK | 06:39 |
| Qiming | one question | 06:45 |
| Qiming | the LBaaS (v1 or v2) only cares about quests, right? | 06:45 |
| Qiming | it is not able to do load balancing based on CPU consumptions on backend nodes? | 06:45 |
| Yanyanhu | yes, I think so | 06:46 |
| idonotknow_ | I only see it is based on network | 06:47 |
| Qiming | because the LB is based on technologies like HAProxy | 06:51 |
| Qiming | they won't have any idea how much resources are actual consumed at the backend | 06:51 |
| Qiming | it only cares about redirecting different requests to different machines | 06:51 |
| Qiming | so ... there is a huge gap here | 06:52 |
| Qiming | it only makes sense when you are running a single-purposed application which exposes a single API | 06:52 |
| Qiming | but that will never be the case | 06:53 |
| Qiming | if you have requests like a 80-20 mix of browsing and buying, the workload are never "evenly" distributed onto the backend nodes, so ... it is not a strict "load-balancing" | 06:54 |
| idonotknow_ | hmmm, I have no idea now.. | 06:56 |
| Qiming | it is perfectly okay to start from the common practices | 06:56 |
| Qiming | I'm talking about a much more broader (more unrealistic) load-balancing | 06:57 |
| *** zzxwill has quit IRC | 07:01 | |
| -openstackstatus- NOTICE: gerrit is going to be restarted due to bad performance | 07:25 | |
| *** ChanServ changes topic to "gerrit is going to be restarted due to bad performance" | 07:25 | |
| *** ChanServ changes topic to "IRCLog: http://eavesdrop.openstack.org/irclogs/%23senlin/ | Bugs: bugs.launchpad.net/senlin | Review: https://review.openstack.org/#/q/project:openstack/senlin,n,z" | 07:28 | |
| *** idonotknow__ has joined #senlin | 07:57 | |
| *** idonotknow_ has quit IRC | 07:59 | |
| *** elynn has quit IRC | 08:34 | |
| openstackgerrit | Yanyan Hu proposed openstack/senlin: Add property updatable check in Senlin profile https://review.openstack.org/289223 | 08:47 |
| *** elynn has joined #senlin | 08:50 | |
| *** elynn has quit IRC | 08:54 | |
| *** elynn has joined #senlin | 08:55 | |
| Qiming | guys, there is a meeting on openstack-meeting, about HA | 09:06 |
| Yanyanhu | openstack HA? | 09:07 |
| Qiming | you can sneak in if you are interested | 09:07 |
| Qiming | no, HA in general | 09:07 |
| Qiming | current topic is about compute node HA | 09:07 |
| Yanyanhu | have join that channel | 09:07 |
| *** elynn has quit IRC | 09:08 | |
| *** elynn has joined #senlin | 09:12 | |
| openstackgerrit | Yanyan Hu proposed openstack/senlin: Add property updatable check in Senlin profile https://review.openstack.org/289223 | 09:15 |
| *** elynn has quit IRC | 09:17 | |
| *** elynn has joined #senlin | 09:18 | |
| *** shu-mutou is now known as shu-mutou-OFF | 09:21 | |
| *** elynn has quit IRC | 09:24 | |
| *** elynn has joined #senlin | 09:38 | |
| *** elynn has quit IRC | 09:42 | |
| *** Yanyanhu has quit IRC | 09:42 | |
| *** elynn has joined #senlin | 09:42 | |
| *** elynn has quit IRC | 09:51 | |
| Qiming | lixinhui, there is a channel welcoming you -- openstack-ha | 09:57 |
| *** Qiming has quit IRC | 10:16 | |
| *** zzxwill has joined #senlin | 10:19 | |
| *** elynn has joined #senlin | 10:21 | |
| *** elynn has quit IRC | 10:26 | |
| *** elynn has joined #senlin | 10:26 | |
| *** elynn_ has joined #senlin | 10:38 | |
| *** elynn has quit IRC | 10:39 | |
| *** zzxwill has quit IRC | 10:56 | |
| *** elynn_ has quit IRC | 10:59 | |
| *** idonotknow__ has quit IRC | 10:59 | |
| *** Qiming has joined #senlin | 11:05 | |
| lixinhui | Okay, Qiming, I will turn on that channel. Thanks. | 12:04 |
| *** zzxwill has joined #senlin | 12:52 | |
| *** zzxwill has quit IRC | 14:16 | |
| *** zzxwill has joined #senlin | 14:24 | |
| *** Qiming has quit IRC | 15:36 | |
| *** zzxwill has quit IRC | 17:15 | |
| *** xuhaiwei has joined #senlin | 23:29 | |
| *** Qiming has joined #senlin | 23:34 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!