13:01:08 <joehuang> #startmeeting tricircle 13:01:09 <openstack> Meeting started Wed Feb 17 13:01:08 2016 UTC and is due to finish in 60 minutes. The chair is joehuang. Information about MeetBot at http://wiki.debian.org/MeetBot. 13:01:10 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 13:01:11 <BeanZhang> Thanks 13:01:13 <openstack> The meeting name has been set to 'tricircle' 13:01:37 <joehuang> #topic rollcall 13:01:46 <joehuang> #info joehuang 13:01:54 <yipei> #info yipei 13:01:54 <BeanZhang> #info BeanZhang 13:02:04 <zhiyuan_> #info zhiyuan 13:02:06 <joehuang> hi Yipei, your connection is not stable 13:02:09 <Pengfei> #info Pengfei 13:02:11 <yipei> #info yipei 13:02:11 <yipei> yes 13:02:31 <joehuang> #topic progress review 13:02:32 <yipei> i have already tried many times 13:02:54 <joehuang> you can use VPN 13:03:53 <joehuang> Hi Yipei, have you report a bug to DevStack for the multi-region installation issue 13:04:11 <yipei> yes, i have 13:04:18 <joehuang> link? 13:04:42 <yipei> https://bugs.launchpad.net/devstack/+bug/1540802 13:04:42 <openstack> Launchpad bug 1540802 in devstack "Installing devstack with multi-region setting fails" [Undecided,New] 13:05:53 <joehuang> Bean and Pengfei, how about your environment setup, is there any challenge? 13:06:15 <Pengfei> Bean and I had try devstack already, and we both succeed install devstack and tricircle plugin. But when we try the Step 6 ,command "nova aggregate-list" cannot show any result. 13:06:32 <BeanZhang> After we installed, we try to create pod instances for Tricircle and bottom OpenStack using curl POST request, but “nova aggregate-list” shows nothing, didn’t show the results as README, Name=ag_Pod1, AZ=az1. 13:07:03 <BeanZhang> we use the latest tricirle code from github, having 130 commits 13:07:21 <BeanZhang> in README, it says Tricircle API service will automatically create a aggregate when user creates a bottom pod 13:07:44 <BeanZhang> we have not found the reason 13:08:08 <joehuang> yes, we have tested many times, it's unusual that it doesn't work 13:08:47 <zhiyuan_> Was the result showing in step 5 as expectd? 13:09:11 <BeanZhang> yes, in step 5, it shows more than README 13:09:14 <joehuang> have you tried to log in to the mysql, and check to see if all tables are there 13:10:00 <BeanZhang> we haven't check the mysql tables 13:10:07 <Pengfei> ubuntu@devstack-1:~$ token=1af602f8c4fd479dab52d8ca2de72903 ubuntu@devstack-1:~$ curl -X POST http://127.0.0.1:19999/v1.0/pods -H "Content-Type: application/json" \ > -H "X-Auth-Token: $token" -d '{"pod": {"pod_name": "RegionOne"}}' {"pod": {"dc_name": "", "pod_az_name": "", "pod_id": "38d38c23-4dd8-4111-be91-313ad18455f4", "az_name": "", "pod_name": "RegionOne"}}ubuntu@devstack-1:~$ ubuntu@devstack-1:~$ ubuntu@devsta 13:11:43 <BeanZhang> How to know there should be how many and which tables? 13:12:50 <BeanZhang> pengfei, zhiyuan and joe have left from my client 13:13:33 <zhiyuan_> login again, the tool I used was suddenly offline 13:13:50 <joehuang> sorry, my connection was lost 13:14:19 <Pengfei> Me too 13:14:21 <Pengfei> TT 13:14:31 <BeanZhang> i use LimeChat as an IRC client and it seems very stable 13:14:34 <zhiyuan_> What about the result of curl -X GET http://127.0.0.1:19999/v1.0/pods? Were the two pods created successfully? 13:14:53 <BeanZhang> let me have a try 13:15:57 <joehuang> It's very strange that there are two region-one records 13:16:54 <joehuang> duplicated entry exception will be raised if insertion from the API 13:16:56 <zhiyuan_> Please show the list of endpoints again, I missed that 13:18:32 <joehuang> region-one pod was added once or twice? 13:18:59 <BeanZhang> I entered curl -X GET http://127.0.0.1:19999/v1.0/pods? -H "Content-Type: application/json" -H "X-Auth-Token: $token" 13:19:22 <yipei> sorry about the bad connection 13:19:23 <BeanZhang> but Authentication required, echo $token 9a3506f375884c399bde48e992cb6634 13:19:54 <joehuang> sure you need -H "X-Auth-Token: $token" 13:20:27 <joehuang> no CLI provided yet to insert the token by default, you have to add the token head 13:20:42 <zhiyuan_> oh, please remove the "?" 13:20:54 <yipei> i still try to solve the bug correctly, and the progress is a little slow. i will speed up in the next week 13:20:58 <Pengfei> curl -X POST http://127.0.0.1:19999/v1.0/pods -H "Content-Type: application/json" \ > -H "X-Auth-Token: $token" -d '{"pod": {"pod_name": "RegionOne"}}' 13:21:10 <Pengfei> {"pod": {"dc_name": "", "pod_az_name": "", "pod_id": "38d38c23-4dd8-4111-be91-313ad18455f4", "az_name": "", "pod_name": "RegionOne"}} 13:21:25 <Pengfei> curl -X POST http://127.0.0.1:19999/v1.0/pods -H "Content-Type: application/json" \ > -H "X-Auth-Token: $token" -d '{"pod": {"pod_name": "Pod1", "az_name": "az1"}}' 13:21:34 <Pengfei> {"pod": {"dc_name": "", "pod_az_name": "", "pod_id": "64f60853-00b4-43c6-a36e-59afdce62f9d", "az_name": "az1", "pod_name": "Pod1"}} 13:21:35 <joehuang> to Yipei, it's ok. Please also send a mail in the M-L for the bug 13:21:45 <yipei> ok 13:22:18 <joehuang> To Pengfei, the last one you post is successful and correct 13:22:46 <Pengfei> nova aggregate-list 13:22:53 <Pengfei> +----+------+-------------------+ | Id | Name | Availability Zone | +----+------+-------------------+ +----+------+-------------------+ 13:23:16 <Pengfei> nothing ...TT 13:23:34 <joehuang> you need top region to insert first 13:23:40 <BeanZhang> curl -X GET http://127.0.0.1:19999/v1.0/pods -H "X-Auth-Token: $token" 13:23:47 <BeanZhang> {"pods": [{"dc_name": "", "pod_az_name": "", "pod_id": "056b3cc9-c8a5-454f-be2b-32bd2bbf57f2", "az_name": "", "pod_name": "RegionOne"}, {"dc_name": "", "pod_az_name": "", "pod_id": "48cd53e0-036f-43d2-908c-daa9769fcd87", "az_name": "az1", "pod_name": "ag_Pod1"}, {"dc_name": "", "pod_az_name": "", "pod_id": "d977b991-44f4-47f6-9ad7-5ffb3623be5e", "az_name": "az1", "pod_name": "Pod1"}]} 13:24:49 <BeanZhang> the token was expired just now 13:25:17 <BeanZhang> are the results right? 13:25:21 <zhiyuan_> A strange pod "ag_Pod1" 13:25:32 <joehuang> could you show the openstack region information 13:25:39 <joehuang> openstack region list 13:26:05 <BeanZhang> Region | Parent Region | Description 13:26:11 <BeanZhang> Pod1 | None | 13:26:16 <joehuang> the ag_Pod1 is post intended ot created by API automaticly? 13:26:17 <BeanZhang> RegionOne | None | 13:26:30 <BeanZhang> ag_Pod1 is created by me 13:26:37 <BeanZhang> post intended 13:26:52 <BeanZhang> i see the 1 | ag_Pod1 | az1 13:27:10 <BeanZhang> so create it 13:27:21 <joehuang> you mean get pod list 13:27:39 <zhiyuan_> Need to check the log of nova_api_gw, pod api will send request to it to create aggregates 13:28:55 <BeanZhang> where is the log of nova_api_gw, /opt/stack/logs? 13:28:57 <joehuang> Hi, Bean, now you have one record in aggregate-list 13:30:39 <BeanZhang> nova aggregate-list shows still nothing 13:31:38 <joehuang> no error returned when post a new pod request? 13:31:46 <BeanZhang> no error 13:31:48 <zhiyuan_> opt/stack/new/screen-logs/screen-xxx 13:32:33 <BeanZhang> i enter the first post request again, it echos Top region already exists 13:32:34 <joehuang> So I think this maybe the nova client is not accessing the tricircle 13:32:57 <BeanZhang> i enter the second post request again, it echos Record already exists 13:33:06 <joehuang> could you use nova --debug aggregate-list to see which region the client issues a command to 13:33:21 <joehuang> sure, duplicated error 13:34:24 <joehuang> maybe the nova-client is issueing the command to a bottom pod 13:36:19 <joehuang> At least two regions in your environment, and if the command is issued to Tricircle Nova-APIGW, 13:37:26 <joehuang> it should look like a request to port: 19998 13:37:39 <Pengfei> sorry about the bad connection, I haven't go to school 13:37:48 <BeanZhang> --debug, it echos a lot of found extension, REQ, RESP 13:38:15 <joehuang> Nova-APIGW listened on port 19998 13:38:21 <joehuang> yes 13:38:31 <zhiyuan_> the import part is the url nova client uses 13:38:49 <zhiyuan_> will showed when using --debug 13:39:35 <BeanZhang> yeah, REQ: curl -g -i -X GET http://127.0.0.1:8774/v2.1/ -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}2b9a5c81209c44dffcce741de090f29552a0435b" 13:39:42 <BeanZhang> "GET /v2.1/ HTTP/1.1" 200 383 13:39:48 <BeanZhang> RESP: [200] Content-Length: 383 X-Compute-Request-Id: req-d3c7bcfc-a55a-4b84-8f5a-b798c0e782d6 Vary: X-OpenStack-Nova-API-Version Connection: keep-alive X-Openstack-Nova-Api-Version: 2.1 Date: Wed, 17 Feb 2016 13:33:33 GMT Content-Type: application/json 13:39:56 <BeanZhang> RESP BODY: {"version": {"status": "CURRENT", "updated": "2013-07-23T11:33:21Z", "links": [{"href": "http://127.0.0.1:8774/v2.1/", "rel": "self"}, {"href": "http://docs.openstack.org/", "type": "text/html", "rel": "describedby"}], "min_version": "2.1", "version": "2.22", "media-types": [{"base": "application/json", "type": "application/vnd.openstack.compute+json;version=2.1"}], "id": "v2.1"}} 13:40:05 <zhiyuan_> so the url is not correct 13:40:31 <joehuang> 8774 is nova port , right? 13:40:33 <BeanZhang> which url? 13:41:01 <joehuang> http://127.0.0.1:8774/v2.1/ is nova endpoint 13:41:14 <zhiyuan_> but we have two nova api 13:41:29 <zhiyuan_> one nova api gateway, one bottom nova api 13:41:55 <zhiyuan_> try "openstack endpoint list" to see which port is for nova api 13:41:57 <joehuang> nova-apigw listened on the port 19998, althouth service type is nova 13:42:14 <BeanZhang> before the results above, there are other logs 13:42:32 <BeanZhang> REQ: curl -g -i -X GET http://127.0.0.1:8774/v2.1/ab54b5c3a73c4dff9762ed7bac5d55a2 -H "User-Agent: python-novaclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}2b9a5c81209c44dffcce741de090f29552a0435b" 13:42:33 <BeanZhang> INFO (connectionpool:207) Starting new HTTP connection (1): 127.0.0.1 13:42:35 <joehuang> From the above information "min_version": "2.1", "version": "2.22", it must be nova 13:43:05 <BeanZhang> but 404 not found,"GET /v2.1/ab54b5c3a73c4dff9762ed7bac5d55a2 HTTP/1.1" 404 52 13:43:11 <BeanZhang> RESP: [404] Date: Wed, 17 Feb 2016 13:33:33 GMT Connection: keep-alive Content-Type: text/plain; charset=UTF-8 Content-Length: 52 X-Compute-Request-Id: req-45145d94-c409-4230-a7c5-5783116d1ccf 13:43:15 <zhiyuan_> try this: nova --os-region-name RegionOne aggregate-list 13:43:23 <joehuang> so Bean and Pengfei, please check endpoint in KeyStone. 13:43:34 <BeanZhang> 1 | ag_Pod1 | az1 13:43:40 <BeanZhang> 3 | ag_ag_Pod1 | az1 13:43:43 <joehuang> yes ZHiyuan's work 13:44:00 <Pengfei> bingo! 13:44:14 <Pengfei> ubuntu@devstack-1:/home/stack/devstack$ nova --os-region-name RegionOne aggregate-list Your request was processed by a Nova API which does not support microversions (X-OpenStack-Nova-API-Version header is missing from response). Warning: Response may be incorrect. +----+---------+-------------------+ | Id | Name | Availability Zone | +----+---------+-------------------+ | 1 | ag_Pod1 | az1 | +----+--------- 13:44:22 <zhiyuan_> so you need to specify Region in your openrc/adminrc file 13:44:23 <BeanZhang> yeah, i know where is the reason 13:44:28 <BeanZhang> yes 13:44:30 <joehuang> Zhiyuan, I am afraid your script change for two nodes may lead to chaos 13:44:34 <Pengfei> haha~ 13:44:55 <joehuang> or the admin-rc issue? 13:45:13 <BeanZhang> maybe in tricircle README, it should point out the suggested admin-openrc.sh 13:45:39 <joehuang> what's your admin-rc configuration? 13:45:42 <zhiyuan_> Yes, maybe you two submit a patch to improve README 13:45:46 <BeanZhang> we use the admin-openrc.sh from the install guide 13:45:52 <BeanZhang> ok 13:46:10 <Pengfei> ok~ 13:46:12 <zhiyuan_> I always specify os_region so didn't find this issue 13:46:13 <BeanZhang> http://docs.openstack.org/liberty/install-guide-ubuntu/keystone-openrc.html 13:46:36 <BeanZhang> this is our openrc file 13:46:46 <BeanZhang> and not specify the os_region 13:46:53 <joehuang> OK. let's continue this in #openstack-tricircle channel for daily discussion 13:47:06 <zhiyuan_> So, the problem is, without specifying os_region, nova client will try to find one nova service, thus it go to the wrong service 13:47:13 <joehuang> and please submit a patch for this 13:47:19 <BeanZhang> yeah 13:47:28 <BeanZhang> OK 13:47:35 <joehuang> Zhiyuan, your recent progress? 13:48:13 <BeanZhang> after specify the os-region-name, it echos Your request was processed by a Nova API which does not support microversions (X-OpenStack-Nova-API-Version header is missing from response). Warning: Response may be incorrect. 13:48:24 <BeanZhang> is it normal? 13:48:37 <joehuang> yes 13:48:43 <BeanZhang> ok~ 13:49:01 <zhiyuan_> this can be changed later, since our nova api gateway doesn't support microversions currently 13:49:14 <zhiyuan_> so nova client raises a warning 13:50:11 <zhiyuan_> For my progress, the L3 N-S networking patch has been merged. Now I am working the security group part. It will take some times since it's also a big stuff...... 13:50:33 <joehuang> please split into into smaller patch 13:51:05 <joehuang> and also for the quota part, we have already have a terrible big patch :( 13:51:15 <joehuang> please review that, thanks 13:51:23 <zhiyuan_> smaller patch? then maybe I can submit the RPC part first 13:51:45 <joehuang> yes 13:51:56 <zhiyuan_> ok, I will find time to review it tomorrow 13:52:25 <joehuang> ok, it's a little bit late, let's have more chat in #openstack-tricircle channel 13:52:42 <zhiyuan_> fine 13:52:44 <joehuang> thanks for your attendence 13:53:08 <joehuang> especially for Bean, Pengfei and Yipei, you are still in vacation 13:53:31 <joehuang> see you next time 13:53:56 <joehuang> #endmeeting