| rwellum | jascott1: for a laugh I just applied weave to an existing VM running kolla-k8s and I think it's blown my VM up... :) | 00:02 |
|---|---|---|
| jascott1 | woops | 00:02 |
| rwellum | Doesn't surprise me - having two CNI's not that clever | 00:02 |
| *** david-lyle has quit IRC | 00:03 | |
| *** dklyle has joined #openstack-kolla | 00:03 | |
| *** kbaegis has quit IRC | 00:04 | |
| *** mnasiadka has joined #openstack-kolla | 00:04 | |
| *** yuanying_ has joined #openstack-kolla | 00:04 | |
| jascott1 | i think i have everything running except my old enemy 'nova-api-create-cell' | 00:04 |
| *** yuanying has quit IRC | 00:05 | |
| *** david-lyle has joined #openstack-kolla | 00:07 | |
| *** dklyle has quit IRC | 00:08 | |
| *** hongbin has quit IRC | 00:08 | |
| *** mnasiadka has quit IRC | 00:09 | |
| jascott1 | rwellum every have problems with nova-api-create-cell? | 00:12 |
| *** dardelean has quit IRC | 00:12 | |
| rwellum | Yeah loads. | 00:12 |
| rwellum | In Pike mainly. | 00:12 |
| jascott1 | Unable to establish connection to http://keystone-internal:/v3/auth/tokens: HTTPConnectionPool(host='keystone-internal', port=80): Max retries exceeded ... | 00:13 |
| jascott1 | Failed to establish a new connection: [Errno 110] Connection timed out' | 00:13 |
| jascott1 | that name is resolvable from containers | 00:13 |
| rwellum | Is that with weave? | 00:13 |
| jascott1 | canal | 00:13 |
| jascott1 | oh that url doesnt have port | 00:14 |
| jascott1 | weird | 00:14 |
| rwellum | Hmmm yeah haven't seen that before I think. | 00:16 |
| *** mnasiadka has joined #openstack-kolla | 00:19 | |
| *** threestrands has quit IRC | 00:21 | |
| *** yangyapeng has quit IRC | 00:22 | |
| *** kbaegis has joined #openstack-kolla | 00:22 | |
| *** salv-orlando has joined #openstack-kolla | 00:23 | |
| *** mnasiadka has quit IRC | 00:24 | |
| *** Pavo has quit IRC | 00:24 | |
| *** salv-orlando has quit IRC | 00:27 | |
| *** dklyle has joined #openstack-kolla | 00:29 | |
| *** david-lyle has quit IRC | 00:30 | |
| *** Pavo has joined #openstack-kolla | 00:31 | |
| *** mnasiadka has joined #openstack-kolla | 00:34 | |
| *** duonghq has joined #openstack-kolla | 00:37 | |
| *** jtriley has joined #openstack-kolla | 00:38 | |
| *** mnasiadka has quit IRC | 00:38 | |
| *** genek has joined #openstack-kolla | 00:38 | |
| *** ntpttr_laptop has quit IRC | 00:46 | |
| *** mnasiadka has joined #openstack-kolla | 00:48 | |
| *** tovin07_ has joined #openstack-kolla | 00:51 | |
| *** zshi has quit IRC | 00:51 | |
| *** zshi has joined #openstack-kolla | 00:52 | |
| *** Pavo has quit IRC | 00:52 | |
| *** mnasiadka has quit IRC | 00:53 | |
| *** yangyapeng has joined #openstack-kolla | 00:55 | |
| *** david-lyle has joined #openstack-kolla | 00:55 | |
| *** dklyle has quit IRC | 00:55 | |
| *** yangyapeng has quit IRC | 00:59 | |
| *** david-lyle has quit IRC | 01:01 | |
| *** tvignaud has quit IRC | 01:02 | |
| *** yangyapeng has joined #openstack-kolla | 01:04 | |
| *** mnasiadka has joined #openstack-kolla | 01:04 | |
| *** mnasiadka has quit IRC | 01:08 | |
| *** tvignaud has joined #openstack-kolla | 01:12 | |
| *** rhallisey has quit IRC | 01:14 | |
| *** mnasiadka has joined #openstack-kolla | 01:19 | |
| *** mnasiadka has quit IRC | 01:23 | |
| *** salv-orlando has joined #openstack-kolla | 01:24 | |
| *** caowei has joined #openstack-kolla | 01:26 | |
| *** yuanying_ has quit IRC | 01:29 | |
| *** salv-orlando has quit IRC | 01:29 | |
| *** yuanying has joined #openstack-kolla | 01:29 | |
| masuberu | good morning, could someone please confirm if kolla containers are deployed in networking host mode in kolla-ansible? thank you | 01:32 |
| jascott1 | masuberu if you have a deployment you can inspect them to be sure | 01:33 |
| *** yuanying_ has joined #openstack-kolla | 01:33 | |
| *** mnasiadka has joined #openstack-kolla | 01:33 | |
| *** daidv has joined #openstack-kolla | 01:34 | |
| *** yuanying has quit IRC | 01:37 | |
| *** mnasiadka has quit IRC | 01:38 | |
| *** kiennt26 has joined #openstack-kolla | 01:47 | |
| *** jbrooks has quit IRC | 01:48 | |
| *** mnasiadka has joined #openstack-kolla | 01:48 | |
| *** mnasiadka has quit IRC | 01:53 | |
| *** zhangfei has joined #openstack-kolla | 02:03 | |
| *** mnasiadka has joined #openstack-kolla | 02:03 | |
| *** rumors has quit IRC | 02:07 | |
| *** mnasiadka has quit IRC | 02:08 | |
| *** dave-mccowan has quit IRC | 02:18 | |
| *** mnasiadka has joined #openstack-kolla | 02:18 | |
| *** mnasiadka has quit IRC | 02:23 | |
| *** salv-orlando has joined #openstack-kolla | 02:24 | |
| *** emccormick has joined #openstack-kolla | 02:25 | |
| *** salv-orlando has quit IRC | 02:29 | |
| *** mnasiadka has joined #openstack-kolla | 02:33 | |
| *** mnasiadka has quit IRC | 02:38 | |
| *** kevinz has quit IRC | 02:39 | |
| *** dave-mccowan has joined #openstack-kolla | 02:45 | |
| *** zhurong has joined #openstack-kolla | 02:46 | |
| *** mnasiadka has joined #openstack-kolla | 02:48 | |
| Jeffrey4l | diablo_rojo, roger. thanks a lot | 02:50 |
| *** mnasiadka has quit IRC | 02:53 | |
| *** threestrands has joined #openstack-kolla | 02:55 | |
| *** rumors has joined #openstack-kolla | 03:00 | |
| *** genek has quit IRC | 03:01 | |
| *** masber has joined #openstack-kolla | 03:03 | |
| *** masuberu has quit IRC | 03:07 | |
| *** unicell has joined #openstack-kolla | 03:09 | |
| *** diablo_rojo has quit IRC | 03:13 | |
| *** unicell has quit IRC | 03:13 | |
| spsurya | morning guys | 03:13 |
| *** absubram has quit IRC | 03:17 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla master: Change more repo url to infra mirror https://review.openstack.org/522318 | 03:24 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla master: Add squid cache proxy to speed up build https://review.openstack.org/521450 | 03:24 |
| *** salv-orlando has joined #openstack-kolla | 03:25 | |
| Jeffrey4l | morning spsurya | 03:25 |
| Jeffrey4l | could you review https://review.openstack.org/518797 spsurya duonghq ;D | 03:26 |
| Jeffrey4l | btw, which timezone do you locate spsurya ? | 03:26 |
| spsurya | India | 03:26 |
| spsurya | looking into it | 03:26 |
| Jeffrey4l | ah, not very far :D | 03:26 |
| spsurya | yeah just reached ofc | 03:27 |
| *** salv-orlando has quit IRC | 03:30 | |
| *** masber has quit IRC | 03:30 | |
| *** masber has joined #openstack-kolla | 03:34 | |
| *** unicell has joined #openstack-kolla | 03:42 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla master: Change more repo url to infra mirror https://review.openstack.org/522318 | 03:45 |
| *** zhurong has quit IRC | 03:46 | |
| *** unicell has quit IRC | 03:47 | |
| spsurya | Jeffrey4l: done | 03:50 |
| Jeffrey4l | thanks. | 03:50 |
| *** yingjun has joined #openstack-kolla | 03:57 | |
| *** Pavo has joined #openstack-kolla | 03:58 | |
| *** genek has joined #openstack-kolla | 04:06 | |
| *** dave-mccowan has quit IRC | 04:07 | |
| openstackgerrit | Merged openstack/kolla-ansible master: Use openstack_service_workers variable for gnocchi metricd workers https://review.openstack.org/523707 | 04:10 |
| *** Pavo has quit IRC | 04:17 | |
| *** coolsvap has joined #openstack-kolla | 04:20 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible stable/pike: Use openstack_service_workers variable for gnocchi metricd workers https://review.openstack.org/524061 | 04:21 |
| *** mdnadeem has joined #openstack-kolla | 04:21 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla master: Add squid cache proxy to speed up build https://review.openstack.org/521450 | 04:21 |
| *** salv-orlando has joined #openstack-kolla | 04:26 | |
| *** salv-orlando has quit IRC | 04:30 | |
| *** Jeffrey4l has quit IRC | 04:42 | |
| *** caowei has quit IRC | 04:44 | |
| llua | after enabling an service, say cinder; all i have to do is run kolla-ansible deploy again right? or is there additional steps | 04:48 |
| *** hrw has quit IRC | 04:49 | |
| *** lukepatrick has joined #openstack-kolla | 04:51 | |
| *** zhangfei has quit IRC | 04:52 | |
| *** skramaja has joined #openstack-kolla | 04:56 | |
| *** xinliang has quit IRC | 05:00 | |
| *** xinliang has joined #openstack-kolla | 05:01 | |
| chason | llua Yeah, no more steps you need. | 05:01 |
| chason | llua suggest to run kolla-ansible deploy -t cinder | 05:01 |
| *** hrw has joined #openstack-kolla | 05:02 | |
| openstackgerrit | Merged openstack/kolla stable/pike: Dockerhub publisher job https://review.openstack.org/518797 | 05:03 |
| *** Jeffrey4l has joined #openstack-kolla | 05:04 | |
| *** hrw has quit IRC | 05:06 | |
| llua | chason: thanks | 05:07 |
| *** hrw has joined #openstack-kolla | 05:08 | |
| *** zhangfei has joined #openstack-kolla | 05:08 | |
| chason | llua yw :) | 05:10 |
| *** janonymous has joined #openstack-kolla | 05:18 | |
| *** janki has joined #openstack-kolla | 05:21 | |
| *** caowei has joined #openstack-kolla | 05:23 | |
| *** lukepatrick has quit IRC | 05:24 | |
| *** salv-orlando has joined #openstack-kolla | 05:27 | |
| *** salv-orlando has quit IRC | 05:32 | |
| *** kiennt26 has quit IRC | 05:40 | |
| *** mikal has quit IRC | 05:47 | |
| *** absubram has joined #openstack-kolla | 05:47 | |
| *** absubram_ has joined #openstack-kolla | 05:50 | |
| *** absubram has quit IRC | 05:52 | |
| *** absubram_ is now known as absubram | 05:52 | |
| *** mikal has joined #openstack-kolla | 05:55 | |
| *** pcaruana has joined #openstack-kolla | 06:05 | |
| *** mikal_ has joined #openstack-kolla | 06:06 | |
| *** mdnadeem_ has joined #openstack-kolla | 06:07 | |
| *** mdnadeem has quit IRC | 06:09 | |
| *** mikal has quit IRC | 06:10 | |
| *** kiennt26 has joined #openstack-kolla | 06:12 | |
| *** salv-orlando has joined #openstack-kolla | 06:12 | |
| *** jbadiapa has quit IRC | 06:15 | |
| *** pcaruana has quit IRC | 06:16 | |
| *** mnasiadka has joined #openstack-kolla | 06:27 | |
| *** zhangfei has quit IRC | 06:32 | |
| *** rwsu has joined #openstack-kolla | 06:33 | |
| *** ipsecguy has quit IRC | 06:40 | |
| *** ipsecguy has joined #openstack-kolla | 06:40 | |
| *** unicell has joined #openstack-kolla | 06:41 | |
| *** xinliang has quit IRC | 06:41 | |
| *** masuberu has joined #openstack-kolla | 06:42 | |
| *** zhangfei has joined #openstack-kolla | 06:45 | |
| *** mikal_ has quit IRC | 06:45 | |
| *** masber has quit IRC | 06:46 | |
| *** mikal has joined #openstack-kolla | 06:47 | |
| *** unicell1 has joined #openstack-kolla | 06:48 | |
| *** unicell has quit IRC | 06:49 | |
| *** threestrands has quit IRC | 06:50 | |
| *** xinliang has joined #openstack-kolla | 06:54 | |
| *** cah_link has joined #openstack-kolla | 06:54 | |
| *** dciabrin_ has joined #openstack-kolla | 07:04 | |
| *** dciabrin__ has quit IRC | 07:05 | |
| *** athomas has joined #openstack-kolla | 07:16 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla-ansible stable/pike: Pull image from dockerhub rather than tarballs.o.o site https://review.openstack.org/521701 | 07:17 |
| *** david-lyle has joined #openstack-kolla | 07:19 | |
| *** rwsu has quit IRC | 07:22 | |
| *** gkadam_ has quit IRC | 07:24 | |
| *** egonzalez has joined #openstack-kolla | 07:26 | |
| *** rwsu has joined #openstack-kolla | 07:35 | |
| *** mdnadeem_ has quit IRC | 07:37 | |
| *** jbadiapa has joined #openstack-kolla | 07:39 | |
| *** mdnadeem has joined #openstack-kolla | 07:44 | |
| *** egonzalez has quit IRC | 07:54 | |
| *** lvdombrkr has quit IRC | 07:54 | |
| *** pcaruana has joined #openstack-kolla | 07:55 | |
| *** b_bezak has joined #openstack-kolla | 08:04 | |
| *** duonghq has quit IRC | 08:08 | |
| openstackgerrit | Merged openstack/kolla-ansible stable/pike: Pull image from dockerhub rather than tarballs.o.o site https://review.openstack.org/521701 | 08:09 |
| *** jaosorior has quit IRC | 08:13 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla stable/pike: Improve gate post jobs https://review.openstack.org/524093 | 08:14 |
| *** kiennt26 has quit IRC | 08:21 | |
| *** genek has quit IRC | 08:22 | |
| *** genek has joined #openstack-kolla | 08:23 | |
| *** kiennt26 has joined #openstack-kolla | 08:25 | |
| *** genek has quit IRC | 08:32 | |
| *** jaosorior has joined #openstack-kolla | 08:34 | |
| *** dardelean has joined #openstack-kolla | 08:43 | |
| *** egonzalez has joined #openstack-kolla | 08:45 | |
| *** gkadam_ has joined #openstack-kolla | 08:46 | |
| *** gfidente has joined #openstack-kolla | 08:48 | |
| *** amoralej|off is now known as amoralej | 08:50 | |
| daidv | Hi Jeffrey4l | 09:02 |
| Jeffrey4l | daidv, yep | 09:06 |
| daidv | I hear that you asked duonghq about default policy file which is read by oslo.policy and I want to confirm this. | 09:07 |
| daidv | the default file is still policy.json file | 09:07 |
| daidv | #link https://github.com/openstack/oslo.policy/blob/master/oslo_policy/opts.py#L29 | 09:08 |
| daidv | #link https://github.com/openstack/oslo.policy/blob/master/oslo_policy/policy.py#L473 | 09:08 |
| Jeffrey4l | i see, thanks. | 09:09 |
| *** mwall has joined #openstack-kolla | 09:09 | |
| daidv | btw, can you review the first change for me? https://review.openstack.org/#/c/514565/ | 09:10 |
| daidv | After that, I will do the same thing for other projects to complete our blueprint ASAP meanwhile TC goal also will complete soon. | 09:11 |
| *** ktibi has joined #openstack-kolla | 09:11 | |
| *** jascott1 has quit IRC | 09:11 | |
| Jeffrey4l | daidv, ok. i will. but need review it later. | 09:12 |
| *** jascott1 has joined #openstack-kolla | 09:12 | |
| daidv | Jeffrey4l, yep, thank you in advance. :D | 09:12 |
| Jeffrey4l | np ;D | 09:13 |
| *** jascott1 has quit IRC | 09:16 | |
| *** zhangfei has quit IRC | 09:36 | |
| *** yingjun has quit IRC | 09:38 | |
| *** EmilienM has quit IRC | 09:39 | |
| *** pcaruana has quit IRC | 09:39 | |
| *** pcaruana has joined #openstack-kolla | 09:40 | |
| *** EmilienM has joined #openstack-kolla | 09:42 | |
| *** mgoddard has joined #openstack-kolla | 09:42 | |
| *** kbyrne has quit IRC | 09:43 | |
| *** magicboiz has quit IRC | 09:44 | |
| *** kbyrne has joined #openstack-kolla | 09:44 | |
| *** k_mouza has joined #openstack-kolla | 09:45 | |
| *** chason has quit IRC | 09:45 | |
| *** chason has joined #openstack-kolla | 09:45 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla stable/pike: Improve gate post jobs https://review.openstack.org/524093 | 09:45 |
| *** magicboiz has joined #openstack-kolla | 09:45 | |
| *** david-lyle has quit IRC | 09:46 | |
| k_mouza | good morning all! I've a restarting fluentd container with these logs: http://paste.openstack.org/show/627823/ | 09:47 |
| k_mouza | I've deployed with kolla-ansible stable/ocata (4.0.3) | 09:48 |
| k_mouza | any ideas please? egonzalez maybe? Thanks a lot! | 09:48 |
| *** zhangfei has joined #openstack-kolla | 09:49 | |
| *** magicboiz has quit IRC | 09:53 | |
| *** coolsvap has quit IRC | 09:59 | |
| *** magicboiz has joined #openstack-kolla | 10:00 | |
| *** sebastien__ has quit IRC | 10:04 | |
| egonzalez | k_mouza, is being backported the fix, fluentd version changed and broke our deployment | 10:04 |
| *** egonzalez has quit IRC | 10:08 | |
| *** egonzalez_afk has joined #openstack-kolla | 10:08 | |
| *** k_mouza has quit IRC | 10:09 | |
| openstackgerrit | Merged openstack/kolla master: Implement nova-mksproxy https://review.openstack.org/512346 | 10:09 |
| *** janonymous has quit IRC | 10:09 | |
| *** hieulq has quit IRC | 10:09 | |
| *** kklimonda has quit IRC | 10:09 | |
| *** kong has quit IRC | 10:09 | |
| *** berendt has quit IRC | 10:09 | |
| *** zhenguo has quit IRC | 10:09 | |
| *** dgonzalez has quit IRC | 10:09 | |
| *** spsurya has quit IRC | 10:09 | |
| *** sgordon has quit IRC | 10:09 | |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla master: Change more repo url to infra mirror https://review.openstack.org/522318 | 10:14 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla master: Add squid cache proxy to speed up build https://review.openstack.org/521450 | 10:15 |
| *** janonymous has joined #openstack-kolla | 10:15 | |
| *** hieulq has joined #openstack-kolla | 10:15 | |
| *** kklimonda has joined #openstack-kolla | 10:15 | |
| *** berendt has joined #openstack-kolla | 10:15 | |
| *** kong has joined #openstack-kolla | 10:15 | |
| *** zhenguo has joined #openstack-kolla | 10:15 | |
| *** dgonzalez has joined #openstack-kolla | 10:15 | |
| *** spsurya has joined #openstack-kolla | 10:15 | |
| *** sgordon has joined #openstack-kolla | 10:15 | |
| *** k_mouza has joined #openstack-kolla | 10:15 | |
| *** tovin07_ has quit IRC | 10:17 | |
| *** calbers has quit IRC | 10:18 | |
| *** kiennt26 has quit IRC | 10:21 | |
| *** calbers has joined #openstack-kolla | 10:22 | |
| openstackgerrit | Merged openstack/kolla-ansible stable/pike: Use openstack_service_workers variable for gnocchi metricd workers https://review.openstack.org/524061 | 10:29 |
| openstackgerrit | Michael Still proposed openstack/kolla-kubernetes master: Fixups to deployment docs. https://review.openstack.org/524125 | 10:31 |
| *** daidv has quit IRC | 10:35 | |
| *** salv-orlando has quit IRC | 10:36 | |
| hrw | morning | 10:38 |
| hrw | can someone look at https://review.openstack.org/#/c/519371/ one so we can finally build mongodb container for Debian? | 10:38 |
| *** pratapagoutham has joined #openstack-kolla | 10:39 | |
| hrw | or https://review.openstack.org/#/c/520059/ which is backport to stable/pike - also for Debian stuff | 10:40 |
| *** salv-orlando has joined #openstack-kolla | 10:48 | |
| *** masuberu has quit IRC | 10:49 | |
| ktibi | Hi Jeffrey4l | 10:54 |
| ktibi | I have issue with gnocchi and ceph, traceback loop (500Mb log file since 2 days) | 10:55 |
| ktibi | Jeffrey4l http://paste.openstack.org/show/627831/ | 10:56 |
| ktibi | I used last change https://review.openstack.org/#/c/523272/ | 10:57 |
| *** caowei has quit IRC | 10:57 | |
| k_mouza | thanks egonzalez. So it's not ready yet for ocata? I have the fluentd image build about a month ago and it was working fine in a deployment I did a couple of weeks ago. The same image now doesn't work | 10:58 |
| *** zhangfei has quit IRC | 11:03 | |
| Jeffrey4l | ktibi, is this a new deployment env or upgrade from exist one. | 11:05 |
| ktibi | new | 11:06 |
| *** k_mouza has quit IRC | 11:06 | |
| *** dardelean has quit IRC | 11:07 | |
| *** dardelean has joined #openstack-kolla | 11:07 | |
| Jeffrey4l | ktibi, which branch are you using? ktibi | 11:08 |
| Jeffrey4l | try run "gnocchi-upgrade" first and check whether it could fix this issue. | 11:08 |
| Jeffrey4l | in my new deployment env ( ocata and pike) it works. | 11:08 |
| *** nhlfr has quit IRC | 11:13 | |
| *** retr0h has quit IRC | 11:13 | |
| *** bachp has quit IRC | 11:14 | |
| ktibi | Jefrey4l where? in gnocchi container ? | 11:17 |
| *** salv-orlando has quit IRC | 11:18 | |
| ktibi | Jeffrey4l ok trace is gone. but logs a lot of http://paste.openstack.org/show/627835/ | 11:19 |
| Jeffrey4l | ktibi, seem OK. | 11:19 |
| Jeffrey4l | except the warning which is another issue. but it won't break anything. | 11:20 |
| Jeffrey4l | ktibi, what did you do? | 11:20 |
| ktibi | docker exec gnocchi_api gnocchi-upgrade | 11:20 |
| ktibi | kolla-ansible run gnocchi upgrade ? https://github.com/openstack/kolla-ansible/search?utf8=%E2%9C%93&q=gnocchi-upgrade&type= | 11:21 |
| Jeffrey4l | ktibi, ok. kolla-ansible should run this. | 11:21 |
| Jeffrey4l | ktibi, could you try to deploy again? this shouldn't be happen. | 11:22 |
| ktibi | ok I'll try tonight | 11:22 |
| Jeffrey4l | btw, which kolla image are you using? ktibi and which branch? | 11:22 |
| ktibi | stable/pike | 11:22 |
| ktibi | centos binary | 11:23 |
| Jeffrey4l | cat the filecontent in gnocchi_api:/usr/local/bin/kolla_extend_start | 11:23 |
| ktibi | yep 1m | 11:23 |
| ktibi | http://paste.openstack.org/show/627836/ | 11:23 |
| Jeffrey4l | sorry this file gnocchi_api:/usr/local/bin/kolla_gnocchi_extend_start ktibi | 11:24 |
| ktibi | http://paste.openstack.org/show/627838/ | 11:25 |
| ktibi | ok found the fail http://paste.openstack.org/show/627839/ | 11:26 |
| Jeffrey4l | seems OK. how about the gnocchi-upgrade.log file? ktibi | 11:26 |
| ktibi | TimedOut: Ioctx.read(gnocchi): failed to read gnocchi-config | 11:27 |
| Jeffrey4l | yes. but what this mean :( | 11:27 |
| ktibi | maybe the ceph cluster is not up when gnocchi try to upgrade | 11:28 |
| Jeffrey4l | impossible. | 11:29 |
| ktibi | so cradox timeout | 11:29 |
| Jeffrey4l | unless ceph-mon is ready. but no ceph osd? | 11:29 |
| ktibi | maybe | 11:29 |
| ktibi | now command works and I have no change | 11:30 |
| Jeffrey4l | ok. no idea. | 11:31 |
| ktibi | only change I need to restart docker daemon on each ceph node because I don't know why for now, but kernel freeze (df command hang) | 11:31 |
| *** hrw has quit IRC | 11:31 | |
| *** salv-orlando has joined #openstack-kolla | 11:32 | |
| *** hrw has joined #openstack-kolla | 11:33 | |
| Jeffrey4l | maybe the freeze issue | 11:33 |
| *** lvdombrkr has joined #openstack-kolla | 11:44 | |
| *** amoralej is now known as amoralej|lunch | 11:57 | |
| *** genek has joined #openstack-kolla | 12:03 | |
| *** dave-mccowan has joined #openstack-kolla | 12:07 | |
| *** coolsvap has joined #openstack-kolla | 12:10 | |
| *** hrw has quit IRC | 12:14 | |
| *** yingjun has joined #openstack-kolla | 12:19 | |
| *** dciabrin has joined #openstack-kolla | 12:24 | |
| *** retr0h has joined #openstack-kolla | 12:26 | |
| *** jchhatbar has joined #openstack-kolla | 12:26 | |
| *** jchhatbar has quit IRC | 12:26 | |
| *** janki has quit IRC | 12:26 | |
| *** jchhatbar has joined #openstack-kolla | 12:26 | |
| *** dciabrin_ has quit IRC | 12:26 | |
| openstackgerrit | Merged openstack/kolla master: Updated from global requirements https://review.openstack.org/520422 | 12:29 |
| *** hrw has joined #openstack-kolla | 12:31 | |
| *** nhlfr has joined #openstack-kolla | 12:33 | |
| *** bachp has joined #openstack-kolla | 12:33 | |
| *** dciabrin_ has joined #openstack-kolla | 12:35 | |
| *** dciabrin has quit IRC | 12:35 | |
| *** pratapagoutham has quit IRC | 12:36 | |
| *** janonymous has quit IRC | 12:36 | |
| *** k_mouza has joined #openstack-kolla | 12:43 | |
| *** rhallisey has joined #openstack-kolla | 13:00 | |
| kbaegis | Morning everyone | 13:09 |
| *** jchhatbar has quit IRC | 13:20 | |
| *** zhangfei has joined #openstack-kolla | 13:22 | |
| *** k_mouza has quit IRC | 13:22 | |
| *** skramaja has quit IRC | 13:22 | |
| *** yangyapeng has quit IRC | 13:26 | |
| *** yangyapeng has joined #openstack-kolla | 13:27 | |
| *** sambetts|afk is now known as sambetts | 13:29 | |
| *** magicboiz has quit IRC | 13:31 | |
| *** amoralej|lunch is now known as amoralej | 13:31 | |
| *** goldyfruit has joined #openstack-kolla | 13:32 | |
| *** yangyapeng has quit IRC | 13:36 | |
| *** magicboiz has joined #openstack-kolla | 13:39 | |
| *** k_mouza has joined #openstack-kolla | 13:40 | |
| *** janki has joined #openstack-kolla | 13:55 | |
| *** magicboiz has quit IRC | 13:56 | |
| *** jbadiapa has quit IRC | 13:58 | |
| ktibi | Hi all, I think I have an issue with docker but I don't know if it's a docker or kolla probleme. After fresh deploy, I can't run command "df". I don't know why but I have issue with /proc/sys/fs/binfmt_misc ! For resolv I need to restart docker deamon. | 13:59 |
| *** jbadiapa has joined #openstack-kolla | 14:05 | |
| *** magicboiz has joined #openstack-kolla | 14:06 | |
| *** egonzalez_afk has quit IRC | 14:10 | |
| *** lukepatrick has joined #openstack-kolla | 14:12 | |
| *** jtriley has quit IRC | 14:13 | |
| *** mdnadeem has quit IRC | 14:16 | |
| *** coolsvap has quit IRC | 14:19 | |
| *** jbadiapa has quit IRC | 14:21 | |
| *** janki has quit IRC | 14:21 | |
| *** janki has joined #openstack-kolla | 14:22 | |
| *** janki has quit IRC | 14:22 | |
| *** janki has joined #openstack-kolla | 14:22 | |
| *** dciabrin_ has quit IRC | 14:26 | |
| *** dciabrin_ has joined #openstack-kolla | 14:27 | |
| *** zhangfei has quit IRC | 14:30 | |
| *** affie has joined #openstack-kolla | 14:36 | |
| *** yingjun has quit IRC | 14:45 | |
| *** jtriley has joined #openstack-kolla | 14:47 | |
| *** lukepatrick has quit IRC | 14:47 | |
| *** jamesbenson has joined #openstack-kolla | 14:52 | |
| *** hongbin has joined #openstack-kolla | 14:59 | |
| *** jrist has quit IRC | 15:08 | |
| *** jrist has joined #openstack-kolla | 15:09 | |
| *** janonymous has joined #openstack-kolla | 15:14 | |
| *** jamesbenson has quit IRC | 15:15 | |
| *** yingjun has joined #openstack-kolla | 15:16 | |
| *** magicboiz has quit IRC | 15:16 | |
| *** magicboiz has joined #openstack-kolla | 15:17 | |
| *** lukepatrick has joined #openstack-kolla | 15:23 | |
| *** lukepatrick has quit IRC | 15:24 | |
| -openstackstatus- NOTICE: if you receieved a result of "RETRY_LIMIT" after 14:15 UTC, it was likely due to an error since corrected. please "recheck" | 15:37 | |
| *** k_mouza has quit IRC | 15:39 | |
| *** k_mouza has joined #openstack-kolla | 15:40 | |
| *** jbadiapa has joined #openstack-kolla | 15:48 | |
| sdake | ktibi havey ou deployed kolla when this problem occurs? | 15:48 |
| ktibi | I think it's issue between docker and centos 7.4 | 15:48 |
| ktibi | sdake I'll try to deploy kolla on centos 7.3 | 15:49 |
| sdake | ktibi this problem doesn't happen before deploy? | 15:49 |
| sdake | ktibi how much diskspace do you ahve ? | 15:49 |
| *** janki has quit IRC | 15:49 | |
| *** mdnadeem has joined #openstack-kolla | 15:51 | |
| ktibi | sdake I have a lot of space (1To) SSD. After deploy, I can use df, I need to restart docker deamon and df works | 15:58 |
| ktibi | but if I reboot node, df no works, need again to restart docker deamo,n | 15:59 |
| ktibi | after deploy I can't use df* | 15:59 |
| sdake | ktibi are you using red hat's docker version or upstream docker? | 16:02 |
| sdake | inc0 any dieas? ^^ | 16:02 |
| *** mwall has quit IRC | 16:02 | |
| *** b_bezak has quit IRC | 16:02 | |
| *** b_bezak has joined #openstack-kolla | 16:03 | |
| ktibi | sdake Docker version 1.12.0, build 8eab29e | 16:04 |
| ktibi | from docker-repo from https://yum.dockerproject.org/repo/main/centos/7 | 16:05 |
| sdake | ktibi what storage backend? | 16:05 |
| ktibi | devicemapper direct lvm | 16:06 |
| openstackgerrit | Jeffrey Zhang proposed openstack/kolla stable/ocata: Improve gate post jobs https://review.openstack.org/524227 | 16:06 |
| *** b_bezak has quit IRC | 16:07 | |
| *** lukepatrick has joined #openstack-kolla | 16:08 | |
| sdake | ktibi not sure - however i have meeting now so cna't help further | 16:08 |
| ktibi | no pb ;) | 16:08 |
| ktibi | I'll try centos 7.3 for to be sure | 16:09 |
| *** magicboiz has quit IRC | 16:09 | |
| *** jbrooks has joined #openstack-kolla | 16:11 | |
| *** magicboiz has joined #openstack-kolla | 16:15 | |
| *** magicboiz has quit IRC | 16:19 | |
| *** magicboiz has joined #openstack-kolla | 16:20 | |
| *** dciabrin has joined #openstack-kolla | 16:23 | |
| *** mnasiadka has quit IRC | 16:24 | |
| *** dciabrin_ has quit IRC | 16:25 | |
| *** mnasiadka has joined #openstack-kolla | 16:25 | |
| *** phuongnh has joined #openstack-kolla | 16:27 | |
| kfox1111 | morning. | 16:27 |
| sdake | sup peeps | 16:27 |
| sdake | sup kfox1111 :) | 16:27 |
| sdake | meeting - bbl :) | 16:27 |
| k_mouza | hello all! | 16:28 |
| k_mouza | I'm trying to deploy ceph (4 storage nodes) but I'm getting this error from ansible: http://paste.openstack.org/show/627891/ | 16:28 |
| *** dciabrin has quit IRC | 16:29 | |
| k_mouza | I've set up storage, api and tunnel interface in the inventory file correctly on the storage nodes | 16:29 |
| k_mouza | not sure why I'm getting this | 16:29 |
| k_mouza | it's on kolla-ansible stable/ocata. Tries with both ansible 2.4.2 and 2.4.0 | 16:30 |
| *** mnasiadka has quit IRC | 16:30 | |
| k_mouza | it's on kolla-ansible stable/ocata. Tried with both ansible 2.4.2 and 2.4.0 | 16:30 |
| k_mouza | anyone seen something similar recently? | 16:30 |
| *** dciabrin has joined #openstack-kolla | 16:31 | |
| *** mnasiadka has joined #openstack-kolla | 16:33 | |
| *** mnasiadka has quit IRC | 16:38 | |
| *** jtriley has quit IRC | 16:40 | |
| *** jtriley has joined #openstack-kolla | 16:45 | |
| *** mnasiadka has joined #openstack-kolla | 16:45 | |
| *** yingjun has quit IRC | 16:46 | |
| *** unicell has joined #openstack-kolla | 16:46 | |
| *** unicell1 has quit IRC | 16:48 | |
| *** mnasiadka has quit IRC | 16:52 | |
| *** mnasiadka has joined #openstack-kolla | 16:55 | |
| *** jamesbenson has joined #openstack-kolla | 16:57 | |
| *** david-lyle has joined #openstack-kolla | 16:59 | |
| *** mnasiadka has quit IRC | 17:00 | |
| *** jbrooks has quit IRC | 17:02 | |
| *** TxGirlGeek has joined #openstack-kolla | 17:03 | |
| *** mnasiadka has joined #openstack-kolla | 17:03 | |
| *** jbrooks has joined #openstack-kolla | 17:05 | |
| *** k_mouza has quit IRC | 17:06 | |
| *** pcaruana has quit IRC | 17:08 | |
| *** mnasiadka has quit IRC | 17:10 | |
| *** dardelean has quit IRC | 17:15 | |
| *** mnasiadka has joined #openstack-kolla | 17:16 | |
| *** dardelean has joined #openstack-kolla | 17:19 | |
| *** kbaegis has quit IRC | 17:19 | |
| *** dardelean has quit IRC | 17:20 | |
| *** mnasiadka has quit IRC | 17:22 | |
| *** gkadam_ has quit IRC | 17:22 | |
| *** jlvacation is now known as jlvillal | 17:25 | |
| *** mnasiadka has joined #openstack-kolla | 17:25 | |
| *** ntpttr_laptop has joined #openstack-kolla | 17:29 | |
| *** mnasiadka has quit IRC | 17:30 | |
| *** mnasiadka has joined #openstack-kolla | 17:33 | |
| *** phuongnh has quit IRC | 17:36 | |
| *** jtriley has quit IRC | 17:37 | |
| *** mnasiadka has quit IRC | 17:40 | |
| *** jtriley has joined #openstack-kolla | 17:42 | |
| *** Pavo has joined #openstack-kolla | 17:44 | |
| *** mnasiadka has joined #openstack-kolla | 17:45 | |
| *** yangyapeng has joined #openstack-kolla | 17:49 | |
| *** mnasiadka has quit IRC | 17:52 | |
| *** mdnadeem has quit IRC | 17:52 | |
| *** yangyapeng has quit IRC | 17:54 | |
| *** mnasiadka has joined #openstack-kolla | 17:56 | |
| *** goldyfruit has quit IRC | 17:57 | |
| *** cah_link has quit IRC | 17:58 | |
| *** ktibi has quit IRC | 18:00 | |
| *** mnasiadka has quit IRC | 18:00 | |
| *** absubram has quit IRC | 18:01 | |
| *** unicell has quit IRC | 18:02 | |
| *** mnasiadka has joined #openstack-kolla | 18:03 | |
| *** dardelean has joined #openstack-kolla | 18:04 | |
| *** Pavo has quit IRC | 18:04 | |
| *** mnasiadka has quit IRC | 18:10 | |
| *** mnasiadka has joined #openstack-kolla | 18:15 | |
| rwellum | Morning kfox1111 - for a laugh I got weave working with k8s this morning. | 18:17 |
| inc0 | rwellum: for a laugh we were just now discussing why weave won't work with kolla-k8s;) | 18:21 |
| inc0 | and I mean we were discussing it *right now*;) | 18:21 |
| inc0 | so your time is imppecable | 18:22 |
| rwellum | Ha! :) | 18:22 |
| *** diablo_rojo has joined #openstack-kolla | 18:22 | |
| inc0 | it seems weave uses parts of ovs underneath | 18:22 |
| *** mnasiadka has quit IRC | 18:22 | |
| inc0 | and that conflicts with kolla-k8s ovs | 18:22 |
| inc0 | Justin ran into it yesterday | 18:23 |
| inc0 | but more research required | 18:23 |
| rwellum | That's why I decided to try it out. | 18:23 |
| rwellum | Because of Justin | 18:23 |
| inc0 | ahh ok | 18:23 |
| inc0 | well | 18:23 |
| inc0 | if you get to work ovs in containers with it | 18:23 |
| inc0 | that means we're missing sth | 18:23 |
| rwellum | About 20m I should know definitely. I got it working but forgot to patch in my libvirt fix, so re-doing. | 18:24 |
| rwellum | So Justin got k8s with weave up but not Openstack? | 18:25 |
| *** mnasiadka has joined #openstack-kolla | 18:25 | |
| kfox1111 | cool. am curious to see the results. :) | 18:26 |
| *** goldyfruit has joined #openstack-kolla | 18:27 | |
| rwellum | I was reading this: http://chunqi.li/2015/11/15/Battlefield-Calico-Flannel-Weave-and-Docker-Overlay-Network/ - while waiting. It's a bit out of date but an interesting comparison. | 18:28 |
| *** affie has quit IRC | 18:29 | |
| *** mnasiadka has quit IRC | 18:30 | |
| rwellum | Of course since this article fnallen and calico have merged into canal. | 18:30 |
| *** goldyfruit has quit IRC | 18:31 | |
| rwellum | (flannel) | 18:32 |
| *** mnasiadka has joined #openstack-kolla | 18:33 | |
| kfox1111 | rwellum: interesting article. thanks for sharing. :) | 18:34 |
| kfox1111 | merged might be too strong a word. | 18:34 |
| kfox1111 | as I don't think flannal or calico has stoped supporting standalone modes. | 18:35 |
| *** gfidente is now known as gfidente|afk | 18:35 | |
| kfox1111 | more of, there's a 3rd option, canal based on the other 2. | 18:35 |
| *** mnasiadka has quit IRC | 18:42 | |
| *** absubram has joined #openstack-kolla | 18:43 | |
| rwellum | kfox1111: inc0 - so I got an operational kolla-k8s openstack using weave running. Only issue I have is I am getting a 'No valid host was found. There are not enough hosts available.' - when I try to create an instance - but I don't think that's networking related. | 18:45 |
| kolla-slack | <inc0> It might | 18:45 |
| *** mnasiadka has joined #openstack-kolla | 18:45 | |
| kolla-slack | <inc0> Check nova compute logs | 18:45 |
| rwellum | Yup will do. | 18:46 |
| rwellum | But is this where Justin got stuck? | 18:46 |
| *** kbaegis has joined #openstack-kolla | 18:46 | |
| *** openstackgerrit has quit IRC | 18:48 | |
| kfox1111 | rwellum: getting that from what? nova when launching a vm? | 18:48 |
| rwellum | Yes | 18:48 |
| rwellum | To both | 18:49 |
| rwellum | https://www.irccloud.com/pastebin/Z7jvbyiZ/ | 18:49 |
| kfox1111 | nova hypervisor-list | 18:49 |
| *** genek has quit IRC | 18:49 | |
| kfox1111 | do you see active hypervisors? | 18:49 |
| rwellum | https://www.irccloud.com/pastebin/UCO6su7A/ | 18:50 |
| *** pcaruana has joined #openstack-kolla | 18:50 | |
| kfox1111 | anything interesting in the logs for nova-scheduler? | 18:52 |
| *** mnasiadka has quit IRC | 18:52 | |
| *** mnasiadka has joined #openstack-kolla | 18:55 | |
| rwellum | I'm just starting to trawl through them. I wish these logs were easier to interpret! :( | 18:57 |
| *** gkadam has joined #openstack-kolla | 18:58 | |
| *** mnasiadka has quit IRC | 19:00 | |
| kfox1111 | heh. you and me both brother. :) | 19:02 |
| kfox1111 | and all the other folks I work with. ;) | 19:02 |
| rwellum | We need to stream the logs to an ELK container or something... | 19:02 |
| kfox1111 | +1. | 19:02 |
| kfox1111 | I have a fair chunk of that in a ps. | 19:02 |
| kfox1111 | I tried running the elasticsearch-operator a coule weeks ago. | 19:03 |
| kfox1111 | ran into a snag. | 19:03 |
| kfox1111 | the dev said he thought it might be fixed now, but havent had time to go try it again. | 19:03 |
| kfox1111 | if you have a bit of time to try it, that would be great. | 19:03 |
| *** mnasiadka has joined #openstack-kolla | 19:03 | |
| kfox1111 | should greatly simplify managing EK. | 19:04 |
| rwellum | Can you give me a pointer to that operator? | 19:04 |
| rwellum | https://github.com/upmc-enterprises/elasticsearch-operator ? | 19:06 |
| *** goldyfruit has joined #openstack-kolla | 19:07 | |
| kfox1111 | yeah. thats the one. | 19:07 |
| kfox1111 | you load the operator, you give it a doc saying give me an EK and your good to go (if it works) | 19:08 |
| *** amoralej is now known as amoralej|off | 19:08 | |
| rwellum | I just kicked it off and it came up into my cluster | 19:09 |
| *** mnasiadka has quit IRC | 19:09 | |
| rwellum | Now trying to figure out how to use it :) | 19:10 |
| *** dardelean has quit IRC | 19:10 | |
| kfox1111 | the operator or a cluster? | 19:11 |
| *** jascott1 has joined #openstack-kolla | 19:11 | |
| rwellum | The operator. | 19:11 |
| jascott1 | rwellum im listening :) | 19:12 |
| rwellum | :) | 19:12 |
| kfox1111 | rwellum: https://pastebin.com/wEZ6QkgR | 19:12 |
| kfox1111 | something like that --^ | 19:12 |
| rwellum | I got weave up jascott1 on pike kolla-k8s - issue with nova hosts still looking at. | 19:13 |
| jascott1 | ovs doesnt break name resolution? | 19:13 |
| SamYaple | jascott1: not by design. without knowing more about your setup, i dont really have an answer to your issues :( | 19:15 |
| *** mnasiadka has joined #openstack-kolla | 19:15 | |
| jascott1 | SamYapple if I change to canal it works. we were theorizing that weave is breaking because it already uses OVS internally | 19:16 |
| rwellum | jascott1: what name resolution is broken? | 19:22 |
| *** mnasiadka has quit IRC | 19:22 | |
| *** mnasiadka has joined #openstack-kolla | 19:25 | |
| jascott1 | container resolution breaks for me when using weave and installing ovs | 19:26 |
| jascott1 | i stopped before installing ovs, exec'd into container and could nslooup things like kubernetes.default and mariadb. after installing ovs, nslookup within container no longer worked | 19:29 |
| *** mnasiadka has quit IRC | 19:30 | |
| *** serlex has joined #openstack-kolla | 19:33 | |
| *** mnasiadka has joined #openstack-kolla | 19:33 | |
| *** serlex has quit IRC | 19:33 | |
| *** lukepatrick has left #openstack-kolla | 19:36 | |
| *** mnasiadka has quit IRC | 19:40 | |
| *** dardelean has joined #openstack-kolla | 19:41 | |
| *** rwsu has quit IRC | 19:42 | |
| SamYaple | jascott1: so maybe its an mtu issue? | 19:42 |
| SamYaple | (ovs on ovs makes me think mtu immedialtly) | 19:43 |
| *** genek has joined #openstack-kolla | 19:45 | |
| *** yangyapeng has joined #openstack-kolla | 19:45 | |
| *** mnasiadka has joined #openstack-kolla | 19:45 | |
| jascott1 | ah good thinkin' | 19:47 |
| *** yangyapeng has quit IRC | 19:50 | |
| *** genek has quit IRC | 19:50 | |
| *** mnasiadka has quit IRC | 19:53 | |
| *** mnasiadka has joined #openstack-kolla | 19:55 | |
| *** rwsu has joined #openstack-kolla | 19:56 | |
| SamYaple | jascott1: wow. now that ive said that let me regauge whats going on here | 19:57 |
| SamYaple | so you have ovs for weave talking to the ovs kernel module | 19:57 |
| SamYaple | then you have a SECOND unrelated ovs talking to the kernel module | 19:57 |
| SamYaple | thats likely your issue | 19:57 |
| jascott1 | sure | 19:57 |
| inc0 | hey, I'm back (after fighting with corporate networking) | 19:57 |
| SamYaple | you canonly have one db per host | 19:57 |
| jascott1 | since ovs is used by weave, how would we run the one for openstack | 19:58 |
| SamYaple | you would have to hook into the weave ovs | 19:58 |
| inc0 | SamYaple: it gets better - from what I read weave wrote *their own* code to meddle with ovs flows in kernel | 19:58 |
| SamYaple | which may not be possible for weave | 19:58 |
| SamYaple | jascott1: try for giggles just exposing the weave socket to neutron | 19:59 |
| inc0 | so there is no way to even reuse ovsdb weave would be working with...cuz there is none | 19:59 |
| SamYaple | oh | 19:59 |
| *** kbaegis has quit IRC | 19:59 | |
| SamYaple | then yea you cant do what youre trying to do | 19:59 |
| SamYaple | cept maybe ovs dpdk. *maybe* | 19:59 |
| jascott1 | we should probably put a note about this somewhere | 19:59 |
| *** mnasiadka has quit IRC | 20:00 | |
| inc0 | https://github.com/weaveworks/go-odp | 20:00 |
| *** jamesbenson has quit IRC | 20:01 | |
| inc0 | they mess with flows, but they do bridging themselves...so it's double fun | 20:02 |
| SamYaple | sounds like weave makes this a no go to be honest | 20:02 |
| inc0 | I mean it's quite cool tech, just makes us hurt | 20:02 |
| SamYaple | ODL might get you what you want, sharing ovs | 20:02 |
| inc0 | or we could try linuxbridge driver | 20:02 |
| SamYaple | thats the real solution here yea | 20:02 |
| inc0 | but I'd say calico is the thing for kolla k8s | 20:03 |
| SamYaple | thats why calico is nice | 20:03 |
| *** mnasiadka has joined #openstack-kolla | 20:03 | |
| SamYaple | yup | 20:03 |
| inc0 | for more than this reason | 20:03 |
| inc0 | you can also run neutron with calico | 20:03 |
| SamYaple | k8s is a flat namespace for networking anyway | 20:03 |
| inc0 | and makes this networking thingy really elegant where vms talk to k8s | 20:03 |
| SamYaple | calico gets you what you need for k8s, ovs gets you what you need for openstack | 20:03 |
| *** gfidente|afk has quit IRC | 20:04 | |
| inc0 | oh the joy, oh the happiness | 20:04 |
| SamYaple | i like that layout to be honest..... | 20:05 |
| *** jascott1 has quit IRC | 20:05 | |
| inc0 | you mean neutron -> calico <- k8s? | 20:05 |
| SamYaple | no. i dont like calico for neutron | 20:05 |
| *** affie has joined #openstack-kolla | 20:06 | |
| SamYaple | calico <-> k8s, ovs<->neutron | 20:06 |
| SamYaple | i like overlapping subnets for vms | 20:06 |
| *** mnasiadka has quit IRC | 20:10 | |
| rwellum | SamYaple: So I have weave and kolla-k8s up and running - had no issues with ovs - but maybe my setup is simpler than Justins | 20:11 |
| rwellum | kfox111 - here is the kubectl log - something looks broke... https://www.irccloud.com/pastebin/WH8k1XV3/ | 20:11 |
| rwellum | kfox1111: "nova_cell0.compute_nodes' doesn't exist" | 20:12 |
| SamYaple | rwellum: so theway the ovskernel module works is it *can* kind of work. but neither weave nor the openstack ovs db has a proper view ofthe kernel state | 20:12 |
| SamYaple | this will eventually lead to breakage,peiod | 20:12 |
| SamYaple | soit might be workingfor you right now,and maybe after a reboot.... but not long term.its unstable and undefined behaviour | 20:13 |
| rwellum | SamYaple: ok. But weird that I get past the point that Justin did. | 20:13 |
| SamYaple | that doesnt shock me. you can drop the ovs db and create a new one without rebooting and still, sometimes, haveworking ovs | 20:13 |
| SamYaple | at least temporarily :) | 20:13 |
| SamYaple | thats the thing with undefined behaviour. its undefined | 20:14 |
| *** mnasiadka has joined #openstack-kolla | 20:16 | |
| *** genek has joined #openstack-kolla | 20:16 | |
| rwellum | So SamYaple you think this might be related to my 'no host' issue above then? | 20:17 |
| SamYaple | nah. that looks like the database isn't sync'dright | 20:18 |
| *** rwsu has quit IRC | 20:19 | |
| *** genek has quit IRC | 20:21 | |
| *** mnasiadka has quit IRC | 20:22 | |
| *** mnasiadka has joined #openstack-kolla | 20:25 | |
| *** TxGirlGeek has quit IRC | 20:26 | |
| *** mnasiadka has quit IRC | 20:31 | |
| *** mnasiadka has joined #openstack-kolla | 20:33 | |
| rwellum | Fair enough | 20:38 |
| rwellum | SamYaple: you think talk to nova guys? | 20:38 |
| SamYaple | rwellum: if this is a new cluster i think just dropping the nova dbs and resyncing is whatyou want | 20:39 |
| SamYaple | the table clearly states its not sync'd | 20:39 |
| *** mnasiadka has quit IRC | 20:40 | |
| rwellum | How do I do the first one? | 20:41 |
| rwellum | As in that a k8s fn, or openstack. | 20:41 |
| *** mgoddard has quit IRC | 20:43 | |
| *** mnasiadka has joined #openstack-kolla | 20:45 | |
| *** genek has joined #openstack-kolla | 20:50 | |
| *** Pavo has joined #openstack-kolla | 20:51 | |
| *** mnasiadka has quit IRC | 20:53 | |
| kfox1111 | rwellum: ah. cell0 is important. | 20:55 |
| *** pcaruana has quit IRC | 20:55 | |
| *** genek has quit IRC | 20:55 | |
| rwellum | Something I am not configuring correctly kfox1111 ? | 20:56 |
| kfox1111 | rwellum: probably? 5 has gotten very little tests I think. | 20:56 |
| kfox1111 | your a bit in uncharted waters I'm afraid. | 20:56 |
| kfox1111 | might try writing a 5.x gate. | 20:57 |
| rwellum | Yup I know. I need pike though because I need Pike Cinder for something. | 20:57 |
| rwellum | How would writing the gate help? :) | 20:57 |
| kfox1111 | would help find bugs caused by differences in 4 vs 5 quicker that way, rather then config differences between your setup and the gate as well as 4 vs 5 issues. | 20:57 |
| kfox1111 | you could try 4 everything but cinder too. | 20:58 |
| kfox1111 | one of the beuties of containerization. not everything has to be the same version. | 20:58 |
| rwellum | kfox1111: haven't had the best luck combining containers from different releases. I know it's possible. | 20:58 |
| kfox1111 | I've had pretty good luck mixing versions. just not versions between os's. | 20:59 |
| kfox1111 | ie, centos 2 and 3, not centos 2 and ubuntu 2 or 3. | 20:59 |
| rwellum | Fair enough. I get the point about the gate too. | 20:59 |
| *** gkadam has quit IRC | 21:01 | |
| rwellum | kfox1111: any idea how to do what SamYaple recommends above? "dropping the nova dbs and resyncing" | 21:02 |
| kfox1111 | with kolla kubernetes it would be to rerun all the init jobs. | 21:03 |
| SamYaple | rwellum: `DROP DATABASE nova/nova_api/nova_cell0;` | 21:03 |
| SamYaple | then rerun init stuff | 21:03 |
| SamYaple | yea | 21:03 |
| kfox1111 | but I'm guessing one of them didn't run and create nova_cell0 properly in the first place? | 21:03 |
| kfox1111 | so may not help. | 21:04 |
| *** mnasiadka has joined #openstack-kolla | 21:04 | |
| *** mnasiadka has quit IRC | 21:08 | |
| *** dardelea_ has joined #openstack-kolla | 21:10 | |
| rwellum | I just popped onto nova irc and they said I am missing a 'nova-manage db sync' after creating the nova_cell0 database. So I've popped onto the nova-conductor container and run the command... | 21:11 |
| *** threestrands has joined #openstack-kolla | 21:12 | |
| *** dardelean has quit IRC | 21:12 | |
| *** mnasiadka has joined #openstack-kolla | 21:16 | |
| rwellum | kfox1111: the above got rid of that database sync error. I still have the 'No valid host was found. There are not enough hosts available' issue though | 21:18 |
| *** mnasiadka has quit IRC | 21:22 | |
| *** pcaruana has joined #openstack-kolla | 21:24 | |
| *** genek has joined #openstack-kolla | 21:25 | |
| *** mnasiadka has joined #openstack-kolla | 21:25 | |
| *** rhallisey has quit IRC | 21:27 | |
| *** genek has quit IRC | 21:29 | |
| *** mnasiadka has quit IRC | 21:30 | |
| *** unicell has joined #openstack-kolla | 21:31 | |
| *** mnasiadka has joined #openstack-kolla | 21:33 | |
| *** dave-mccowan has quit IRC | 21:35 | |
| *** pcaruana has quit IRC | 21:39 | |
| *** mnasiadka has quit IRC | 21:40 | |
| *** mnasiadka has joined #openstack-kolla | 21:51 | |
| *** genek has joined #openstack-kolla | 21:52 | |
| rwellum | kfox1111: so on the scheduler the following worked to make nova happy: | 21:53 |
| rwellum | https://www.irccloud.com/pastebin/pzLmjBLB/ | 21:54 |
| rwellum | Basically same as kolla-ansible is doing. | 21:54 |
| *** affie has quit IRC | 21:54 | |
| rwellum | Qn is - This change can go in: /helm/microservice/nova-api-create-simple-cell-job/templates/nova-api-create-cell.yaml - but will this break ocata. | 21:55 |
| *** TxGirlGeek has joined #openstack-kolla | 21:56 | |
| *** genek has quit IRC | 22:00 | |
| *** mnasiadka has quit IRC | 22:00 | |
| *** affie has joined #openstack-kolla | 22:02 | |
| *** TxGirlGeek has quit IRC | 22:07 | |
| *** mnasiadka has joined #openstack-kolla | 22:08 | |
| *** mnasiadka has quit IRC | 22:13 | |
| *** jtriley has quit IRC | 22:14 | |
| *** TxGirlGeek has joined #openstack-kolla | 22:14 | |
| *** mnasiadka has joined #openstack-kolla | 22:15 | |
| *** kbaegis has joined #openstack-kolla | 22:17 | |
| *** mnasiadka has quit IRC | 22:23 | |
| *** TxGirlGeek has quit IRC | 22:23 | |
| *** goldyfruit has quit IRC | 22:25 | |
| *** TxGirlGeek has joined #openstack-kolla | 22:27 | |
| *** jascott1 has joined #openstack-kolla | 22:27 | |
| *** genek has joined #openstack-kolla | 22:28 | |
| *** TxGirlGeek has quit IRC | 22:31 | |
| *** TxGirlGeek has joined #openstack-kolla | 22:32 | |
| *** genek has quit IRC | 22:33 | |
| *** mnasiadka has joined #openstack-kolla | 22:35 | |
| *** genek has joined #openstack-kolla | 22:37 | |
| jascott1 | rwellum thats the error I am getting too 'No valid host was found. There are not enough hosts available' | 22:37 |
| rwellum | This is pike right? | 22:38 |
| *** TxGirlGeek has quit IRC | 22:39 | |
| *** TxGirlGeek has joined #openstack-kolla | 22:39 | |
| *** mnasiadka has quit IRC | 22:39 | |
| jascott1 | 4.0 | 22:41 |
| *** genek has quit IRC | 22:41 | |
| rwellum | Ah | 22:41 |
| rwellum | With weave? | 22:42 |
| jascott1 | what should I be using you think? | 22:42 |
| jascott1 | canal | 22:42 |
| jascott1 | gave up on weave | 22:42 |
| rwellum | From what SamYaple was saying I think weave can't be trusted. | 22:42 |
| rwellum | to work | 22:43 |
| *** TxGirlGeek has quit IRC | 22:43 | |
| *** TxGirlGeek has joined #openstack-kolla | 22:44 | |
| rwellum | Is this a new issue for you jascott1 then? Because I haven't seen that on ocata for a while. | 22:45 |
| rwellum | I'm getting the issue on pike and it's because the cell implementation changed so much I think. | 22:45 |
| jascott1 | yeah this after getting all the compute kit pods up | 22:45 |
| jascott1 | i went into horizon and could create "all the things" except when I launch VM i get that error | 22:45 |
| *** mnasiadka has joined #openstack-kolla | 22:46 | |
| jascott1 | acutally I couldnt create volumes | 22:46 |
| jascott1 | that might be related | 22:46 |
| jascott1 | i need the glue for openstack to use external ceph | 22:46 |
| rwellum | Ah - I'm not using ceph - that's a big difference. | 22:48 |
| rwellum | I'm not convinced you can use ceph for an AIO tbh | 22:48 |
| rwellum | I was surprised when you said it was working. | 22:48 |
| *** mnasiadka has quit IRC | 22:50 | |
| jascott1 | its not AIO | 22:51 |
| jascott1 | master with 3 nodes | 22:51 |
| rwellum | Oh in that case ignore my ceph comment :) | 22:52 |
| jascott1 | ceph works fine for k8s volumes, even have it as default storage but still need to look into letting openstack use it | 22:52 |
| *** affie has quit IRC | 22:52 | |
| *** affie has joined #openstack-kolla | 22:53 | |
| *** mnasiadka has joined #openstack-kolla | 22:53 | |
| *** genek has joined #openstack-kolla | 22:53 | |
| rwellum | I thought you had that sorted with the playbook - but I guess this is more workload based. | 22:56 |
| *** Pavo has quit IRC | 22:56 | |
| *** TxGirlGeek has quit IRC | 22:56 | |
| *** TxGirlGeek has joined #openstack-kolla | 22:57 | |
| *** jascott1 has quit IRC | 22:57 | |
| *** jascott1 has joined #openstack-kolla | 22:58 | |
| *** mnasiadka has quit IRC | 22:58 | |
| *** genek has quit IRC | 22:58 | |
| *** jascott1 has quit IRC | 22:59 | |
| *** jascott1 has joined #openstack-kolla | 22:59 | |
| *** jascott1 has quit IRC | 23:00 | |
| *** mnasiadka has joined #openstack-kolla | 23:01 | |
| *** TxGirlGeek has quit IRC | 23:02 | |
| *** TxGirlGeek has joined #openstack-kolla | 23:03 | |
| *** genek has joined #openstack-kolla | 23:05 | |
| *** mnasiadka has quit IRC | 23:05 | |
| *** mnasiadka has joined #openstack-kolla | 23:06 | |
| *** TxGirlGeek has quit IRC | 23:08 | |
| *** TxGirlGeek has joined #openstack-kolla | 23:08 | |
| *** genek has quit IRC | 23:09 | |
| *** mnasiadka has quit IRC | 23:10 | |
| *** jascott1 has joined #openstack-kolla | 23:11 | |
| *** TxGirlGeek has quit IRC | 23:14 | |
| *** TxGirlGeek has joined #openstack-kolla | 23:15 | |
| *** mnasiadka has joined #openstack-kolla | 23:15 | |
| jascott1 | rwellum what are you using for volumes? | 23:16 |
| *** TxGirlGeek has quit IRC | 23:17 | |
| *** TxGirlGeek has joined #openstack-kolla | 23:17 | |
| rwellum | LVM jascott1 - nothing complicated | 23:20 |
| *** Pavo has joined #openstack-kolla | 23:22 | |
| *** Pavo has quit IRC | 23:22 | |
| *** mnasiadka has quit IRC | 23:23 | |
| *** athomas has quit IRC | 23:24 | |
| *** mnasiadka has joined #openstack-kolla | 23:25 | |
| *** pbourke has quit IRC | 23:28 | |
| *** ntpttr_laptop has quit IRC | 23:28 | |
| *** pbourke has joined #openstack-kolla | 23:30 | |
| *** mnasiadka has quit IRC | 23:30 | |
| *** mnasiadka has joined #openstack-kolla | 23:32 | |
| *** itlinux has joined #openstack-kolla | 23:35 | |
| *** mnasiadka has quit IRC | 23:38 | |
| *** genek has joined #openstack-kolla | 23:38 | |
| *** salv-orlando has quit IRC | 23:41 | |
| *** genek has quit IRC | 23:44 | |
| *** mnasiadka has joined #openstack-kolla | 23:45 | |
| *** mnasiadka has quit IRC | 23:53 | |
| *** mnasiadka has joined #openstack-kolla | 23:55 | |
| *** itlinux has quit IRC | 23:56 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!