| *** carloss has quit IRC | 00:19 | |
| *** ivan_lin has quit IRC | 00:25 | |
| *** ivan_lin has joined #openstack-manila | 00:26 | |
| *** ivan_lin has quit IRC | 02:20 | |
| *** ivan_lin has joined #openstack-manila | 02:21 | |
| *** vhari has quit IRC | 02:40 | |
| *** vhari has joined #openstack-manila | 02:47 | |
| *** fzzf has joined #openstack-manila | 03:35 | |
| *** fzzf1 has joined #openstack-manila | 06:18 | |
| *** fzzf has quit IRC | 06:21 | |
| *** fzzf1 has quit IRC | 06:33 | |
| *** lpetrut has joined #openstack-manila | 06:36 | |
| *** sfernand has quit IRC | 07:05 | |
| *** tosky has joined #openstack-manila | 07:49 | |
| openstackgerrit | Lin PeiWen proposed openstack/manila master: Fix the failure of share create form snapshot specifies'--availability-zone' https://review.opendev.org/c/openstack/manila/+/787507 | 08:48 |
|---|---|---|
| openstackgerrit | Lin PeiWen proposed openstack/manila master: Fix the failure of share create form snapshot specifies'--availability-zone' https://review.opendev.org/c/openstack/manila/+/787507 | 08:49 |
| noonedeadpunk | o/ | 10:18 |
| noonedeadpunk | regarding stack trace I saw yestarday http://paste.openstack.org/show/804717/ | 10:21 |
| noonedeadpunk | it's caused by command `/openstack/venvs/manila-22.0.0.0rc2.dev166/bin/manila type-list` and goes in loop until api service restart | 10:22 |
| noonedeadpunk | same thing happens with `openstack share list` | 10:23 |
| *** vhari has quit IRC | 10:28 | |
| *** vhari has joined #openstack-manila | 10:34 | |
| *** toabctl has joined #openstack-manila | 10:39 | |
| noonedeadpunk | well, I kind of found what's wrong with it | 10:55 |
| noonedeadpunk | funny thing :) | 10:56 |
| noonedeadpunk | related to policy-to-yanl topic | 10:56 |
| *** carloss has joined #openstack-manila | 11:03 | |
| noonedeadpunk | https://bugs.launchpad.net/manila/+bug/1925475 | 11:07 |
| openstack | Launchpad bug 1925475 in OpenStack Shared File Systems Service (Manila) "Manila API fails to handle policy issue logging when journal is used" [Undecided,New] | 11:07 |
| *** lkuchlan has joined #openstack-manila | 11:19 | |
| noonedeadpunk | but, I how new error in tempest now one this is resolved :( | 11:25 |
| noonedeadpunk | https://zuul.opendev.org/t/openstack/build/07e0e0d7160b4196ae3c3ee01b773b9b/log/logs/host/manila-share.service.journal-17-44-54.log.txt#1802 | 11:30 |
| noonedeadpunk | but it's already with vkmc patch applied... as I see ceph caps set like http://paste.openstack.org/show/804721/ | 11:31 |
| noonedeadpunk | And not sure what subvolume is... | 11:31 |
| noonedeadpunk | I think manila has been operating with rbd drives themselves previously? | 11:31 |
| noonedeadpunk | well, I see for devstack you using following caps ["mon", "allow *", "osd", "allow rw", "mds", "allow *", "mgr", "allow *"] | 11:37 |
| gouthamr | noonedeadpunk: thanks for reporting LP 1925475 | 11:39 |
| openstack | Launchpad bug 1925475 in OpenStack Shared File Systems Service (Manila) "Manila API fails to handle policy issue logging when journal is used" [Undecided,New] https://launchpad.net/bugs/1925475 | 11:39 |
| gouthamr | So iiuc this issue manifests when there is a json policy file? does it go away when using a policy.yaml file? | 11:41 |
| noonedeadpunk | the issue raised when there's any logging regarding policy file | 11:41 |
| noonedeadpunk | For example, I created policy.yaml with wrong permissions (ie owned by root) and it was the same | 11:42 |
| gouthamr | Oh! | 11:42 |
| noonedeadpunk | but I was able to set use_journal=false and saw warnings/errors regardign policies | 11:42 |
| gouthamr | Awesome, ty - that info gives me some more context; | 11:43 |
| gouthamr | noonedeadpunk: when you say vkmc's patch, do you mean this one? https://review.opendev.org/c/openstack/manila/+/786912 | 11:45 |
| vkmc | hey o/ | 11:45 |
| noonedeadpunk | I meant https://opendev.org/openstack/openstack-ansible/commit/7a769bb9216693f8045e2a987c911004c06d5a15 :) | 11:45 |
| noonedeadpunk | but probably not that | 11:46 |
| noonedeadpunk | let me try using origin/master | 11:47 |
| gouthamr | noonedeadpunk: we caught a regression unfortunately with the caps, and https://review.opendev.org/c/openstack/manila/+/786912 is the fix | 11:47 |
| gouthamr | noonedeadpunk: i haven't requested a release tag out of stable/wallaby yet | 11:47 |
| noonedeadpunk | having `mon "allow *" osd "allow rw" mds "allow *" mgr "allow *"` didn't help me though | 11:48 |
| gouthamr | yep | 11:48 |
| gouthamr | oh, hold on - vkmc: wasn't the regression that we needed "mon allow rw" | 11:48 |
| gouthamr | * would mean rw, no? | 11:48 |
| vkmc | the caps we were missing in openstack-ansible was the mgr ones | 11:49 |
| vkmc | mgr "allow *" | 11:49 |
| vkmc | I don't know how is this related with policies | 11:50 |
| vkmc | are you seeing an error in the share-mgr log having cephfs as a backend? | 11:50 |
| noonedeadpunk | nah, these are 2 separate things) | 11:50 |
| noonedeadpunk | yep - https://zuul.opendev.org/t/openstack/build/07e0e0d7160b4196ae3c3ee01b773b9b/log/logs/host/manila-share.service.journal-17-44-54.log.txt#1802 | 11:51 |
| vkmc | checking | 11:51 |
| * gouthamr notes that https://zuul.opendev.org/t/openstack/build/07e0e0d7160b4196ae3c3ee01b773b9b/log/logs/host/manila-share.service.journal-17-44-54.log.txt#1957-1997 is a bug as well | 11:51 | |
| gouthamr | the main issue however is https://zuul.opendev.org/t/openstack/build/07e0e0d7160b4196ae3c3ee01b773b9b/log/logs/host/manila-share.service.journal-17-44-54.log.txt#1915-1920 | 11:52 |
| vkmc | gouthamr, so it's the same issue we were hitting on the rdo ci | 11:52 |
| gouthamr | yep | 11:52 |
| * noonedeadpunk checking in sandbox if fix is helping here | 11:53 | |
| vkmc | noonedeadpunk, check if this one is in https://review.opendev.org/c/openstack/manila/+/786912 | 11:53 |
| noonedeadpunk | nah, originally it was not. | 11:53 |
| noonedeadpunk | we bumped the day before it merged https://opendev.org/openstack/openstack-ansible/src/branch/master/playbooks/defaults/repo_packages/openstack_services.yml#L140 | 11:54 |
| gouthamr | oh nice, you guys don't use tags | 11:55 |
| noonedeadpunk | nope:) | 11:55 |
| noonedeadpunk | but it's confusing in prod envs tbh | 11:56 |
| noonedeadpunk | as you always end up with some dev version if checking pip | 11:56 |
| vkmc | noonedeadpunk, let us know if applying that fix solves the issue :) | 12:06 |
| noonedeadpunk | seems it does:) `Apr 22 12:07:22 aio1 manila-share[227264]: 2021-04-22 12:07:22.292 227264 INFO manila.share.manager [req-e513f491-6d6b-4b11-96df-3128f867ac6d 26f2e1378890443eb935631220937b9c 9ad37a93ffc94403a016148df70bf45a - - -] Share instance a3b75a29-eacf-4031-878b-b63b4cac6e2e created successfully.` | 12:07 |
| vkmc | \o/ | 12:08 |
| gouthamr | awesome! i'll request a tag right away for anyone that might need it | 12:08 |
| gouthamr | while we're doing that, can you help land two more bugfixes? https://review.opendev.org/q/project:openstack/manila+status:open+branch:stable/wallaby | 12:11 |
| gouthamr | vkmc carloss tbarron dviroel ^ | 12:11 |
| carloss | sure! | 12:12 |
| vkmc | sure! | 12:13 |
| noonedeadpunk | ok, thanks for your time! tempest just passed successfully for me! | 12:17 |
| gouthamr | noonedeadpunk: ty for finding these bugs; i'm glad osa consumers won't see them! :) | 12:19 |
| noonedeadpunk | well, we haven't released W so yeah... | 12:19 |
| noonedeadpunk | good to have some more time | 12:20 |
| *** ecsantos has joined #openstack-manila | 12:23 | |
| vkmc | ++ | 12:24 |
| *** lpetrut_ has joined #openstack-manila | 12:27 | |
| *** lpetrut has quit IRC | 12:29 | |
| *** ivan_lin has quit IRC | 13:04 | |
| *** ivan_lin has joined #openstack-manila | 13:05 | |
| *** ivan_lin has quit IRC | 13:05 | |
| *** ivan_lin has joined #openstack-manila | 13:06 | |
| *** brinzhang_ has joined #openstack-manila | 13:14 | |
| *** lpetrut_ has quit IRC | 13:45 | |
| *** brinzhang_ has quit IRC | 13:50 | |
| openstackgerrit | Merged openstack/manila stable/wallaby: Document policy rule using the description parameter https://review.opendev.org/c/openstack/manila/+/785387 | 13:59 |
| *** disap has joined #openstack-manila | 14:05 | |
| noonedeadpunk | fwiw in CI things does not look so good | 15:20 |
| noonedeadpunk | eventually more permissions are required that was provided atm | 15:20 |
| noonedeadpunk | as localy I was testing with having `mon "allow *" osd "allow rw" mds "allow *" mgr "allow *"` | 15:20 |
| noonedeadpunk | And CI ends up with rados.Error: [errno -13] access denied: does your client key have mgr caps? | 15:21 |
| noonedeadpunk | https://zuul.opendev.org/t/openstack/build/668e476ba50a45cd882e3b9b3d79c44a/log/logs/host/manila-share.service.journal-14-23-55.log.txt#1905 | 15:21 |
| *** lkuchlan has quit IRC | 15:21 | |
| gouthamr | crap, i forgot to hit "resume recording" for vb123 and ecsantos's session :( : | 15:24 |
| dviroel | :( | 15:24 |
| gouthamr | damn sorry about that, ecsantos | 15:24 |
| gouthamr | noonedeadpunk: ouch, we're in our PTG sessions again today - vkmc/I will take a look | 15:25 |
| vkmc | noonedeadpunk, looking | 15:26 |
| noonedeadpunk | gouthamr: yeah, sure, I'm just saying kind of) | 15:26 |
| noonedeadpunk | I believe caps should be like that atm http://paste.openstack.org/show/804721/ | 15:26 |
| vkmc | caps mgr = "allow r" <- "allow *" | 15:27 |
| noonedeadpunk | so for mgr it should be * ? | 15:27 |
| vkmc | yes | 15:28 |
| vkmc | wonder how it was set to r | 15:28 |
| noonedeadpunk | well, it was your patch I believe? https://opendev.org/openstack/openstack-ansible/commit/7a769bb9216693f8045e2a987c911004c06d5a15 | 15:28 |
| noonedeadpunk | So I thought that `r` is correct thing.... | 15:29 |
| vkmc | that's not right, no | 15:29 |
| noonedeadpunk | :( | 15:29 |
| vkmc | let me fix that | 15:29 |
| noonedeadpunk | I will adjust my patch that fixing tempest settings | 15:30 |
| noonedeadpunk | or maybe it's really worth having as a separate patch... | 15:31 |
| vkmc | noonedeadpunk, give me a sec while I check the correct caps | 15:31 |
| noonedeadpunk | would be great if you could push it to speed up review process :) | 15:31 |
| vkmc | definitely | 15:31 |
| noonedeadpunk | in devstack it's * | 15:32 |
| vkmc | noonedeadpunk, just to make sure, you are using a deployment with manila wallaby and ceph octopus | 15:33 |
| vkmc | devstack caps are broader than we advise | 15:34 |
| vkmc | noonedeadpunk, I see this line here https://zuul.opendev.org/t/openstack/build/668e476ba50a45cd882e3b9b3d79c44a/log/logs/host/manila-share.service.journal-14-23-55.log.txt#56 | 15:35 |
| vkmc | noonedeadpunk, is the rados module installed? | 15:37 |
| noonedeadpunk | I think that's fine kind of? as we later restart service | 15:37 |
| noonedeadpunk | on L716 | 15:37 |
| noonedeadpunk | I thin kwe could do better, but we install venv, deploy systemd file and only later configure service and symlink extra modules into the venv (since librados is not smth you can install with pip) | 15:38 |
| noonedeadpunk | yes, that patch depends-on change that updates SHA https://review.opendev.org/c/openstack/openstack-ansible/+/787540/1/playbooks/defaults/repo_packages/openstack_services.yml inlcuding your fix | 15:39 |
| vkmc | it's fine yes, just checking | 15:39 |
| noonedeadpunk | without it it was failing worse | 15:39 |
| noonedeadpunk | well, I still have sandbox where tempest passed with devstack permissions | 15:40 |
| noonedeadpunk | I can change them to what we use and see the result | 15:41 |
| vkmc | noonedeadpunk, can you try in your env changing the caps to mgr allow 'rw'? | 15:41 |
| vkmc | if that works for you, will submit the fix to osa | 15:41 |
| vkmc | rw should be enough | 15:41 |
| noonedeadpunk | running tempest with `ceph auth caps client.manila mon "allow *" osd "allow rw" mds "allow *" mgr "allow rw"` | 15:42 |
| vkmc | noonedeadpunk++ | 15:42 |
| vkmc | thanks for that | 15:42 |
| noonedeadpunk | {0} manila_tempest_tests.tests.scenario.test_share_basic_ops.TestShareBasicOpsNFS.test_mount_share_one_vm [158.363910s] ... ok | 15:44 |
| vkmc | :D | 15:44 |
| vkmc | ok, will submit the fix to osa | 15:45 |
| noonedeadpunk | cool, thanks! | 15:45 |
| *** dviroel is now known as dviroel|lunch | 15:46 | |
| noonedeadpunk | hm..... | 15:48 |
| noonedeadpunk | ceph auth caps client.manila mon "allow *" osd "allow rw" mds "allow *" mgr "allow r" also worked | 15:50 |
| * noonedeadpunk needs more testing... | 15:51 | |
| noonedeadpunk | I suppose that in caps mon permissions were missing..... | 15:52 |
| noonedeadpunk | as currently I have `caps mon = "allow *"` | 15:52 |
| vkmc | noonedeadpunk, https://review.opendev.org/c/openstack/openstack-ansible/+/787591 | 15:54 |
| openstackgerrit | Tom Barron proposed openstack/manila master: Remove deprecated config and auth https://review.opendev.org/c/openstack/manila/+/745206 | 16:06 |
| openstackgerrit | Victoria Martinez de la Cruz proposed openstack/manila master: Updates CephFS drivers docs https://review.opendev.org/c/openstack/manila/+/782871 | 16:08 |
| ecsantos | np gouthamr don't worry | 16:45 |
| *** dviroel|lunch is now known as dviroel | 16:58 | |
| tbarron | ecsantos: great preso, btw! | 17:02 |
| ecsantos | tbarron: thanks a lot :D | 17:06 |
| *** disap has quit IRC | 17:16 | |
| dviroel | ecsantos++ | 17:19 |
| openstackgerrit | Victoria Martinez de la Cruz proposed openstack/manila master: Updates CephFS drivers docs https://review.opendev.org/c/openstack/manila/+/782871 | 17:23 |
| vkmc | noonedeadpunk, seems we needed to fix other caps apart from adding the mgr rw one, so the last patch I submitted to osa should be what you were missing | 17:38 |
| *** ianychoi_ has joined #openstack-manila | 17:38 | |
| vkmc | noonedeadpunk, I linked to a patch in devstack-plugin-ceph that we test this... the ci is running on that change, devstack-plugin-ceph-cephfs-native passed, now we are waiting for the nfs gate | 17:39 |
| *** ecsantos has quit IRC | 17:42 | |
| *** ianychoi__ has quit IRC | 17:48 | |
| *** jv has quit IRC | 17:48 | |
| *** carthaca has quit IRC | 17:48 | |
| *** kopecmartin has quit IRC | 17:48 | |
| *** eharney has quit IRC | 17:49 | |
| *** jv has joined #openstack-manila | 17:54 | |
| *** eharney has joined #openstack-manila | 17:55 | |
| *** lbragstad_ has joined #openstack-manila | 18:29 | |
| *** lbragstad has quit IRC | 18:32 | |
| openstackgerrit | Victoria Martinez de la Cruz proposed openstack/manila master: Updates CephFS drivers docs https://review.opendev.org/c/openstack/manila/+/782871 | 19:31 |
| *** arne_wiebalck has quit IRC | 19:51 | |
| *** rajinir has quit IRC | 19:51 | |
| *** almir-okato has quit IRC | 19:52 | |
| *** lbragstad_ is now known as lbragstad | 19:56 | |
| *** arne_wiebalck has joined #openstack-manila | 19:57 | |
| *** rajinir has joined #openstack-manila | 19:57 | |
| *** almir-okato has joined #openstack-manila | 19:57 | |
| *** irclogbot_0 has quit IRC | 21:50 | |
| *** irclogbot_2 has joined #openstack-manila | 21:54 | |
| *** tosky has quit IRC | 23:17 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!