| opendevreview | Joan Gilabert proposed openstack/watcher master: Disable cinder volume service in subnode https://review.opendev.org/c/openstack/watcher/+/964462 | 08:00 |
|---|---|---|
| opendevreview | Joan Gilabert proposed openstack/watcher master: Disable cinder volume service in subnode https://review.opendev.org/c/openstack/watcher/+/964462 | 10:14 |
| opendevreview | Joan Gilabert proposed openstack/watcher-tempest-plugin master: Skip Storage Capacity test until it's fixed https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/964532 | 11:16 |
| opendevreview | Joan Gilabert proposed openstack/watcher master: Disable cinder volume service in subnode https://review.opendev.org/c/openstack/watcher/+/964462 | 11:18 |
| opendevreview | Merged openstack/watcher master: Remove unused glance client integration https://review.opendev.org/c/openstack/watcher/+/963217 | 11:44 |
| jgilaber | sean-k-mooney chandankumar when you have some time please review https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/964532 and https://review.opendev.org/c/openstack/watcher/+/964462 | 12:38 |
| jgilaber | they are needed to forward with the addition of volume migration tempest tests | 12:39 |
| sean-k-mooney | looks like chandan approved the first already so ill look at teh other now | 12:43 |
| sean-k-mooney | im not sure i agree with the second patch | 12:44 |
| sean-k-mooney | we really shoudl have the backend on diffent hosts | 12:44 |
| sean-k-mooney | why woudl we want to test with 2 backends on the constoelr when that is not really reflective of how it would be deployed in production | 12:45 |
| jgilaber | it came from a discussion with dviroel in https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/958644 | 12:46 |
| jgilaber | this one in particular https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/958644/comment/a081ec2e_c35d284e/ | 12:46 |
| dviroel | for lvm it is true that both will be on the same host, but the usual deploy is to have multiple backends configured in a single cinder config file | 12:47 |
| jgilaber | I'll admit I'm not very familiar with the difference between Active-Active or Active-Passives modes of Cinder (I'm trying to read on that= | 12:47 |
| dviroel | we shouldn't deploy multiple c-vols | 12:47 |
| sean-k-mooney | but that testing somethign else | 12:48 |
| sean-k-mooney | it very common to have multipel cinder voluems each pointing at diffent backends | 12:48 |
| sean-k-mooney | this si not really related to active active vs active passive | 12:48 |
| sean-k-mooney | its about ensureing we have 2 seperate dataplanes | 12:48 |
| dviroel | and it is now how cinder is testing on upstream too | 12:49 |
| sean-k-mooney | we coudl do that with the lvm dirver on one host but we would need to loopback device with two diffen lvm volume pools | 12:49 |
| sean-k-mooney | there are two diffent test senairos | 12:51 |
| sean-k-mooney | movign between too pools on one backend | 12:51 |
| sean-k-mooney | and moving between 2 backedns | 12:51 |
| sean-k-mooney | we supprot both in watcher | 12:51 |
| sean-k-mooney | and the exsting test are mainly for migration between seperate backedn which is why we are testing with 2 c-vols on diffent hosts | 12:52 |
| dviroel | iiuc the second one is not working in this strategy, and requires a fix | 12:52 |
| jgilaber | the current setup is configuring the same backend on different hosts, right? | 12:53 |
| jgilaber | both the controller and compute configure a 'lvmdriver-1' backend | 12:53 |
| sean-k-mooney | the second config is the default we use in all devstack multi node jobs https://github.com/openstack/devstack/blob/master/.zuul.yaml#L699 | 12:53 |
| sean-k-mooney | jgilaber: yep which is a vlid config | 12:54 |
| jgilaber | right now that is the only configuration supported for volume migrations in watcher | 12:54 |
| dviroel | yes, they will be different since the host is different | 12:54 |
| sean-k-mooney | the backend is defien by not jsut the drive rname but also the host | 12:55 |
| sean-k-mooney | so right now we have supprot for migrating between storage backedn and or between pools | 12:55 |
| sean-k-mooney | the same opertoin is used for both cases in cinder | 12:56 |
| jgilaber | not between storage backend, since the migrate method expects the destination to have the same 'volume_backend_name' | 12:57 |
| sean-k-mooney | not on the cider side | 12:57 |
| dviroel | no, this is the bug in watcher side | 12:57 |
| sean-k-mooney | if we require that that is a bug which i tough twe dicused | 12:57 |
| sean-k-mooney | right | 12:57 |
| jgilaber | yes, that is a bug, and I have a fix for it up | 12:58 |
| sean-k-mooney | right so when fixign that bug we will need to still have two c-vol instnaces | 12:58 |
| dviroel | it should work with 2 backends, despite the c-vol? | 12:59 |
| sean-k-mooney | yes | 13:00 |
| sean-k-mooney | the presence of the c-vol on the compute is not incorrect | 13:00 |
| sean-k-mooney | and it need to work in that case | 13:00 |
| dviroel | ok, i just don't see that clear anywhere too | 13:01 |
| dviroel | but it was working | 13:01 |
| sean-k-mooney | its valid for example to migrate form lvm on the contoler to ceph or netapp on the compute/subnode | 13:01 |
| sean-k-mooney | dviroel: that how we test volume migation in the nova gate | 13:01 |
| sean-k-mooney | we alwasy have 2 c-vols | 13:01 |
| sean-k-mooney | and we test migrating between the 2 backend hosts to make sure the data micration actully works | 13:02 |
| sean-k-mooney | its not related to active active or active passive at all | 13:02 |
| dviroel | so if is a valid config and works, that's fine them | 13:06 |
| dviroel | but maybe the watcher is not really prepared for that, when looking only at "backend_name" | 13:06 |
| dviroel | since they deploy with the same name | 13:06 |
| sean-k-mooney | if we want to run 2 backend on a single host with one cinder-vol instnace we need to make sure both backend are using diffent stroage. menaing 2 loopback device with 2 seperate lvm volume groups | 13:06 |
| dviroel | i guess that this is coverage somehow, since we have some cinder jobs using 2 lvm backends in the same host | 13:07 |
| dviroel | but i didn't check relly | 13:07 |
| sean-k-mooney | i have never seen an lvm job like that for what its worth | 13:08 |
| sean-k-mooney | so this looks like a very non standard job proposal | 13:08 |
| sean-k-mooney | it may be valid but i think we need to look at how this is respened on the dataplane side and in cidner api to confirm | 13:09 |
| dviroel | this one https://zuul.opendev.org/t/openstack/build/3c783bfaa7824d7b8f572860b8942166 | 13:09 |
| sean-k-mooney | the fact that that is non-voting is not promising | 13:10 |
| jgilaber | this is the cinder conf from my patch https://07c2e7671593cc30b79e-a759d6b54561529b072782a6b0052389.ssl.cf2.rackcdn.com/openstack/87adae320fa64ae283202ebe60313f01/controller/logs/etc/cinder/cinder_conf.txt | 13:10 |
| jgilaber | with the two backends in the controller | 13:10 |
| opendevreview | Merged openstack/watcher-tempest-plugin master: Skip Storage Capacity test until it's fixed https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/964532 | 13:17 |
| dviroel | this ^ test wasn't trigger even with 2 backends enabled, because they had the same name? | 13:20 |
| * dviroel need to check the skip_checks in this case | 13:20 | |
| jgilaber | it relies on the tempest config backend_names | 13:21 |
| sean-k-mooney | ok so that is using two lvm volume groups | 13:21 |
| jgilaber | which is not set in the current job | 13:21 |
| sean-k-mooney | so that is also valid but that not really what i woudl consdier mutli backend | 13:21 |
| sean-k-mooney | that is more a single backend with 2 pools | 13:21 |
| sean-k-mooney | well i dont know its maybe multi backend but its not hwo we normally test that | 13:22 |
| jgilaber | ack, so the question is do want to test both scenarios? So instead of changing the existing job, add a new one? | 13:22 |
| dviroel | ack, i f you want to migrate across nodes, the previous would be the right one | 13:22 |
| sean-k-mooney | we definlty want to have at least one job mighrating the data across nodes | 13:23 |
| dviroel | not needed I think, should be the more complex/common one | 13:23 |
| dviroel | across node in this case | 13:23 |
| sean-k-mooney | nomrally when you have diffent backend deploy they dont end up usign the same driver i.e. they would be netapp and cpeh | 13:23 |
| sean-k-mooney | so im concerned that if we are not testign phsycially sepeate host we may hit issues or miss bugs that we test in the more standard case | 13:24 |
| dviroel | it depends, it is common two have more than 2 netapp backend yes | 13:25 |
| sean-k-mooney | jgilaber: do you happen to know what cidner returns for the pool list in both cases | 13:25 |
| jgilaber | this is from the decision-engine logs with the current setup | 13:26 |
| jgilaber | Oct 21 19:30:37.095967 np87be28cf22444 watcher-decision-engine[93381]: <StorageNode host="np47ad865b91dc4@lvmdriver-1" zone="nova" status="enabled" state="up" volume_type="['lvmdriver-1']" uuid="" human_id=""> | 13:26 |
| jgilaber | Oct 21 19:30:37.095967 np87be28cf22444 watcher-decision-engine[93381]: <Pool name="np47ad865b91dc4@lvmdriver-1#lvmdriver-1" total_volumes="1" total_capacity_gb="28" free_capacity_gb="28" provisioned_capacity_gb="0" allocated_capacity_gb="0" virtual_free="0" uuid="" human_id=""/> | 13:26 |
| jgilaber | Oct 21 19:30:37.095967 np87be28cf22444 watcher-decision-engine[93381]: </StorageNode> | 13:26 |
| jgilaber | Oct 21 19:30:37.095967 np87be28cf22444 watcher-decision-engine[93381]: <StorageNode host="np87be28cf22444@lvmdriver-1" zone="nova" status="enabled" state="up" volume_type="['lvmdriver-1']" uuid="" human_id=""> | 13:26 |
| jgilaber | Oct 21 19:30:37.095967 np87be28cf22444 watcher-decision-engine[93381]: <Pool name="np87be28cf22444@lvmdriver-1#lvmdriver-1" total_volumes="1" total_capacity_gb="28" free_capacity_gb="28" provisioned_capacity_gb="0" allocated_capacity_gb="0" virtual_free="0" uuid="" human_id=""/> | 13:26 |
| jgilaber | Oct 21 19:30:37.095967 np87be28cf22444 watcher-decision-engine[93381]: </StorageNode> | 13:26 |
| jgilaber | Oct 21 19:30:37.095967 np87be28cf22444 watcher-decision-engine[93381]: </ModelRoot> | 13:26 |
| jgilaber | not great formatting, you can see it by searching for 'StorageNode' in https://daf6a8c4e4a7fc0beca6-64bde2a99f8c0525b91ac2e42ffc128a.ssl.cf5.rackcdn.com/openstack/ee005da80be347b69b4efcf6df0adbae/controller/logs/screen-watcher-decision-engine.txt | 13:27 |
| jgilaber | this is from the modified job with the two backends in the controller https://07c2e7671593cc30b79e-a759d6b54561529b072782a6b0052389.ssl.cf2.rackcdn.com/openstack/87adae320fa64ae283202ebe60313f01/controller/logs/screen-watcher-decision-engine.txt | 13:28 |
| jgilaber | https://pastebin.com/tCi5C1Bc here is hopefully more legible | 13:29 |
| sean-k-mooney | so https://docs.openstack.org/api-ref/block-storage/v3/index.html?expanded=detach-volume-from-server-detail#list-all-back-end-storage-pools is what i really want to see for both toplogies | 13:31 |
| jgilaber | that is also in the decision engine logs, give a minute and I'll collect it for both jobs | 13:32 |
| dviroel | not sure if we log that in dec-eng | 13:32 |
| jgilaber | we do when building the storage data model, here is the response for both jobs https://pastebin.com/vw1eY05h | 13:36 |
| dviroel | jgilaber: so, in the end is perfectly fine to have the 2 c-vols working without any AA or AP mode, sorry about the confusion | 13:37 |
| dviroel | we could just drop the second change e and focus on the fix now? | 13:38 |
| jgilaber | no problem dviroel I wasn't sure about it when we mentioned, I find the naming around most cinder concepts quite confusing, so this discussion has been quite helpful | 13:40 |
| jgilaber | I have the series for volume migration tests in tempest https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/958644 the first one has a workaround for the bug about the backend name | 13:40 |
| dviroel | yeah, the thing is that there was 2 pools, from different backends, but that have the same backend_name, but they are still 2 different pools | 13:41 |
| jgilaber | the last patch of the series removes the workaround but depends on the bug fix | 13:42 |
| jgilaber | so we could also merge the bug fix first and then add the testing | 13:42 |
| dviroel | i would go to try to fix the bug first, yes | 13:43 |
| jgilaber | this is the bug fix for reference https://review.opendev.org/c/openstack/watcher/+/963857 | 13:43 |
| sean-k-mooney | so in the first case https://pastebin.com/vw1eY05h we have 2 diffent backend both with a singel pools with the same poolname. and in the second case we have 2 backend with 2 diffent pools names | 14:01 |
| sean-k-mooney | the pool name cant be assumed to be unique in the cluster as far as i am aware | 14:01 |
| dviroel | we can't just have 2 host@backend#pool | 14:03 |
| dviroel | the other options are valid | 14:04 |
| dviroel | what we need to take care is that, if they have the same backend_name, and one pool each, doesn't mean that we have a backend with 2 pools | 14:05 |
| dviroel | which is different | 14:06 |
| dviroel | only means that we have 2 pools | 14:06 |
| dviroel | the scheduler should acepts the migration since they have the same backend_name in this case | 14:07 |
| dviroel | if we set the backend_name in the volume_type | 14:07 |
| jgilaber | I that internally watcher only deals with pools, so whether they are part of the same backend or not should not make any difference I think | 14:07 |
| jgilaber | at least for zone migration | 14:07 |
| sean-k-mooney | we cant have 2 identical ones no but as long as one oof host, backend or pool are diffent | 14:08 |
| sean-k-mooney | then we can do a volume migrate between them | 14:08 |
| sean-k-mooney | it shoudl be noted too that pools are entirly optional in the cinder api | 14:08 |
| sean-k-mooney | and cider volume migrate was origianl for migrativ between diffent host and/or diffent backends | 14:09 |
| sean-k-mooney | what shold not be assimign that there are diffent pools in the zone migration stragey | 14:10 |
| sean-k-mooney | https://opendev.org/openstack/watcher/src/branch/master/watcher/decision_engine/strategy/strategies/storage_capacity_balance.py is the only one that is ment to be pool specific | 14:12 |
| sean-k-mooney | i guess the you can argure that zone migration is prhased in term of pools as well | 14:13 |
| sean-k-mooney | but it really shoudl be prahsed in terms of the triple in liekly both cases | 14:13 |
| sean-k-mooney | i.e. the fact that this has pool in the name is rathr unfortunet https://opendev.org/openstack/watcher/src/branch/master/watcher/decision_engine/strategy/strategies/zone_migration.py#L114-L123 | 14:14 |
| jgilaber | tbh looking through the zone migration code now I don't think it requires a pool name at all | 14:14 |
| jgilaber | it's just using pool to refer to some storage host | 14:14 |
| sean-k-mooney | https://docs.openstack.org/api-ref/block-storage/v3/index.html?expanded=detach-volume-from-server-detail#migrate-a-volume | 14:14 |
| jgilaber | I think just passing some host@backend would work just fine | 14:14 |
| sean-k-mooney | so volume migrate on the sider ssid does not | 14:15 |
| sean-k-mooney | and technially the document usin git for host@backend not host@backend#pool | 14:15 |
| dviroel | i think that the strategy should only mention pools if the idea is just to migrate between pools, of the same backend. which is not the case | 14:15 |
| dviroel | and then only storages that supports multiple pools would be considered | 14:16 |
| dviroel | these ones are migrating between backends | 14:16 |
| sean-k-mooney | ya so long term watcher shoudl supprot migraton acrros backend host or pool because cidner support all 3 | 14:17 |
| dviroel | we would just need to fix the parameters naming, and docs to make that clear | 14:18 |
| dviroel | and fix bugs | 14:18 |
| sean-k-mooney | yep which is what i tought was the orginal plan. not the parmater namign but just to supprot all the migration cases | 14:19 |
| jgilaber | I don't think there is much to change in the zone migration, the issues will mostly be in the volume migration action and the cinder helper | 14:19 |
| sean-k-mooney | we can deprecate the existign parmater and add a src/dest storage_backend or similar | 14:19 |
| jgilaber | there is where the assumptions are made | 14:19 |
| sean-k-mooney | how do we want to proceed | 14:24 |
| sean-k-mooney | changing parmater like this is not a bug | 14:24 |
| sean-k-mooney | it technaily shoudl have a spec to cover the api and upgrade impact | 14:24 |
| sean-k-mooney | do we want to fucs on jsut the pool case whre we stictly only supprot migrating between pools on the same "backend" even thogh that is not a documetned usecase fo rthe cinder api | 14:25 |
| sean-k-mooney | we coudl talks about it at the ptg or in the irc call tomorow | 14:25 |
| dviroel | i think that we can properly document that for now, and we should still support all migration types, and not restrict to the same backend and so | 14:26 |
| jgilaber | other that the known bug, I don't think we need to change much to support any migration that is supported by cinder | 14:27 |
| dviroel | jgilaber: yeah, we are just talking that the name is confusing in the parameter, which we can discuss on chaging in the future | 14:28 |
| jgilaber | I can write a spec for the parameters change, but I think we can do that while moving forward with the tempest testing | 14:28 |
| dviroel | other than that, we would just fix docs and any other bug | 14:28 |
| jgilaber | ack, +1 from me | 14:29 |
| jgilaber | I'll bring the topic up tomorrow in the irc meeting for visibility | 14:29 |
| sean-k-mooney | +1 | 14:31 |
| dviroel | ack | 14:31 |
| sean-k-mooney | if we need to update the ci job temporaly we can | 14:33 |
| sean-k-mooney | i just dont want that to be the long term state | 14:33 |
| jgilaber | if the current job is valid I don't think we need to, we just need to fix the known bug | 14:36 |
| sean-k-mooney | its valid if we supprot cross backend migration | 14:36 |
| sean-k-mooney | or cross host migration | 14:37 |
| jgilaber | It should work https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/963860 removes the problematic check and it works fine | 14:37 |
| sean-k-mooney | but its not valid if we only supprot cross poll migration on the same host and backend | 14:37 |
| jgilaber | https://softwarefactory-project.io/zuul/t/rdoproject.org/build/eb3f8a5518e44b8d9e1bf2a39cadd35a migrates volumes across two different backends | 14:38 |
| sean-k-mooney | right my point is it depend on what we claim to support rather then what actuly works | 14:38 |
| sean-k-mooney | so this is not really testing the right thing https://review.opendev.org/c/openstack/watcher-tempest-plugin/+/963860/5/watcher_tempest_plugin/tests/scenario/test_execute_zone_migration.py#136 | 14:39 |
| sean-k-mooney | in that its eplicktly testing for multipel pool form a stinle backend on a singel host | 14:39 |
| sean-k-mooney | i.e. that stil enfoceing the srict case | 14:40 |
| sean-k-mooney | {pool['name'] for pool in pools['pools']} is still depending on unique pool names at the cinder level | 14:41 |
| jgilaber | ah ok, I see what you mean | 14:41 |
| dviroel | right, we need to take care with that, only host@backend#pool are unique | 14:42 |
| jgilaber | actually I don't think that is wrong, the pool['name'] returns the host@backend#pool | 14:43 |
| jgilaber | pool['capabilities']['pool_name'] would return only the pool name, which will not be necessarily unique | 14:43 |
| dviroel | "pools": [{"name": "np6144f3e50f784@lvmdriver-1#lvmdriver-1" | 14:44 |
| dviroel | right | 14:44 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: APISchedulingService migrate audits also on first discovery of services https://review.opendev.org/c/openstack/watcher/+/963981 | 15:13 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Move decision-engine monitoring service to the decision-engine https://review.opendev.org/c/openstack/watcher/+/963252 | 15:13 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Add second instance of watcher-decision-engine in the compute node https://review.opendev.org/c/openstack/watcher/+/964546 | 15:13 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: APISchedulingService migrate audits also on first discovery of services https://review.opendev.org/c/openstack/watcher/+/963981 | 16:24 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Move decision-engine monitoring service to the decision-engine https://review.opendev.org/c/openstack/watcher/+/963252 | 16:24 |
| opendevreview | Alfredo Moralejo proposed openstack/watcher master: Add second instance of watcher-decision-engine in the compute node https://review.opendev.org/c/openstack/watcher/+/964546 | 16:24 |
| sean-k-mooney | sigh ok | 16:44 |
| sean-k-mooney | it does not help that thre are like 3 overloaded meaning of pool vs backend ectra | 16:45 |
| sean-k-mooney | most people i think woulc all the triple of host@backend#pool the storage backend where as cider calls it the host for the migrate api and we are acllign it the pool or pool['name'] | 16:47 |
| jgilaber | I agree the naming is quite confusing, I still get confused by it, and it makes the discussions longer than necessary since we often means different things | 16:57 |
| dviroel | manila has the same host,backend,pool naming :) | 17:11 |
| dviroel | in manila docs, https://docs.openstack.org/api-ref/shared-file-system/#start-migration - host format is defined as host@backend#pool | 17:13 |
| jgilaber | the host for a volume in cinder is also host@backend#pool, so the substitute for src_pool should probably be src_host with the full name | 17:17 |
| jgilaber | and funny enough the help for the openstack volume migrate shows also the full name: | 17:19 |
| jgilaber | --host <host> | 17:19 |
| jgilaber | Destination host (takes the form: host@backend-name#pool) | 17:19 |
| jgilaber | I think that was the format in the documentation not long ago, I think it might have been changed recently | 17:19 |
| dviroel | hum, so it doesn't support only host@backend? | 17:21 |
| dviroel | IIRC, in manila, for driver that don't report pools, it creates a default pool for the entire backend in this case... | 17:21 |
| sean-k-mooney | at somepoint they added added pool | 17:22 |
| sean-k-mooney | but they never updated the api ref to mention it | 17:22 |
| sean-k-mooney | https://docs.openstack.org/api-ref/block-storage/v3/index.html#migrate-a-volume sayus | 17:23 |
| sean-k-mooney | host is "The target host for the volume migration. Host format is host@backend. Required before microversion 3.16." | 17:23 |
| dviroel | yeah | 17:23 |
| dviroel | cinder also creates a default pool for drivers that don't report pools | 17:24 |
| dviroel | https://github.com/openstack/cinder/blob/d02171164bdd702b12b59888b744d172f30d712d/cinder/scheduler/host_manager.py#L252-L260 | 17:24 |
| sean-k-mooney | you say that but we have a bug report where it does not | 17:24 |
| sean-k-mooney | and i have checked that with ceph i think | 17:24 |
| sean-k-mooney | and it didint | 17:24 |
| dviroel | yeah, i remember | 17:24 |
| sean-k-mooney | https://bugs.launchpad.net/watcher/+bug/2088118 | 17:26 |
| jgilaber | yes I think I also checked with ceph and nfs and the pool_name is not there when reporting the pool capabilities | 17:26 |
| jgilaber | in my env this command works well openstack volume migrate test_migrate --host jgilaber-watcher-2@lvmdriver-2 | 17:26 |
| jgilaber | and so does openstack volume migrate test_migrate --host jgilaber-watcher-3@lvmdriver-3#lvmdriver-3 | 17:27 |
| jgilaber | unless this has changed in the flamingo release, my env is a bit old | 17:27 |
| sean-k-mooney | it should not | 17:27 |
| sean-k-mooney | i think pool is only requried if there are pools even then that not entrily clear | 17:28 |
| sean-k-mooney | jgilaber: can you run https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/volume-backend.html#volume-backend-pool-list | 17:28 |
| sean-k-mooney | openstack volume backend pool list --long | nc termbin.com 9999 | 17:29 |
| jgilaber | sure https://termbin.com/zyvr | 17:33 |
| jgilaber | I've messed a bit with the backend configs in my environment, so it doesn't follow the default deployment config | 17:33 |
| jgilaber | I've changed backend and pool names | 17:33 |
| sean-k-mooney | ack you have 4 backends on 3 hosts | 17:34 |
| sean-k-mooney | 2 on jgilaber-watcher-3 and one on the othef 2 | 17:34 |
| jgilaber | correct | 17:34 |
| jgilaber | I also changed poolnames to configure types that would only be scheduled on some hosts | 17:35 |
| sean-k-mooney | can you do the same with --debug i want to see what the raw json api output is | 17:35 |
| sean-k-mooney | well i can kidn of see what i want i guess | 17:36 |
| sean-k-mooney | name is the fully qualifed name and pool_name is just the part oafter hte # | 17:36 |
| jgilaber | here it is just in case https://termbin.com/iy5b | 17:36 |
| jgilaber | that's right | 17:37 |
| jgilaber | also the pool name matches the value of volume_backend_name in the backend configuration, which I found surprising | 17:37 |
| dviroel | yeah, that's what the scheduler does if no pool is reported | 17:38 |
| dviroel | it creates one pool with the same name of the backend | 17:38 |
| dviroel | so we always have a pool | 17:38 |
| dviroel | the problem is that not always have the attribute: "pool_name" | 17:39 |
| jgilaber | ack, that makes sense, and tbh I found it documented somewhere | 17:39 |
| dviroel | https://www.irccloud.com/pastebin/9ST8oQXS/ | 17:39 |
| sean-k-mooney | jgilaber: so the reason i wanted the debug with the raw json was to see if it was the client or the api that was doing tata | 17:39 |
| dviroel | check this get_pools from a cinder job with ceph backend | 17:40 |
| dviroel | "name": "np3b49a2e5bd594@ceph#ceph" - "but no pool_name" | 17:40 |
| sean-k-mooney | right | 17:40 |
| jgilaber | dviroel, that matches what I remember, same for nfs | 17:40 |
| sean-k-mooney | so its not in the api resonce | 17:40 |
| jgilaber | it's driver dependent | 17:41 |
| sean-k-mooney | its added to the name but the pool_name key is not injected | 17:41 |
| sean-k-mooney | ya which also matches teh bug | 17:41 |
| sean-k-mooney | https://bugs.launchpad.net/watcher/+bug/2088118 | 17:41 |
| sean-k-mooney | so name will alwasy have a pool which will just be a copy of the backend name if there is no pool name | 17:42 |
| jgilaber | so it looks to me like watcher is doing the right thing, but it should call it 'host' instead of 'pool' | 17:43 |
| dviroel | yes, the pool really exist in the scheduler, but it doesn not populate all capabilites, since it does not populate "pool_name", not sure about others | 17:44 |
| sean-k-mooney | we shoudl proably ignore the pool_name filed entirly and just parse "name" | 17:44 |
| sean-k-mooney | and ya in the input we shoudl jsut call it host since that is the api field or maybe something like FQBN fully qulaifed backend name | 17:46 |
| dviroel | yeah, we should | 17:47 |
| sean-k-mooney | host is clsoe to what the cli uses but i think the imporant thing for use is just to docuemnt the format proplry and nodte that the pool (after the #) is option | 17:47 |
| sean-k-mooney | *optional | 17:47 |
| jgilaber | ack, I'll get started on a spec for that tomorrow | 17:49 |
| sean-k-mooney | ok im going to call it a day. cool can you summerise how we want to proceed tomorrwo in the irc meeting | 17:49 |
| sean-k-mooney | i.e what part you want to do as a bug vs the rest | 17:50 |
| jgilaber | I will also redeploy my env and try again, to ensure that with an up to date deployment the os-migrate_volume api accepts both with and pool name | 17:50 |
| jgilaber | yep, I'll do that tomorrow | 17:50 |
| sean-k-mooney | thanks | 17:50 |
| jgilaber | thanks for the discussion, this has been immensely helpful! | 17:50 |
| dviroel | ++ tks folks | 17:54 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!