*** tkajinam has quit IRC | 00:12 | |
*** tejdeep has quit IRC | 00:16 | |
*** takamatsu has quit IRC | 00:18 | |
*** dklyle has joined #openstack-cinder | 00:28 | |
*** lixiaoy1 has joined #openstack-cinder | 00:33 | |
*** brinzhang has joined #openstack-cinder | 00:38 | |
*** brinzhang_ has joined #openstack-cinder | 00:45 | |
*** brinzhang has quit IRC | 00:47 | |
*** brinzhang_ has quit IRC | 00:52 | |
*** brinzhang has joined #openstack-cinder | 00:54 | |
*** Liang__ has joined #openstack-cinder | 01:02 | |
*** spatel has joined #openstack-cinder | 01:16 | |
*** imacdonn has quit IRC | 01:17 | |
*** imacdonn has joined #openstack-cinder | 01:18 | |
*** spatel has quit IRC | 01:20 | |
*** Kuirong has joined #openstack-cinder | 01:21 | |
*** ruffian_sheep has joined #openstack-cinder | 01:25 | |
ruffian_sheep | whoami-rajat:Hello, last week's driver submission did not receive feedback on the changes. Is it already starting to consider joining the main line?;) | 01:32 |
---|---|---|
openstackgerrit | Chris M proposed openstack/cinder master: Fix dothill multiattach support https://review.opendev.org/671942 | 01:40 |
*** ruffian_sheep has quit IRC | 01:41 | |
*** ruffian_sheep has joined #openstack-cinder | 01:46 | |
*** ruffian_sheep has quit IRC | 02:08 | |
*** zul has quit IRC | 02:09 | |
*** tejdeep has joined #openstack-cinder | 02:17 | |
*** ruffian_sheep has joined #openstack-cinder | 02:21 | |
*** bhagyashris has joined #openstack-cinder | 02:27 | |
*** tejdeep has quit IRC | 02:30 | |
*** ruffian_sheep has quit IRC | 02:31 | |
*** ruffian_sheep has joined #openstack-cinder | 02:37 | |
openstackgerrit | Chris M proposed openstack/cinder master: Fix dothill multiattach support https://review.opendev.org/671942 | 02:38 |
openstackgerrit | Chris M proposed openstack/cinder master: WIP: Create Seagate driver from dothill driver https://review.opendev.org/671195 | 02:38 |
*** ruffian_sheep27 has joined #openstack-cinder | 02:38 | |
*** ruffian_sheep has quit IRC | 02:41 | |
*** baojg has quit IRC | 03:01 | |
*** m75abrams has joined #openstack-cinder | 03:05 | |
*** baojg has joined #openstack-cinder | 03:07 | |
*** psachin has joined #openstack-cinder | 03:32 | |
*** ruffian_sheep27 is now known as ruffian_sheep | 03:36 | |
*** baojg has quit IRC | 03:47 | |
*** baojg has joined #openstack-cinder | 03:58 | |
*** udesale has joined #openstack-cinder | 04:02 | |
*** baojg has quit IRC | 04:28 | |
*** tejdeep has joined #openstack-cinder | 04:55 | |
*** ircuser-1 has quit IRC | 04:57 | |
*** threestrands has joined #openstack-cinder | 05:00 | |
*** davee_ has quit IRC | 05:03 | |
*** davee_ has joined #openstack-cinder | 05:03 | |
*** Luzi has joined #openstack-cinder | 05:03 | |
openstackgerrit | Chris M proposed openstack/cinder master: Create Seagate driver from dothill driver https://review.opendev.org/671195 | 05:44 |
-openstackstatus- NOTICE: Due to a failure on the logs.openstack.org volume, old logs are unavailable while partition is recovered. New logs are being stored. ETA for restoration probably ~Mon Jul 22 12:00 UTC 2019 | 06:03 | |
*** ChanServ changes topic to "Due to a failure on the logs.openstack.org volume, old logs are unavailable while partition is recovered. New logs are being stored. ETA for restoration probably ~Mon Jul 22 12:00 UTC 2019" | 06:03 | |
*** whoami-rajat has joined #openstack-cinder | 06:05 | |
*** vishalmanchanda has joined #openstack-cinder | 06:05 | |
*** udesale has quit IRC | 06:16 | |
*** ChanServ changes topic to "The Block Storage Project | https://wiki.openstack.org/wiki/Cinder | https://tiny.cc/CinderPriorities" | 06:22 | |
-openstackstatus- NOTICE: logs.openstack.org volume has been restored. please report any issues in #openstack-infra | 06:22 | |
*** markvoelker has quit IRC | 06:32 | |
*** pcaruana has joined #openstack-cinder | 06:32 | |
*** baojg has joined #openstack-cinder | 06:41 | |
ruffian_sheep | whoami-rajat:Hello, last week's driver submission did not receive feedback on the changes. Is it already starting to consider joining the main line?;) | 06:44 |
ruffian_sheep | https://review.opendev.org/#/c/612311/ | 06:45 |
*** udesale has joined #openstack-cinder | 06:47 | |
*** boxiang has joined #openstack-cinder | 06:57 | |
*** sahid has joined #openstack-cinder | 06:58 | |
*** tesseract has joined #openstack-cinder | 07:01 | |
*** markvoelker has joined #openstack-cinder | 07:04 | |
*** sahid has quit IRC | 07:05 | |
*** rcernin has quit IRC | 07:07 | |
*** tkajinam has joined #openstack-cinder | 07:08 | |
*** tosky has joined #openstack-cinder | 07:21 | |
*** tkajinam has quit IRC | 07:36 | |
*** tejdeep has quit IRC | 07:52 | |
*** nikeshm has joined #openstack-cinder | 08:01 | |
*** davidsha has joined #openstack-cinder | 08:06 | |
*** m75abrams has quit IRC | 08:12 | |
*** sahid has joined #openstack-cinder | 08:14 | |
*** threestrands has quit IRC | 08:14 | |
*** sahid has quit IRC | 08:24 | |
*** sahid has joined #openstack-cinder | 08:24 | |
*** m75abrams has joined #openstack-cinder | 08:28 | |
*** ociuhandu has joined #openstack-cinder | 08:37 | |
*** ociuhandu has quit IRC | 08:39 | |
*** ociuhandu has joined #openstack-cinder | 08:39 | |
*** lixiaoy1 has quit IRC | 08:53 | |
*** sahid has quit IRC | 08:55 | |
*** sahid has joined #openstack-cinder | 08:56 | |
*** e0ne has joined #openstack-cinder | 09:04 | |
openstackgerrit | ye proposed openstack/cinder master: This fix let the delete err-info more precisely https://review.opendev.org/671996 | 09:06 |
*** sahid has quit IRC | 09:17 | |
*** lemko has joined #openstack-cinder | 09:17 | |
*** ruffian_sheep has quit IRC | 09:24 | |
*** Liang__ has quit IRC | 09:37 | |
*** udesale has quit IRC | 09:40 | |
*** udesale has joined #openstack-cinder | 09:40 | |
*** udesale has quit IRC | 09:42 | |
*** udesale has joined #openstack-cinder | 09:42 | |
*** FlorianFa has joined #openstack-cinder | 09:42 | |
*** ociuhandu has quit IRC | 09:45 | |
*** ociuhandu has joined #openstack-cinder | 09:48 | |
*** m75abrams has quit IRC | 09:54 | |
openstackgerrit | Raghavendra Tilay proposed openstack/cinder master: 3PAR: Provide new option to specify NSP for single path attachments https://review.opendev.org/657585 | 09:58 |
*** raghavendrat has joined #openstack-cinder | 09:59 | |
*** nikeshm has quit IRC | 10:11 | |
*** udesale has quit IRC | 10:28 | |
*** sahid has joined #openstack-cinder | 10:31 | |
*** nikeshm has joined #openstack-cinder | 10:42 | |
openstackgerrit | Raghavendra Tilay proposed openstack/cinder master: 3PAR: Provide new option to specify NSP for single path attachments https://review.opendev.org/657585 | 10:46 |
openstackgerrit | Brin Zhang proposed openstack/cinder master: Rollback the quota_usages table when failed to create a incremental backup https://review.opendev.org/626790 | 10:50 |
openstackgerrit | Merged openstack/cinder master: Cleanup api-ref sample files https://review.opendev.org/668972 | 10:58 |
*** bhagyashris has quit IRC | 10:59 | |
*** ruffian_sheep has joined #openstack-cinder | 11:02 | |
*** rosmaita has joined #openstack-cinder | 11:07 | |
*** ruffian_sheep has quit IRC | 11:08 | |
*** kaisers has quit IRC | 11:35 | |
*** kaisers has joined #openstack-cinder | 11:36 | |
*** tejdeep has joined #openstack-cinder | 11:42 | |
*** sahid has quit IRC | 11:49 | |
*** psachin has quit IRC | 11:50 | |
*** spatel has joined #openstack-cinder | 11:54 | |
*** sahid has joined #openstack-cinder | 12:03 | |
*** carloss has joined #openstack-cinder | 12:06 | |
*** udesale has joined #openstack-cinder | 12:07 | |
*** Kuirong has quit IRC | 12:09 | |
*** spatel has quit IRC | 12:13 | |
*** tejdeep has quit IRC | 12:14 | |
*** raghavendrat has quit IRC | 12:17 | |
*** _erlon_ has joined #openstack-cinder | 12:21 | |
*** raghavendrat has joined #openstack-cinder | 12:23 | |
*** dklyle has quit IRC | 12:36 | |
*** david-lyle has joined #openstack-cinder | 12:36 | |
openstackgerrit | Pawel Kaminski proposed openstack/os-brick master: connectors/nvme: Wait utill nvme device show up in kernel https://review.opendev.org/672031 | 12:39 |
*** irclogbot_0 has quit IRC | 12:39 | |
*** irclogbot_1 has joined #openstack-cinder | 12:42 | |
*** sahid has quit IRC | 12:45 | |
*** sahid has joined #openstack-cinder | 12:46 | |
*** boxiang has quit IRC | 12:50 | |
*** beraldo has joined #openstack-cinder | 12:51 | |
beraldo | Hi all, hope you guys help me here. I'm trying to deploy openstack with two AZ and one cinder server on each AZ. Looks like it is working fine when I'm trying to create a volume. If a choose the proper AZ, the volume will be deployed correctly. The problem is when I try to lanch an instance and choose any zone. The instance is not deployed because looks like the filters is retuning a empty | 12:56 |
beraldo | resources list because by default is going to "nova" AZ. Anyone here can help ? I already did a debug on code but without success... Please let me know if this is not the channel. Thanks in advance | 12:56 |
*** mchlumsky has quit IRC | 13:01 | |
*** mchlumsky has joined #openstack-cinder | 13:01 | |
raghavendrat | beraldo: as per my knowledge, generic queries can be posted at #openstack-qa | 13:07 |
beraldo | thanks raghavendrat | 13:07 |
openstackgerrit | Pawel Kaminski proposed openstack/os-brick master: connectors/nvme: Wait utill nvme device show up in kernel https://review.opendev.org/672031 | 13:10 |
*** eharney has joined #openstack-cinder | 13:10 | |
*** mriedem has joined #openstack-cinder | 13:11 | |
*** sahid has quit IRC | 13:22 | |
*** sahid has joined #openstack-cinder | 13:25 | |
openstackgerrit | Chris M proposed openstack/cinder master: Fix dothill multiattach support https://review.opendev.org/671942 | 13:26 |
*** sahid has quit IRC | 13:30 | |
*** sahid has joined #openstack-cinder | 13:33 | |
*** sahid has quit IRC | 13:38 | |
*** sahid has joined #openstack-cinder | 13:38 | |
*** spatel has joined #openstack-cinder | 13:40 | |
*** Luzi has quit IRC | 13:48 | |
*** raghavendrat has quit IRC | 13:49 | |
*** nikeshm has quit IRC | 13:49 | |
*** TxGirlGeek has joined #openstack-cinder | 13:49 | |
*** TxGirlGeek has quit IRC | 13:52 | |
*** tesseract has quit IRC | 13:55 | |
*** tesseract has joined #openstack-cinder | 14:00 | |
mnaser | so i think i'm running into this again -- https://bugs.launchpad.net/cinder/+bug/1641312 | 14:03 |
openstack | Launchpad bug 1641312 in Cinder "CleanableInUse error in a deployment with just one cinder-volume" [High,Fix released] - Assigned to Gorka Eguileor (gorka) | 14:03 |
*** senrique_ has joined #openstack-cinder | 14:04 | |
mnaser | ok, so doing a little bit of reserach, is there any reason why the worker for create_volume is created inside the scheduler rpcapi layer | 14:11 |
mnaser | but the rest are not? | 14:11 |
mnaser | i'm trying to avoid this race condition by trying to bubble up the worker creation code to be somewhere where no race conditions are possible (like api layer or scheduler) | 14:12 |
openstackgerrit | Mohammed Naser proposed openstack/cinder master: docs: fix incorrect reference in ha docs https://review.opendev.org/672054 | 14:19 |
mnaser | well, now i understand it | 14:27 |
mnaser | i'm wondering if this is because there are multiple cinder-volumes with the same `host` value | 14:28 |
mnaser | so doing a cast means multiple workers are picking it up | 14:28 |
mnaser | and they just kinda race there | 14:28 |
mnaser | i wonder why create_snapshot() needs to do a cast, shouldn't it be a call because we know the c-vol we're hitting anyways? | 14:29 |
*** m75abrams has joined #openstack-cinder | 14:47 | |
mnaser | ok so the more i look at this, the more i feel like this should be a log message, and not an exception -- https://github.com/openstack/cinder/blob/33fa38b0350ed17c87858563898ff2f55b9f368b/cinder/objects/cleanable.py#L152-L155 | 14:53 |
mnaser | because if we reach that part of the code, it means that a worker *was* created under another service_id | 14:53 |
mnaser | which means, we raced, but lost, so nothing *bad* happened | 14:54 |
*** spatel has quit IRC | 14:55 | |
*** ircuser-1 has joined #openstack-cinder | 14:57 | |
geguileo | mnaser: what PyMySQL version are you using? | 15:00 |
mnaser | geguileo: this is an OSA deployment, so we use whatever upper constraints / requirements ships | 15:01 |
mnaser | let em verify | 15:01 |
openstackgerrit | Pawel Kaminski proposed openstack/cinder master: target/spdknvmf: Add configuration parameter https://review.opendev.org/672064 | 15:03 |
mnaser | geguileo: according to upper constraints, PyMySQL===0.8.0 | 15:04 |
* geguileo looks if they messed it up again... | 15:05 | |
mnaser | i'm double checking the actual environment | 15:05 |
mnaser | that was just based on reading upper-constraints | 15:05 |
mnaser | /me looks at queens CI deployas | 15:05 |
hemna | mep | 15:07 |
mnaser | yeah we're doing pymysql-0.8.0 | 15:07 |
geguileo | mnaser: on what operation are you seeing the error? | 15:07 |
mnaser | geguileo: most commonly, snapshot create and volume deletes | 15:08 |
mnaser | which are the ones that the worker is created inside the c-vol instead of c-sch | 15:08 |
mnaser | (im not sure why we don't create the worker all the way up at the http api layer) | 15:08 |
mnaser | but then i'd imagine we'd have a different type of race.. | 15:09 |
mnaser | context: this is one of those deployments where there is multiple c-vol against a single ceph cluster with the same `host` value | 15:09 |
geguileo | wait, wait, wait | 15:09 |
geguileo | mnaser: are multiple c-vol services running with the same host value? | 15:10 |
mnaser | geguileo: yes, so cinder service-list reports a single host in this case. i thought at some point that this was an ok thing to do | 15:10 |
mnaser | i dont remember where/how/when but i recall that | 15:10 |
geguileo | NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO | 15:10 |
mnaser | :( | 15:10 |
geguileo | YOU CANNOT DO THAT | 15:10 |
mnaser | we recently landed a change in osa that did it for everyone | 15:10 |
geguileo | and that is the problem | 15:10 |
mnaser | ha | 15:11 |
mnaser | https://review.opendev.org/#/c/623490/ | 15:11 |
mnaser | so this is a bad idea(tm) ? | 15:11 |
geguileo | that is how you do it if you are running Active-Passive | 15:12 |
geguileo | But only 1 c-vol service can be running at a time | 15:12 |
mnaser | so that implies that only 1 c-vol at a time | 15:12 |
mnaser | ah ok | 15:12 |
mnaser | i wonder how tripleo handles this | 15:12 |
geguileo | deploying active-passive | 15:13 |
geguileo | XD | 15:13 |
mnaser | using pacemaker ? | 15:13 |
geguileo | yup | 15:13 |
mnaser | sigh | 15:13 |
mnaser | hm | 15:13 |
geguileo | there is an ongoing effort to start adding active-active | 15:13 |
geguileo | which means that you start using cluster | 15:13 |
geguileo | which is probably what you want to do | 15:13 |
geguileo | leave the host alone | 15:13 |
geguileo | and set cluster | 15:13 |
mnaser | but that hasn't landed yet? so that means really what needs to be done is: revert that commit | 15:13 |
geguileo | what hasn't landed? | 15:14 |
mnaser | active-active | 15:14 |
geguileo | the cluster stuff? | 15:14 |
geguileo | it has | 15:14 |
mnaser | oh, master or? | 15:14 |
geguileo | just replace backend_host with cluster and it should be fine | 15:14 |
mnaser | how long has it been around for? | 15:15 |
geguileo | I don't remember | 15:15 |
geguileo | the cluster feature a while | 15:15 |
mnaser | ok, ill look for it | 15:15 |
mnaser | and then fix osa | 15:15 |
geguileo | RBD supporting active-active less time | 15:15 |
mnaser | ok ill check out the rbd driver logs | 15:15 |
geguileo | not all drivers support A/A deployments | 15:15 |
geguileo | mnaser: there's a class attribute saying that it is supported | 15:15 |
geguileo | mmmmm, I don't see it there anymore... | 15:17 |
eharney | SUPPORTS_ACTIVE_ACTIVE is in RBDDriver | 15:18 |
geguileo | eharney: I was looking in github... | 15:19 |
geguileo | mnaser: https://opendev.org/openstack/cinder/src/branch/master/cinder/volume/drivers/rbd.py#L223 | 15:19 |
geguileo | eharney: and apparently that oaf | 15:19 |
hemna | morning | 15:19 |
geguileo | eharney: oh, I hit rsd intead of rbd (facepalm) | 15:19 |
geguileo | mnaser: looks like it was added in Rocky https://github.com/openstack/cinder/commit/245a488c36003764e3550c2c95fa4bef6119e0ea | 15:21 |
hemna | geguileo: so that issue with changing the scan mode to manual is still around | 15:22 |
jbernard | mnaser: i did test it when i submitted that patch, appeared to work at the time | 15:22 |
hemna | the customer can't isolate the iscsi sessions to just 1 or a subset of their target portals | 15:22 |
hemna | due to needing multipath attachments | 15:22 |
hemna | so they are basically hosed with os-brick forcing it to manual | 15:23 |
hemna | rebooting the host loses drives | 15:23 |
hemna | since the scan mode is set to manual | 15:23 |
hemna | so I think we need a config option to allow folks to force os-brick to leave it to auto | 15:23 |
hemna | but adding that config in nova is a major pita | 15:24 |
geguileo | I think we need to look for an alternative way to make both be able to coexist | 15:24 |
hemna | I'm not sure how to solve it | 15:24 |
geguileo | in os-brick itself | 15:24 |
hemna | both settings having problems | 15:24 |
geguileo | I'll finish with what I'm working on and see if I can come up with something... | 15:25 |
mnaser | ok great, so we'll fix it in openstack-ansible, thanks jbernard and geguileo. | 15:26 |
*** lpetrut has joined #openstack-cinder | 15:30 | |
mnaser | geguileo: well i guess this is still relevant https://review.opendev.org/#/c/672054/ ? | 15:32 |
geguileo | mnaser: that sentence is now also incorrect, because snapshot creation goes through the scheduler as well | 15:34 |
geguileo | and I don't know if we are creating the worker in the scheduler as we should or not | 15:34 |
mnaser | geguileo: we are def creating it in the volume rpcapi in what i see | 15:35 |
mnaser | create_worker() happens only for new volume creation in scheduler | 15:35 |
geguileo | mnaser: then that's probably a bug as well | 15:35 |
mnaser | for delete vol and create snapshot, it happens in vol api | 15:35 |
mnaser | ok, let me fix that then | 15:35 |
geguileo | mnaser: but I'm not 100% sure, it needs to be checked | 15:36 |
*** lemko has quit IRC | 15:37 | |
openstackgerrit | Mohammed Naser proposed openstack/cinder master: workers: create snapshot worker in scheduler rpcapi https://review.opendev.org/672054 | 15:39 |
mnaser | geguileo: ^ thats to be checked by whomever | 15:39 |
geguileo | mnaser: you'll need to update the unittests | 15:40 |
mnaser | oh yeah | 15:40 |
openstackgerrit | Mohammed Naser proposed openstack/cinder master: workers: create snapshot worker in scheduler rpcapi https://review.opendev.org/672054 | 15:42 |
*** henriqueof has quit IRC | 15:43 | |
*** henriqueof has joined #openstack-cinder | 15:54 | |
*** sahid has quit IRC | 15:54 | |
*** henriqueof has quit IRC | 15:55 | |
mnaser | i cant seem to find any docs on how to add/configure active-active | 15:57 |
mnaser | am i not googliing the right terms? | 15:57 |
*** henriqueof has joined #openstack-cinder | 15:57 | |
jungleboyj | mnaser: I thought geguileo had a blog on best practices for setting up an active/active env. | 15:57 |
geguileo | jungleboyj: he's looking into how it should be deployed for osa | 15:58 |
geguileo | and I'm not sure we have that doc anywhere | 15:58 |
jungleboyj | geguileo: Ah, ok. Yeah, we probably don't have that documented yet. | 15:58 |
jungleboyj | :-( | 15:59 |
mnaser | nono | 15:59 |
mnaser | i will make OSA support it :) | 15:59 |
mnaser | so docs about how its done in cinder period are good enough | 15:59 |
geguileo | mnaser: yeah, I know | 15:59 |
geguileo | but you are looking for how it should be deployed so you can add it to osa | 15:59 |
mnaser | yep.. i cant find any docs right now | 16:00 |
*** sahid has joined #openstack-cinder | 16:00 | |
geguileo | that's what I meant to say (and I think jungleboyj knows me enough to understand the meaning from my weird sentences) | 16:00 |
geguileo | mnaser: https://docs.openstack.org/cinder/latest/contributor/high_availability.html | 16:00 |
geguileo | mnaser: that's what I wrote for developers | 16:01 |
geguileo | but it doesn't cover the deployment | 16:01 |
jungleboyj | Yeah, I knew what geguileo meant. | 16:01 |
*** tejdeep has joined #openstack-cinder | 16:01 | |
geguileo | jeje | 16:01 |
jungleboyj | That was the documentation I found as well. | 16:01 |
mnaser | yeah that seems awesomely detailed :P | 16:01 |
geguileo | mnaser: iirc you only need to set the cluster conf option | 16:01 |
mnaser | "For Active-Active configurations we need to include the Volume services that will be managing the same backends on the cluster. To include a node in a cluster, we need to define its name in the [DEFAULT] section using the cluster configuration option, and start or restart the service." | 16:02 |
geguileo | and not set the host field or backend_host fields | 16:02 |
mnaser | "The name of the cluster must be unique and cannot match any of the host or backend_host values. Non unique values will generate duplicated names for message queues." | 16:02 |
mnaser | that poretty much covers it i guess | 16:02 |
geguileo | yup | 16:02 |
geguileo | and NOT defining the same host conf on multiple c-vol services | 16:02 |
geguileo | or c-scheduler services | 16:03 |
mnaser | can i put this inside the backend settings? | 16:03 |
mnaser | or is it a c-vol top level config | 16:03 |
mnaser | https://github.com/openstack/cinder | 16:03 |
mnaser | o no is broken | 16:03 |
geguileo | I may have set it to be a global thing... | 16:04 |
* geguileo looks | 16:04 | |
geguileo | mnaser: it's global, like "host" | 16:05 |
mnaser | ah ack | 16:05 |
mnaser | ok will implement in osa | 16:05 |
*** e0ne has quit IRC | 16:05 | |
*** tejdeep has quit IRC | 16:06 | |
*** udesale has quit IRC | 16:08 | |
openstackgerrit | Pawel Kaminski proposed openstack/os-brick master: connectors/nvme: Wait utill nvme device show up in kernel https://review.opendev.org/672031 | 16:08 |
mnaser | geguileo: https://review.opendev.org/#/c/672078/ | 16:14 |
*** lpetrut has quit IRC | 16:16 | |
*** senrique_ is now known as enriquetaso | 16:21 | |
*** sahid has quit IRC | 16:23 | |
*** davidsha has quit IRC | 16:24 | |
openstackgerrit | Andreas Jaeger proposed openstack/cinder master: Update api-ref location https://review.opendev.org/672087 | 16:28 |
*** deiter has joined #openstack-cinder | 16:29 | |
*** baojg has quit IRC | 16:42 | |
*** baojg has joined #openstack-cinder | 16:43 | |
*** dviroel has joined #openstack-cinder | 16:43 | |
*** baojg has quit IRC | 16:43 | |
*** baojg has joined #openstack-cinder | 16:43 | |
*** baojg has quit IRC | 16:44 | |
*** baojg has joined #openstack-cinder | 16:44 | |
*** baojg has quit IRC | 16:44 | |
*** baojg has joined #openstack-cinder | 16:45 | |
*** baojg has quit IRC | 16:45 | |
*** baojg has joined #openstack-cinder | 16:46 | |
*** baojg has quit IRC | 16:46 | |
*** jrubenst has joined #openstack-cinder | 16:46 | |
*** baojg has joined #openstack-cinder | 16:47 | |
*** baojg has quit IRC | 16:47 | |
*** baojg has joined #openstack-cinder | 16:47 | |
*** baojg has quit IRC | 16:48 | |
*** baojg has joined #openstack-cinder | 16:48 | |
*** baojg has quit IRC | 16:48 | |
*** baojg has joined #openstack-cinder | 16:49 | |
*** baojg has quit IRC | 16:49 | |
*** baojg has joined #openstack-cinder | 16:50 | |
*** baojg has quit IRC | 16:50 | |
*** baojg has joined #openstack-cinder | 16:51 | |
*** baojg has quit IRC | 16:51 | |
*** baojg has joined #openstack-cinder | 16:51 | |
*** baojg has quit IRC | 16:52 | |
*** baojg has joined #openstack-cinder | 16:52 | |
*** baojg has quit IRC | 16:52 | |
*** baojg has joined #openstack-cinder | 16:53 | |
*** baojg has quit IRC | 16:53 | |
*** baojg has joined #openstack-cinder | 16:54 | |
*** baojg has quit IRC | 16:54 | |
*** baojg has joined #openstack-cinder | 16:55 | |
*** baojg has quit IRC | 16:55 | |
*** baojg has joined #openstack-cinder | 16:55 | |
*** baojg has quit IRC | 16:56 | |
*** baojg has joined #openstack-cinder | 16:56 | |
*** baojg has quit IRC | 16:56 | |
*** baojg has joined #openstack-cinder | 16:57 | |
*** baojg has quit IRC | 16:57 | |
*** baojg has joined #openstack-cinder | 16:58 | |
*** henriqueof has quit IRC | 16:58 | |
*** baojg has quit IRC | 16:58 | |
*** baojg has joined #openstack-cinder | 16:58 | |
*** baojg has quit IRC | 16:59 | |
*** baojg has joined #openstack-cinder | 16:59 | |
*** baojg has quit IRC | 16:59 | |
*** e0ne has joined #openstack-cinder | 17:05 | |
*** mvkr_ has quit IRC | 17:18 | |
jungleboyj | e0ne: Are you still around? | 17:22 |
e0ne | jungleboyj: hi. yes | 17:30 |
jungleboyj | e0ne: Hey, I think I got the problem resolved. Had Horizon throwing a 500 error. For some reason it thought the logs were inaccessible. | 17:31 |
jungleboyj | I restarted the container and then it was fine. | 17:31 |
jungleboyj | Ever seen anything like that? | 17:31 |
e0ne | need to see logs | 17:32 |
*** jmlowe has quit IRC | 17:34 | |
openstackgerrit | Sofia Enriquez proposed openstack/cinder master: Support Incremental Backup Completion In RBD https://review.opendev.org/627941 | 17:35 |
jungleboyj | Ok. | 17:38 |
jungleboyj | https://www.irccloud.com/pastebin/S0Z5MlYO/ | 17:39 |
jungleboyj | e0ne: ^^^ That was what I was seeing. | 17:39 |
e0ne | jungleboyj: looks like you don't have permissions to write logs:( | 17:46 |
*** ociuhandu_ has joined #openstack-cinder | 17:47 | |
BLZbubba | hi guys, i'm doing some benchmarks here on a cinder-lvm/iscsi volume and the effective queue depth on the 4 RAID10 devices is < 1, but the guest OS fio process has a queue depth of 128 (iostat shows effectively it is well over 100). obviously this is killing performance on the SSDs which like a queue depth of 50+ if possible | 17:47 |
BLZbubba | block size is 1M | 17:48 |
jungleboyj | Yep, that was what I determined too. Restarting the container fixed it. | 17:48 |
BLZbubba | is there a good way to determine who is limiting the number of simultaneous operations? i am actually starting to suspect mdadm | 17:49 |
e0ne | hm... did you deploy your env via openstack-helm? | 17:49 |
jungleboyj | Probably should have poked around in the containter more first. | 17:49 |
jungleboyj | e0ne: This was an RHOSP deployment | 17:49 |
*** ociuhandu has quit IRC | 17:49 | |
*** ociuhandu_ has quit IRC | 17:52 | |
e0ne | BLZbubba: what target driver do you use? if it's tgtd you can try to change it's configuration via 'iscsi_target_flags'iscsi_target_flags config opt | 17:52 |
e0ne | jungleboyj: I didn't try that distro | 17:56 |
BLZbubba | yes tgtd, but I was using a non-raid local hard drive earlier and it didn't seem to have the same problem. guess i should get a second local drive & RAID 1 and see what happens | 17:56 |
jungleboyj | Ok. | 17:57 |
BLZbubba | the sad thing is i was trying this method because ceph kept nuking my effective queue depth to below 1 | 17:57 |
*** jmlowe has joined #openstack-cinder | 17:59 | |
*** tesseract has quit IRC | 18:01 | |
*** tejdeep has joined #openstack-cinder | 18:13 | |
*** deiter has quit IRC | 18:25 | |
*** m75abrams has quit IRC | 18:26 | |
*** jmlowe has quit IRC | 18:39 | |
*** henriqueof has joined #openstack-cinder | 18:43 | |
*** baojg has joined #openstack-cinder | 19:01 | |
*** e0ne has quit IRC | 19:01 | |
*** baojg has quit IRC | 19:05 | |
openstackgerrit | Brian Rosmaita proposed openstack/cinder master: Add options to [service_user] sample config https://review.opendev.org/672145 | 19:09 |
openstackgerrit | Eric Harney proposed openstack/cinder master: Prevent double-attachment race in attachment_reserve https://review.opendev.org/671370 | 19:17 |
*** deiter has joined #openstack-cinder | 19:19 | |
deiter | Hello, please review https://review.opendev.org/#/c/669736/ thank you! | 19:22 |
*** eharney has quit IRC | 19:23 | |
*** Roamer` has quit IRC | 19:30 | |
*** beraldo has quit IRC | 19:32 | |
*** e0ne has joined #openstack-cinder | 19:50 | |
*** eharney has joined #openstack-cinder | 20:02 | |
*** e0ne has quit IRC | 20:04 | |
*** jmlowe has joined #openstack-cinder | 20:10 | |
*** whoami-rajat has quit IRC | 21:01 | |
*** mvkr_ has joined #openstack-cinder | 21:04 | |
*** enriquetaso has quit IRC | 21:05 | |
*** pcaruana has quit IRC | 21:10 | |
*** jrubenst has quit IRC | 21:11 | |
*** jrubenst has joined #openstack-cinder | 21:31 | |
*** jrubenst has quit IRC | 21:44 | |
*** jrubenst has joined #openstack-cinder | 21:54 | |
*** jrubenst has quit IRC | 22:02 | |
*** jrubenst has joined #openstack-cinder | 22:05 | |
*** jrubenst has quit IRC | 22:10 | |
*** jrubenst has joined #openstack-cinder | 22:11 | |
*** Kuirong has joined #openstack-cinder | 22:13 | |
*** jrubenst has quit IRC | 22:19 | |
*** rcernin has joined #openstack-cinder | 22:30 | |
*** mriedem has quit IRC | 22:39 | |
*** tkajinam has joined #openstack-cinder | 22:39 | |
*** tosky has quit IRC | 22:49 | |
*** enriquetaso has joined #openstack-cinder | 22:58 | |
*** baojg has joined #openstack-cinder | 23:02 | |
*** baojg has quit IRC | 23:07 | |
*** henriqueof has quit IRC | 23:14 | |
*** carloss has quit IRC | 23:15 | |
*** tejdeep has quit IRC | 23:34 | |
*** Kuirong has quit IRC | 23:53 | |
*** TxGirlGeek has joined #openstack-cinder | 23:54 | |
*** tejdeep has joined #openstack-cinder | 23:56 | |
*** TxGirlGeek has quit IRC | 23:56 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!