Tuesday, 2021-11-02

timburke_andybotting, proxyfs is still a thing, but i'm not entirely sure of its development state. i know it's moved homes -- now at https://github.com/NVIDIA/proxyfs -- and my understanding is that it's moving away from trying to enable bimodal access via proxy-server middleware toward having a local FUSE-based client that just happens to use swift as a storage backend00:12
timburke_no idea about the state of keystone support there; my assumption is that no one has tried it, but even that i don't know for certain00:13
andybottingA FUSE model sounds perfect for us00:16
timburke_fwiw, other things you might look into are https://github.com/kahing/goofys, https://github.com/dask/s3fs/, or https://rclone.org/commands/rclone_mount/ -- though i don't have much of any experience with any of them00:16
andybottingI remember seeing OVH had something they wrote that looked similar too, but they say it's not being developed any more00:17
andybottingWhat we would really like to do is create a keystone application credential (on behalf of the user) and inject that into their VM and automatically mount their swift storage00:18
andybottingI'm not sure if the S3 interface would be practical for that workflow, but I haven't looked much into it ether00:19
timburke_andybotting, out of curiosity, what all requirements do you have? will the volume need to be r/w, or could it be read-only? will multiple clients need to be able to mount the same volume, or will it be a 1:1 mapping?00:19
andybottingWe need r/w but expect that it would be written to from one source at a time00:20
andybottingIts for a virtual desktop service. Users will have one desktop at a time, but would mount swift as their way of getting data in/out00:21
andybottingThen they could use that same storage for other services too00:21
andybottingBut the I expect the usage of it would be more for shuffling data in and out, rather than doing processing on directly00:22
timburke_makes sense; have local scratch space and use swift for inputs/outputs00:23
andybottingthat's it00:23
andybottinggoofys looks pretty neat00:24
timburke_rclone mount might be worth checking out, too, if you'd rather not add in the s3api complexity00:25
timburke_it should have native swift support00:25
andybottingyeah, just looking now00:26
timburke_for all these things, i expect the vfs cache tuning is going to be key in determining whether you can get acceptable performance00:27
andybottingYeah. I think we'll need to make sure the users have a realistic expectation of what this storage will be for00:27
andybottings3api might be ok, if I (as an admin) can generate a credential on the user's behalf00:29
andybottingwhich it looks like I can :)00:30
andybottingI'll have a play with these and see how I go. Thanks timburke_00:33
timburke_andybotting, no prob, happy to help! good luck!00:34
mor3snot sure how it happened but now i have messages on ring.gz being obsolete "Ring file account.ring.gz is obsolete01:48
mor3s" .. how do i resolve this?01:48
kotamor3s: it seems like your builder file have new changes that is not dumped to the ring file yet04:51
kotamor3s: you can see your ring balance via `swift-ring-builder <builder file>` and can check the balance is now as you expected.05:00
mor3sthe rebalance was done long time ago05:03
kotausually, `swift-ring-builder <builder file> rebalance` will create the newer balanced ring.gz file, then the message should be changed to "Ring file account.ring.gz is up-to-date"05:03
kotamor3s: https://gist.github.com/bloodeagle40234/9c44f3a7bab47a2327364ccf6577f010 <- is an example for you to get swift-ring-file builder behaviro05:08
mor3sdoes it sounds like someone made changes to the builder files (eg add devices) and forgot to run the rebalance command?05:10
kotait's... likely that builder file is staging state, then ring file is the committed one. the way to commit from the builder file to the ring is either `rebalance` command or `write_ring` IIRC. 05:10
mor3scan we see the content of gz files?05:11
kotaah... the way I remember is using swift's Ring class but... it's not from command line interface...05:12
kotalet me look for the way... 1 sec05:12
kotahmm... not found05:14
kotaone way to recover the builder file from the ring file (i.e. convert ring to builder) but it should not ensure the complete builder file IIRC.05:15
kotamor3s: I'm not sure, it's that you are looking for but https://gist.github.com/bloodeagle40234/0670f4ff390ee1049efb2bf021816bac is one way to extract ring file data to meaningful instance.05:18
mor3slet me take a look at it... thanks kota05:19
kotabut usually it's an instance with iterator of nodes and partition tables so it may be hard to get difference with your builder file without deeper swift knowledge05:19
timburke__might try something like https://gist.github.com/matthewoliver/ce5172dbf1bcf8143184686517ad67ca -- see mattoliver's comment a few years ago: https://meetings.opendev.org/irclogs/%23openstack-swift/%23openstack-swift.2018-01-19.log.html#t2018-01-19T03:55:4114:44
timburke__mor3s, ^^^14:44
opendevreviewAndre Aranha proposed openstack/swift master: WIP/DNM: Add FIPS CI jobs  https://review.opendev.org/c/openstack/swift/+/79605717:02
opendevreviewAndre Aranha proposed openstack/swift master: WIP/DNM: Add FIPS CI jobs  https://review.opendev.org/c/openstack/swift/+/79605717:06
opendevreviewAndre Aranha proposed openstack/swift master: WIP/DNM: Add FIPS CI jobs  https://review.opendev.org/c/openstack/swift/+/79605718:03
opendevreviewAndre Aranha proposed openstack/swift master: WIP/DNM: Add FIPS CI jobs  https://review.opendev.org/c/openstack/swift/+/79605718:04
reid_gAnybody good at integrating keystone auth w/swift? I am able to generate my token with `swift auth` but any other swift commands are failing with 401.18:16
DHEI'm doing it. It's pretty easy if you have the URL and auth token... curl -H "X-Auth-Token: $OS_AUTH_TOKEN" $OS_STORAGE_URL/$containername/$objectname" -o saved-object-name.ext18:33
DHEthat would save the indicated $objectname from $container to a local file18:33
timburke_reid_g, sounds like an issue in the [filter:keystoneauth] section in proxy-server.conf -- what keystone roles does the user have for the project? what roles are listed for operator_roles and reseller_admin_role? does the reseller_prefix match how you set up the storage endpoint in keystone?18:36
reid_gIn Keystone I created a role called osg_user and my user is assigned to that. In my swift-proxy.conf I have operator_roles=admin, osg_user. I did not specify a reseller_admin_role but the resller_prefix=AUTH_18:43
reid_gMy OS_STORAGE_URL=http://swiftproxy:8080/v1/AUTH_eead7736f5374734b99b9294d06abcfb18:44
DHE[filter:keystoneauth] use = egg:swift#keystoneauth   operator_roles = admin, swiftoperator19:52
DHEthis is what i have. users with swift access have the swiftoperator role, so I can split it off from virtual machine stuff in openstack19:52
reid_gmaybe dumb question. Using puppet-swift and it calls it keystone in pipline and creates a section called [filter:keystone] use=egg:swift#keystoneauth in swift-proxy.conf. Does that wording matter?20:11
DHEI don't think so. it has the correct use= value which should be the important part20:20
timburke_reid_g, huh. proxy conf seems right then -- might try turning up the log level, see what extra info you can get out of it. sorry, it's been a bit since i last set up swift+keystone, so i don't entirely remember20:38
timburke_the different filter name is definitely fine, though -- important bits are having the name in the pipeline and the name in the [filter:...] section match, and having the use line right20:39
timburke_sanity check -- what's the pipeline order? auth_token is left of keystone, yeah?20:40
reid_gYes "authtoken s3api s3token keystone"20:41

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!