radosstriper

[aep]

would it make sense to include the radosstriper api?
i'm a bit confused about the multiple incompatible ways ceph stores things, so i'm hoping at least rados_stripe is compatible with rgw?

[phlogistonjohn]

Could you link to an example of this API please? Either code or docs would be suitable.

Also, I'm not clear what you mean by "multiple incompatible ways". Do you mean like rbd vs. cephfs vs. rgw?

[aep]

there's no docs, so i have no idea how it works either. I spotted it when trying to figure out how to access rgw objects without going through the AWS api, which requires a SeekReader.

https://github.com/ceph/ceph/blob/master/src/include/radosstriper/libradosstriper.h

Do you mean like rbd vs. cephfs vs. rgw?

yeah, it's confusing why they're all different implementations on top of rados objects. I'm not sure if radosstriper uses the same object format as rbd or rgw.

[phlogistonjohn]

there's no docs, so i have no idea how it works either. I spotted it when trying to figure out how to access rgw objects without going through the AWS api, which requires a SeekReader.

https://github.com/ceph/ceph/blob/master/src/include/radosstriper/libradosstriper.h

Thanks. It certainly appears that this is designed to help build striped objects on top of rados, but I don't see much that uses it. There are very few PRs I see that refer to it by name. @tchaikov might know a bit more about it as one of the last people to make changes to those files.

Do you mean like rbd vs. cephfs vs. rgw?

yeah, it's confusing why they're all different implementations on top of rados objects. I'm not sure if radosstriper uses the same object format as rbd or rgw.

They're all trying to meet the needs of different use cases while using rados as a common layer. If they didn't use rados each one would need to (re)invent it's own back end storage [1]. The cephfs subsystem implements posix file system semantics where rgw is an http object store. I've not been involved with ceph a very long time so I'm probably not the best to explain this though. If you're interesting on delving deeper into this topic I suggest emailing the ceph list.

I'm skeptical that the rgw layer uses radosstriper because very few PRs in ceph seem to reference it. ceph is a very active repo so I'd assume I'd see more if it was a component of a hot topic like rgw. But I only looked for a couple of minutes so take what I say here with a grain of salt.

[1] - There are both upsides and downsides to this I'm sure, but that's how ceph does it. :-)

[phlogistonjohn]

PR #1015 adds a rados/striper package. AFAIK the C radosstriper package is not used by either RGW, RBD, or CephFS but is similar to it. Either way you can now use (some) radosstriper APIs via go-ceph and therefore I think it's safe to close this issue.