What problems is IBM Storage Ceph solving and how is that benefiting you?
We were creating SDS as a product based on CEPH. The main idea was to share multiple iSCSI targets via multiple paths. It was a production-ready solution with real customers, as a result we made hundreds of CEPH clusters (0.87-12.2) and collected an immense amount of experience in that. Let me share some examples:
* If you have a time difference of more than 30m, a CEPH cluster will get stuck.
* If you change time back (1 ms is enough), a CEPH cluster will crash.
* Nobody checks the crushmap instead of you. You have to understand what you are doing & how it is affecting a cluster.
* You can face tons of problems if you use a non-standard cluster name. i.e., after mounting, images are presented in the system using image name & pool name, it might be the same in another cluster, you will have a problem in that case.
* Before 12.2, the CEPH S3 implementation was extremely immature.
* OSDs start via udev trigger, the interesting thing is that you might face a situation where there is no network at this time.
* Import-diff allows you to import a delta from another image. As a result, you can lose data.
* It does not have an optimal recovery mechanism, as a result, you face sticking on client systems during recovery. Review collected by and hosted on G2.com.