Refactored JCloud Tiered Storage (#6335)
Motivation
In order to facilitate the support of additional JClouds-supported providers, we first needed to clean up the existing code, as there were a lot of if/then/else constructs throughout the code that were based on the assumption that we either supported AWS S3 or Google Cloud Storage. I didn't want to keep adding else if's to these code blocks for every new provider we add, so I decided to refactor the code to make it a bit cleaner
Modifications
in addition to being home for most of the aforementioned if/then/else blocks, the BlobStoreManagedLedgerOffloader class had multiple responsibilities in addition to providing an implementation for the LedgerOffloader interface. My goal was to simplify this class such that its only responsibility was to implement the LedgerOffloader interface.
The other major change was the addition of the JCloudBlobStoreProvider enum, which implements 3 interfaces that allow for it to handle the provider specific logic for things such as acquiring the credentials, validating the configuration, and creating a provider-specific instance of BlobStore.
Result
After this change, we will be able to easily add support for additional JClouds-supported providers by simply adding new elements to the JCloudBlobStoreProvider Enums since the other logic has been isolated and is not vendor specific.
See #2865 for more details
* Refactored JCloud Tiered Storage
* Refactored JCloud Tiered Storage
* Added missing import statements
* Refactored JCloud Tiered Storage
* Refactored JCloud Tiered Storage
* Added missing import statements
* Refactored JCloud Tiered Storage
* Refactored JCloud Tiered Storage
* Added missing import statements
* fix test
* add test logs
* fix logs
* fix test
* fix
* fix
* fix broker log
* fix configuration
* fix
* fix
* fix
* fix get BlobStore
* repair test presto query tiered storage data
* fix test
* fix test
* fix test TestPrestoQueryTieredStorage
* fix
* fix
Co-authored-by: Sijie Guo <sijie@apache.org>
Co-authored-by: gaoran10 <gaoran_10@126.com>
Co-authored-by: xiaolong.ran <rxl@apache.org>
4 files changed