tree 83a16334621bda4d609f78f544afb4a5acec9e0d
parent 69b0c6a47bb96d4e2955a8a651af34bcf81376e0
author Abhishek Agarwal <1477457+abhishekagarwal87@users.noreply.github.com> 1626806659 +0530
committer GitHub <noreply@github.com> 1626806659 +0530
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsBcBAABCAAQBQJg9xmDCRBK7hj4Ov3rIwAAI0MIAIi7EC62rgG8qX1uyBGUhHao
 BCOBdovIQLTPPTw3bsLpGt/QJZ9PN/xQEqFnUfNkhTPAlKZdmAEEOpQOx8JHFkTG
 STMJX6G1RC79EC2lXp0WkoFIkNH5E3CXiawKDFre07b9s9xINabFiptNSedT07rs
 OHmzXacCpTTwBRlcCdjo5o25H4lNrMbc59698loAvSe4mwePvWNS9rLpBYOdxqpb
 kOm5hoP02cFb6DT4A5yUnZqVziZdQOM+5g4R8ro2C8noN0KrgG1XR/ZFT3MjA8fX
 JoJrPl4wb5MWysWI5vsIgHaQ4W+NiQZQLSgL4hOTds8gVYasJ0m3Lx4R3oIlh2Q=
 =/MHA
 -----END PGP SIGNATURE-----
 

Split SegmentLoader into SegmentLoader and SegmentCacheManager (#11466)

This PR splits current SegmentLoader into SegmentLoader and SegmentCacheManager.

SegmentLoader - this class is responsible for building the segment object but does not expose any methods for downloading, cache space management, etc. Default implementation delegates the download operations to SegmentCacheManager and only contains the logic for building segments once downloaded. . This class will be used in SegmentManager to construct Segment objects.

SegmentCacheManager - this class manages the segment cache on the local disk. It fetches the segment files to the local disk, can clean up the cache, and in the future, support reserve and release on cache space. [See https://github.com/Make SegmentLoader extensible and customizable #11398]. This class will be used in ingestion tasks such as compaction, re-indexing where segment files need to be downloaded locally.