5db0f34e29c68557842b9892bc83e93f7ef58307 - hadoop

commit	5db0f34e29c68557842b9892bc83e93f7ef58307	[log] [tgz]
author	Mukund Thakur <mthakur@cloudera.com>	Sat Apr 30 04:17:33 2022 +0530
committer	Steve Loughran <stevel@cloudera.com>	Wed Jun 22 17:29:32 2022 +0100
tree	56eb3ada1d256f18d2c3f9d0ec72992050bca09e
parent	2daf0a814f800c260251b2cd4bf0ece3642c3aa8 [diff]

HADOOP-18104: S3A: Add configs to configure minSeekForVectorReads and maxReadSizeForVectorReads (#3964)

Part of HADOOP-18103.
Introducing fs.s3a.vectored.read.min.seek.size and fs.s3a.vectored.read.max.merged.size
to configure min seek and max read during a vectored IO operation in S3A connector.
These properties actually define how the ranges will be merged. To completely
disable merging set fs.s3a.max.readsize.vectored.read to 0.

Contributed By: Mukund Thakur

hadoop-common-project/hadoop-common/src/site/markdown/filesystem/fsdatainputstream.md[diff]
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/contract/AbstractContractVectoredReadTest.java[diff]
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/impl/TestVectoredReadUtils.java[diff]
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/MoreAsserts.java[diff]
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/Constants.java[diff]
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java[diff]
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AInputStream.java[diff]
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AReadOpContext.java[diff]
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/VectoredIOContext.java[Added - diff]
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/performance.md[diff]
hadoop-tools/hadoop-aws/src/test/java/org/apache/hadoop/fs/contract/s3a/ITestS3AContractVectoredRead.java[diff]

11 files changed