[fix](s3-client) Fix s3-client max-connection default value which cause s3 to be slow and unable to handle high concurrency in many-files scenarios. (#58511) (#58626)
### What problem does this PR solve?
Related PR: #51825, #58514
Problem Summary:
### Release note
Cherry-pick #58511
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->diff --git a/be/src/util/s3_util.cpp b/be/src/util/s3_util.cpp
index a0e87fe..bd0629a 100644
--- a/be/src/util/s3_util.cpp
+++ b/be/src/util/s3_util.cpp
@@ -369,8 +369,7 @@
if (s3_conf.max_connections > 0) {
aws_config.maxConnections = s3_conf.max_connections;
} else {
- // AWS SDK max concurrent tcp connections for a single http client to use. Default 25.
- aws_config.maxConnections = std::max(config::doris_scanner_thread_pool_thread_num, 25);
+ aws_config.maxConnections = 102400;
}
aws_config.requestTimeoutMs = 30000;
