tree cfeac4996d57023659b6016516ba079b18ef1504
parent 95e68926750b55196cf9da53c25359c98ef58a4f
author Steve Loughran <stevel@cloudera.com> 1616592774 +0000
committer GitHub <noreply@github.com> 1616592774 +0000
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsBcBAABCAAQBQJgWz+GCRBK7hj4Ov3rIwAAdHIIAIwnZBZ2zsfV4h1vHP6hxzjv
 1118ceIBy3Bcm2xQO7RSWGgSBxbWjNHYOkQrETBvGEPPDLOeyE90mrQKIcKoMLbu
 s8sVSE5zWQxUa8Ot9vztn2r21GHtI3HALUcYANhjqap7NGaIeTed/BndIw+JF//A
 wyde1010hAkg5EtaozAsdXR6d0pIUDh9iK6lcrtHjhBlBFOfJfdj1zscdH9ZYOqF
 BWmdJ+ydteZkLHAtBXi9fCGOvAinj25ifSHbq8HeRR25zKrgsKSuLJaTWTkil8e6
 sQDkohUd7z/UvwEhFU17lit7iRh0JMFMyjLKqrH4nzuotNd37gNCCC0Ll4crwhM=
 =jh9C
 -----END PGP SIGNATURE-----
 

HADOOP-13551. AWS metrics wire-up (#2778)


Moves to the builder API for AWS S3 client creation, and
offers a similar style of API to the S3A FileSystem and tests, hiding
the details of which options are client, which are in AWS Conf,
and doing the wiring up of S3A statistics interfaces to the AWS
SDK internals. S3A Statistics, including IOStatistics, should now
count throttling events handled in the AWS SDK itself.

This patch restores endpoint determination by probes to US-East-1
if the client isn't configured with fs.s3a.endpoint.

Explicitly setting the endpoint will save the cost of these probe
HTTP requests.

Contributed by Steve Loughran.