% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/filesystem.R
\docType{class}
\name{FileSystem}
\alias{FileSystem}
\alias{LocalFileSystem}
\alias{S3FileSystem}
\alias{GcsFileSystem}
\alias{AzureFileSystem}
\alias{SubTreeFileSystem}
\title{FileSystem classes}
\description{
\code{FileSystem} is an abstract file system API,
\code{LocalFileSystem} is an implementation accessing files
on the local machine. \code{SubTreeFileSystem} is an implementation that delegates
to another implementation after prepending a fixed base path
}
\section{Factory}{


\code{LocalFileSystem$create()} returns the object and takes no arguments.

\code{SubTreeFileSystem$create()} takes the following arguments:
\itemize{
\item \code{base_path}, a string path
\item \code{base_fs}, a \code{FileSystem} object
}

\code{S3FileSystem$create()} optionally takes arguments:
\itemize{
\item \code{anonymous}: logical, default \code{FALSE}. If true, will not attempt to look up
credentials using standard AWS configuration methods.
\item \code{access_key}, \code{secret_key}: authentication credentials. If one is provided,
the other must be as well. If both are provided, they will override any
AWS configuration set at the environment level.
\item \code{session_token}: optional string for authentication along with
\code{access_key} and \code{secret_key}
\item \code{role_arn}: string AWS ARN of an AccessRole. If provided instead of \code{access_key} and
\code{secret_key}, temporary credentials will be fetched by assuming this role.
\item \code{session_name}: optional string identifier for the assumed role session.
\item \code{external_id}: optional unique string identifier that might be required
when you assume a role in another account.
\item \code{load_frequency}: integer, frequency (in seconds) with which temporary
credentials from an assumed role session will be refreshed. Default is
900 (i.e. 15 minutes)
\item \code{region}: AWS region to connect to. If omitted, the AWS library will
provide a sensible default based on client configuration, falling back
to "us-east-1" if no other alternatives are found.
\item \code{endpoint_override}: If non-empty, override region with a connect string
such as "localhost:9000". This is useful for connecting to file systems
that emulate S3.
\item \code{scheme}: S3 connection transport (default "https")
\item \code{proxy_options}: optional string, URI of a proxy to use when connecting
to S3
\item \code{background_writes}: logical, whether \code{OutputStream} writes will be issued
in the background, without blocking (default \code{TRUE})
\item \code{allow_bucket_creation}: logical, if TRUE, the filesystem will create
buckets if \verb{$CreateDir()} is called on the bucket level (default \code{FALSE}).
\item \code{allow_bucket_deletion}: logical, if TRUE, the filesystem will delete
buckets if\verb{$DeleteDir()} is called on the bucket level (default \code{FALSE}).
\item \code{check_directory_existence_before_creation}: logical, check if directory
already exists or not before creation. Helpful for cloud storage operations
where object mutation operations are rate limited or existing directories
are read-only. (default \code{FALSE}).
\item \code{request_timeout}: Socket read time on Windows and macOS in seconds. If
negative, the AWS SDK default (typically 3 seconds).
\item \code{connect_timeout}: Socket connection timeout in seconds. If negative, AWS
SDK default is used (typically 1 second).
}

\code{GcsFileSystem$create()} optionally takes arguments:
\itemize{
\item \code{anonymous}: logical, default \code{FALSE}. If true, will not attempt to look up
credentials using standard GCS configuration methods.
\item \code{access_token}: optional string for authentication. Should be provided along
with \code{expiration}
\item \code{expiration}: \code{POSIXct}. optional datetime representing point at which
\code{access_token} will expire.
\item \code{json_credentials}: optional string for authentication. Either a string
containing JSON credentials or a path to their location on the filesystem.
If a path to credentials is given, the file should be UTF-8 encoded.
\item \code{endpoint_override}: if non-empty, will connect to provided host name / port,
such as "localhost:9001", instead of default GCS ones. This is primarily useful
for testing purposes.
\item \code{scheme}: connection transport (default "https")
\item \code{default_bucket_location}: the default location (or "region") to create new
buckets in.
\item \code{retry_limit_seconds}: the maximum amount of time to spend retrying if
the filesystem encounters errors. Default is 15 seconds.
\item \code{default_metadata}: default metadata to write in new objects.
\item \code{project_id}: the project to use for creating buckets.
}

\code{AzureFileSystem$create()} takes following required argument:
\itemize{
\item \code{account_name}: Azure Blob Storage account name.
}

\code{AzureFileSystem$create()} takes following optional arguments:
\itemize{
\item \code{account_key}: Account key of the storage account. Cannot be used with
\code{sas_token}.
\item \code{blob_storage_authority}: Hostname of the blob service, defaulting to
\code{"blob.core.windows.net"}.
\item \code{blob_storage_scheme}: Either \code{"http"} or \code{"https"} (the default).
\item \code{client_id}: The client/application ID for Azure Active Directory
authentication. If used with \code{client_secret} and \code{tenant_id} then it is the
application ID for a registered Azure AD application. Otherwise, it is the
client ID of a user-assigned managed identity.
\item \code{client_secret}: Client secret for Azure Active Directory authentication.
Must be provided with both \code{client_id} and \code{tenant_id}.
\item \code{dfs_storage_authority}: Hostname of the data lake (gen 2) service,
defaulting to \code{"dfs.core.windows.net"}.
\item \code{dfs_storage_scheme}: Either \code{"http"} or \code{"https"} (the default).
\item \code{sas_token}: Shared access signature (SAS) token for the storage account.
Cannot be used with \verb{account key}.
\item \code{tenant_id}: Tenant ID for Azure Active Directory authentication. Must
be provided with both \code{client_id} and \code{client_secret}.
}
}

\section{Methods}{

\itemize{
\item \code{path(x)}: Create a \code{SubTreeFileSystem} from the current \code{FileSystem}
rooted at the specified path \code{x}.
\item \code{cd(x)}: Create a \code{SubTreeFileSystem} from the current \code{FileSystem}
rooted at the specified path \code{x}.
\item \code{ls(path, ...)}: List files or objects at the given path or from the root
of the \code{FileSystem} if \code{path} is not provided. Additional arguments passed
to \code{FileSelector$create}, see \link{FileSelector}.
\item \verb{$GetFileInfo(x)}: \code{x} may be a \link{FileSelector} or a character
vector of paths. Returns a list of \link{FileInfo}
\item \verb{$CreateDir(path, recursive = TRUE)}: Create a directory and subdirectories.
\item \verb{$DeleteDir(path)}: Delete a directory and its contents, recursively.
\item \verb{$DeleteDirContents(path)}: Delete a directory's contents, recursively.
Like \verb{$DeleteDir()},
but doesn't delete the directory itself. Passing an empty path (\code{""}) will
wipe the entire filesystem tree.
\item \verb{$DeleteFile(path)} : Delete a file.
\item \verb{$DeleteFiles(paths)} : Delete many files. The default implementation
issues individual delete operations in sequence.
\item \verb{$Move(src, dest)}: Move / rename a file or directory. If the destination
exists:
if it is a non-empty directory, an error is returned
otherwise, if it has the same type as the source, it is replaced
otherwise, behavior is unspecified (implementation-dependent).
\item \verb{$CopyFile(src, dest)}: Copy a file. If the destination exists and is a
directory, an error is returned. Otherwise, it is replaced.
\item \verb{$OpenInputStream(path)}: Open an \link[=InputStream]{input stream} for
sequential reading.
\item \verb{$OpenInputFile(path)}: Open an \link[=RandomAccessFile]{input file} for random
access reading.
\item \verb{$OpenOutputStream(path)}: Open an \link[=OutputStream]{output stream} for
sequential writing.
\item \verb{$OpenAppendStream(path)}: Open an \link[=OutputStream]{output stream} for
appending.
}
}

\section{Active bindings}{

\itemize{
\item \verb{$type_name}: string filesystem type name, such as "local", "s3", etc.
\item \verb{$region}: string AWS region, for \code{S3FileSystem} and \code{SubTreeFileSystem}
containing a \code{S3FileSystem}
\item \verb{$base_fs}: for \code{SubTreeFileSystem}, the \code{FileSystem} it contains
\item \verb{$base_path}: for \code{SubTreeFileSystem}, the path in \verb{$base_fs} which is considered
root in this \code{SubTreeFileSystem}.
\item \verb{$options}: for \code{GcsFileSystem}, the options used to create the
\code{GcsFileSystem} instance as a \code{list}
}
}

\section{Notes}{


On S3FileSystem, \verb{$CreateDir()} on a top-level directory creates a new bucket.
When S3FileSystem creates new buckets (assuming allow_bucket_creation is TRUE),
it does not pass any non-default settings. In AWS S3, the bucket and all
objects will be not publicly visible, and will have no bucket policies
and no resource tags. To have more control over how buckets are created,
use a different API to create them.

On S3FileSystem, output is only produced for fatal errors or when printing
return values. For troubleshooting, the log level can be set using the
environment variable \code{ARROW_S3_LOG_LEVEL} (e.g.,
\code{Sys.setenv("ARROW_S3_LOG_LEVEL"="DEBUG")}). The log level must be set prior
to running any code that interacts with S3. Possible values include 'FATAL'
(the default), 'ERROR', 'WARN', 'INFO', 'DEBUG' (recommended), 'TRACE', and
'OFF'.

On \code{AzureFileSystem}, passing no arguments for authentication uses the
\code{AzureDefaultCredential} for authentication, so that several authentication
types are tried until one succeeds.

\code{AzureFileSystem} is not presently supported on Windows due to upstream compatibility
issues between the Azure C++ SDK and the MinGW toolchain.
}