blob: 927ad74f5a8325f85ff57014af86293a48988aa4 [file] [log] [blame]
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>Overview on Ozone</title>
<link>/</link>
<description>Recent content in Overview on Ozone</description>
<generator>Hugo -- gohugo.io</generator>
<language>en-us</language><atom:link href="/index.xml" rel="self" type="application/rss+xml" />
<item>
<title>Overview</title>
<link>/concept/overview.html</link>
<pubDate>Tue, 10 Oct 2017 00:00:00 +0000</pubDate>
<guid>/concept/overview.html</guid>
<description>Ozone&amp;rsquo;s overview and components that make up Ozone.</description>
</item>
<item>
<title>Ofs (Hadoop compatible)</title>
<link>/interface/ofs.html</link>
<pubDate>Thu, 14 Sep 2017 00:00:00 +0000</pubDate>
<guid>/interface/ofs.html</guid>
<description>Hadoop Compatible file system allows any application that expects an HDFS like interface to work against Ozone with zero changes. Frameworks like Apache Spark, YARN and Hive work against Ozone without needing any change. &lt;strong&gt;Global level view.&lt;/strong&gt;</description>
</item>
<item>
<title>High Availability</title>
<link>/feature/ha.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/feature/ha.html</guid>
<description>HA setup for Ozone to avoid any single point of failure.</description>
</item>
<item>
<title>Securing Ozone</title>
<link>/security/secureozone.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/security/secureozone.html</guid>
<description>Overview of Ozone security concepts and steps to secure Ozone Manager and SCM.</description>
</item>
<item>
<title>O3fs (Hadoop compatible)</title>
<link>/interface/o3fs.html</link>
<pubDate>Thu, 14 Sep 2017 00:00:00 +0000</pubDate>
<guid>/interface/o3fs.html</guid>
<description>Hadoop Compatible file system allows any application that expects an HDFS like interface to work against Ozone with zero changes. Frameworks like Apache Spark, YARN and Hive work against Ozone without needing any change. &lt;strong&gt;Bucket level view.&lt;/strong&gt;</description>
</item>
<item>
<title>Ozone Manager</title>
<link>/concept/ozonemanager.html</link>
<pubDate>Thu, 14 Sep 2017 00:00:00 +0000</pubDate>
<guid>/concept/ozonemanager.html</guid>
<description>Ozone Manager is the principal name space service of Ozone. OM manages the life cycle of volumes, buckets and Keys.</description>
</item>
<item>
<title>Topology awareness</title>
<link>/feature/topology.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/feature/topology.html</guid>
<description>Configuration for rack-awarness for improved read/write</description>
</item>
<item>
<title>Transparent Data Encryption</title>
<link>/security/securingtde.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/security/securingtde.html</guid>
<description>TDE allows data on the disks to be encrypted-at-rest and automatically decrypted during access.</description>
</item>
<item>
<title>Storage Container Manager</title>
<link>/concept/storagecontainermanager.html</link>
<pubDate>Thu, 14 Sep 2017 00:00:00 +0000</pubDate>
<guid>/concept/storagecontainermanager.html</guid>
<description>Storage Container Manager or SCM is the core metadata service of Ozone. SCM provides a distributed block layer for Ozone.</description>
</item>
<item>
<title>GDPR in Ozone</title>
<link>/security/gdpr.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/security/gdpr.html</guid>
<description>Support to implement the &amp;ldquo;Right to be Forgotten&amp;rdquo; requirement of GDPR</description>
</item>
<item>
<title>S3 Protocol</title>
<link>/interface/s3.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/interface/s3.html</guid>
<description>Ozone supports Amazon&amp;rsquo;s Simple Storage Service (S3) protocol. In fact, You can use S3 clients and S3 SDK based applications without any modifications with Ozone.</description>
</item>
<item>
<title>Securing Datanodes</title>
<link>/security/securingdatanodes.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/security/securingdatanodes.html</guid>
<description>Explains different modes of securing data nodes. These range from kerberos to auto approval.</description>
</item>
<item>
<title>Command Line Interface</title>
<link>/interface/cli.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/interface/cli.html</guid>
<description>Ozone shell is the primary interface to interact with Ozone from the command line. Behind the scenes it uses the Java API.
There are some functionality which couldn&amp;rsquo;t be accessed without using ozone sh commands. For example:
Creating volumes with quota Managing internal ACLs Creating buckets with encryption key All of these are one-time, administration tasks. Applications can use Ozone without this CLI using other interface like Hadoop Compatible File System (o3fs or ofs) or S3 interface.</description>
</item>
<item>
<title>Quota in Ozone</title>
<link>/feature/quota.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/feature/quota.html</guid>
<description>Introduction to Ozone Quota</description>
</item>
<item>
<title>Recon API</title>
<link>/interface/reconapi.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/interface/reconapi.html</guid>
<description>Recon server supports HTTP endpoints to help troubleshoot and monitor Ozone cluster.</description>
</item>
<item>
<title>Securing HTTP</title>
<link>/security/securingozonehttp.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/security/securingozonehttp.html</guid>
<description>Secure HTTP web-consoles for Ozone services</description>
</item>
<item>
<title>Java API</title>
<link>/interface/javaapi.html</link>
<pubDate>Thu, 14 Sep 2017 00:00:00 +0000</pubDate>
<guid>/interface/javaapi.html</guid>
<description>Ozone has a set of Native RPC based APIs. This is the lowest level API&amp;rsquo;s on which all other protocols are built. This is the most performant and feature-full of all Ozone protocols.</description>
</item>
<item>
<title>Containers</title>
<link>/concept/containers.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/concept/containers.html</guid>
<description>Description of the Containers, the replication unit of Ozone.</description>
</item>
<item>
<title>Securing S3</title>
<link>/security/securings3.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/security/securings3.html</guid>
<description>Ozone supports S3 protocol, and uses AWS Signature Version 4 protocol which allows a seamless S3 experience.</description>
</item>
<item>
<title>CSI Protocol</title>
<link>/interface/csi.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/interface/csi.html</guid>
<description>Ozone supports Container Storage Interface(CSI) protocol. You can use Ozone by mounting an Ozone volume by Ozone CSI.</description>
</item>
<item>
<title>Ozone ACLs</title>
<link>/security/securityacls.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/security/securityacls.html</guid>
<description>Native Ozone Authorizer provides Access Control List (ACL) support for Ozone without Ranger integration.</description>
</item>
<item>
<title>Datanodes</title>
<link>/concept/datanodes.html</link>
<pubDate>Thu, 14 Sep 2017 00:00:00 +0000</pubDate>
<guid>/concept/datanodes.html</guid>
<description>Ozone supports Amazon&amp;rsquo;s Simple Storage Service (S3) protocol. In fact, You can use S3 clients and S3 SDK based applications without any modifications with Ozone.</description>
</item>
<item>
<title>Apache Ranger</title>
<link>/security/securitywithranger.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/security/securitywithranger.html</guid>
<description>Apache Ranger is a framework to enable, monitor and manage comprehensive data security across the Hadoop platform.</description>
</item>
<item>
<title>Recon Server</title>
<link>/feature/recon.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/feature/recon.html</guid>
<description>Recon is the Web UI and analysis service for Ozone</description>
</item>
<item>
<title>Recon</title>
<link>/concept/recon.html</link>
<pubDate>Tue, 27 Oct 2020 00:00:00 +0000</pubDate>
<guid>/concept/recon.html</guid>
<description>Recon serves as a management and monitoring console for Ozone.</description>
</item>
<item>
<title>Observability</title>
<link>/feature/observability.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/feature/observability.html</guid>
<description>Different tools for Ozone to increase Observability</description>
</item>
<item>
<title>Simple Single Ozone</title>
<link>/start/startfromdockerhub.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/start/startfromdockerhub.html</guid>
<description>Requirements Working docker setup AWS CLI (optional) Ozone in a Single Container The easiest way to start up an all-in-one ozone container is to use the latest docker image from docker hub:
docker run -p 9878:9878 -p 9876:9876 apache/ozone This command will pull down the ozone image from docker hub and start all ozone services in a single container. This container will run the required metadata servers (Ozone Manager, Storage Container Manager) one data node and the S3 compatible REST server (S3 Gateway).</description>
</item>
<item>
<title>Ozone On Premise Installation</title>
<link>/start/onprem.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/start/onprem.html</guid>
<description>If you are feeling adventurous, you can setup ozone in a real cluster. Setting up a real cluster requires us to understand the components of Ozone. Ozone is designed to work concurrently with HDFS. However, Ozone is also capable of running independently. The components of ozone are the same in both approaches.
Ozone Components Ozone Manager - Is the server that is in charge of the namespace of Ozone. Ozone Manager is responsible for all volume, bucket and key operations.</description>
</item>
<item>
<title>Minikube &amp; Ozone</title>
<link>/start/minikube.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/start/minikube.html</guid>
<description>Requirements Working minikube setup kubectl kubernetes/examples folder of the ozone distribution contains kubernetes deployment resource files for multiple use cases. By default the kubernetes resource files are configured to use apache/ozone image from the dockerhub.
To deploy it to minikube use the minikube configuration set:
cd kubernetes/examples/minikube kubectl apply -f . And you can check the results with
kubectl get pod Note: the kubernetes/examples/minikube resource set is optimized for minikube usage:</description>
</item>
<item>
<title>Ozone on Kubernetes</title>
<link>/start/kubernetes.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/start/kubernetes.html</guid>
<description>Requirements Working kubernetes cluster (LoadBalancer, PersistentVolume are not required) kubectl As the apache/ozone docker images are available from the dockerhub the deployment process is very similar to Minikube deployment. The only big difference is that we have dedicated set of k8s files for hosted clusters (for example we can use one datanode per host) Deploy to kubernetes
kubernetes/examples folder of the ozone distribution contains kubernetes deployment resource files for multiple use cases.</description>
</item>
<item>
<title>Pseudo-cluster</title>
<link>/start/runningviadocker.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/start/runningviadocker.html</guid>
<description>Requirements docker and docker-compose Download the Ozone binary tarball and untar it.
Go to the directory where the docker compose files exist and tell docker-compose to start Ozone in the background. This will start a small ozone instance on your machine.
cd compose/ozone/ docker-compose up -d To verify that ozone is working as expected, let us log into a data node and run freon, the load generator for Ozone.</description>
</item>
<item>
<title>From Source</title>
<link>/start/fromsource.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/start/fromsource.html</guid>
<description>Requirements Java 1.8 Maven This is a guide on how to build the ozone sources. If you are not planning to build sources yourself, you can safely skip this page.
If you are a Hadoop ninja, and wise in the ways of Apache, you already know that a real Apache release is a source release.
If you want to build from sources, Please untar the source tarball (or clone the latest code from the git repository) and run the ozone build command.</description>
</item>
<item>
<title>Proposed persistent OM connection for S3 gateway</title>
<link>/design/s3-performance.html</link>
<pubDate>Mon, 09 Nov 2020 00:00:00 +0000</pubDate>
<guid>/design/s3-performance.html</guid>
<description>Proposal to use per-request authentication and persistent connections between S3g and OM</description>
</item>
<item>
<title>Erasure Coding in Ozone</title>
<link>/design/ec.html</link>
<pubDate>Tue, 30 Jun 2020 00:00:00 +0000</pubDate>
<guid>/design/ec.html</guid>
<description>Use Erasure Coding algorithm for efficient storage</description>
</item>
<item>
<title>Implement new Ozone FileSystem scheme ofs://</title>
<link>/design/ofs.html</link>
<pubDate>Tue, 30 Jun 2020 00:00:00 +0000</pubDate>
<guid>/design/ofs.html</guid>
<description>A new schema structure for Hadoop compatible file system</description>
</item>
<item>
<title>Storage Class</title>
<link>/design/storage-class.html</link>
<pubDate>Mon, 08 Jun 2020 00:00:00 +0000</pubDate>
<guid>/design/storage-class.html</guid>
<description>New abstraction to configure replication methods.</description>
</item>
<item>
<title>Ozone Volume Management</title>
<link>/design/volume-management.html</link>
<pubDate>Thu, 02 Apr 2020 00:00:00 +0000</pubDate>
<guid>/design/volume-management.html</guid>
<description>A simplified version of mapping between S3 buckets and Ozone volume/buckets</description>
</item>
<item>
<title>NFS support Ozone</title>
<link>/design/nfs.html</link>
<pubDate>Wed, 25 Mar 2020 00:00:00 +0000</pubDate>
<guid>/design/nfs.html</guid>
<description>Adopt NFS gateway of HDFS and provide NFS file system view</description>
</item>
<item>
<title>Ozone Admin</title>
<link>/tools/admin.html</link>
<pubDate>Wed, 25 Mar 2020 00:00:00 +0000</pubDate>
<guid>/tools/admin.html</guid>
<description>Ozone Admin command can be used for all the admin related tasks.</description>
</item>
<item>
<title>SCM HA support</title>
<link>/design/scmha.html</link>
<pubDate>Thu, 05 Mar 2020 00:00:00 +0000</pubDate>
<guid>/design/scmha.html</guid>
<description>HA for Storage Container Manager using Ratis to replicate data</description>
</item>
<item>
<title>Ozone FS namespace</title>
<link>/design/namespace-support.html</link>
<pubDate>Mon, 20 Jan 2020 00:00:00 +0000</pubDate>
<guid>/design/namespace-support.html</guid>
<description>Use additional prefix table for indexed data retrieval</description>
</item>
<item>
<title>Generic Extensible Token support for Ozone</title>
<link>/design/token.html</link>
<pubDate>Thu, 09 Jan 2020 00:00:00 +0000</pubDate>
<guid>/design/token.html</guid>
<description>Extend existing token architecture to all entities of Ozone</description>
</item>
<item>
<title>Ozone Trash Feature</title>
<link>/design/trash.html</link>
<pubDate>Thu, 07 Nov 2019 00:00:00 +0000</pubDate>
<guid>/design/trash.html</guid>
<description>Feature provides a user with the ability to recover keys that may have been deleted accidentally. (similar to the HDFS trash feature).</description>
</item>
<item>
<title>Recon server v2</title>
<link>/design/recon2.html</link>
<pubDate>Mon, 19 Aug 2019 00:00:00 +0000</pubDate>
<guid>/design/recon2.html</guid>
<description>Second phase of Recon development. Recon acts as a passive SCM.</description>
</item>
<item>
<title>Decommissioning in Ozone</title>
<link>/design/decommissioning.html</link>
<pubDate>Wed, 31 Jul 2019 00:00:00 +0000</pubDate>
<guid>/design/decommissioning.html</guid>
<description>Formal process to shut down machines in a safe way after the required replications.</description>
</item>
<item>
<title>Ozone GDPR framework</title>
<link>/design/gdpr.html</link>
<pubDate>Thu, 20 Jun 2019 00:00:00 +0000</pubDate>
<guid>/design/gdpr.html</guid>
<description>Crypto key management to handle GDPR &amp;ldquo;right to be forgotten&amp;rdquo; feature</description>
</item>
<item>
<title>Ozone locks in OM</title>
<link>/design/locks.html</link>
<pubDate>Thu, 20 Jun 2019 00:00:00 +0000</pubDate>
<guid>/design/locks.html</guid>
<description>Hierarchical locking structure for OM elements (volumes/keys/buckets)</description>
</item>
<item>
<title>Ozone Enhancement Proposals</title>
<link>/design/ozone-enhancement-proposals.html</link>
<pubDate>Fri, 07 Jun 2019 00:00:00 +0000</pubDate>
<guid>/design/ozone-enhancement-proposals.html</guid>
<description>Definition of the process to share new technical proposals with the Ozone community.</description>
</item>
<item>
<title>Configless Ozone service management</title>
<link>/design/configless.html</link>
<pubDate>Sat, 25 May 2019 00:00:00 +0000</pubDate>
<guid>/design/configless.html</guid>
<description>Distribute only minimal configuration and download all the remaining before start</description>
</item>
<item>
<title>Ozone multi-raft support</title>
<link>/design/multiraft.html</link>
<pubDate>Tue, 21 May 2019 00:00:00 +0000</pubDate>
<guid>/design/multiraft.html</guid>
<description>Datanodes can be part of multiple independent RAFT groups / pipelines</description>
</item>
<item>
<title>Type-safe configuration API</title>
<link>/design/typesafeconfig.html</link>
<pubDate>Thu, 25 Apr 2019 00:00:00 +0000</pubDate>
<guid>/design/typesafeconfig.html</guid>
<description>Inject configuration values based on annotations instead of using constants and Hadoop API</description>
</item>
<item>
<title>Recon server 1</title>
<link>/design/recon1.html</link>
<pubDate>Tue, 19 Feb 2019 00:00:00 +0000</pubDate>
<guid>/design/recon1.html</guid>
<description>Second phase of Recon development. Recon acts as a passive SCM.</description>
</item>
<item>
<title>Ozone Encryption At Rest</title>
<link>/design/tde.html</link>
<pubDate>Fri, 08 Feb 2019 00:00:00 +0000</pubDate>
<guid>/design/tde.html</guid>
<description>Transparent Data Encryption encrypts the data before store it</description>
</item>
<item>
<title>Generate Configurations</title>
<link>/tools/genconf.html</link>
<pubDate>Tue, 18 Dec 2018 00:00:00 +0000</pubDate>
<guid>/tools/genconf.html</guid>
<description>Tool to generate default configuration</description>
</item>
<item>
<title>Audit Parser</title>
<link>/tools/auditparser.html</link>
<pubDate>Mon, 17 Dec 2018 00:00:00 +0000</pubDate>
<guid>/tools/auditparser.html</guid>
<description>Audit Parser tool can be used for querying the ozone audit logs.</description>
</item>
<item>
<title>Topology-awareness</title>
<link>/design/topology.html</link>
<pubDate>Fri, 16 Nov 2018 00:00:00 +0000</pubDate>
<guid>/design/topology.html</guid>
<description>Placement policy to use rack information for read and write</description>
</item>
<item>
<title>S3 protocol support for Ozone</title>
<link>/design/s3gateway.html</link>
<pubDate>Thu, 27 Sep 2018 00:00:00 +0000</pubDate>
<guid>/design/s3gateway.html</guid>
<description>Support any AWS S3 compatible client with dedicated REST endpoint</description>
</item>
<item>
<title>Ozone Manager HA</title>
<link>/design/omha.html</link>
<pubDate>Tue, 18 Sep 2018 00:00:00 +0000</pubDate>
<guid>/design/omha.html</guid>
<description>Support HA for Ozone Manager with the help of RATIS</description>
</item>
<item>
<title>Access Ozone object store with Amazon Boto3 client</title>
<link>/recipe/botoclient.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/recipe/botoclient.html</guid>
<description>How to access Ozone object store with Boto3 client?</description>
</item>
<item>
<title>Monitoring with Prometheus</title>
<link>/recipe/prometheus.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/recipe/prometheus.html</guid>
<description>A Simple recipe to monitor Ozone using Prometheus</description>
</item>
<item>
<title>Spark in Kubernetes with OzoneFS</title>
<link>/recipe/sparkozonefsk8s.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/recipe/sparkozonefsk8s.html</guid>
<description>How to use Apache Spark with Ozone on K8s?</description>
</item>
<item>
<title>Testing tools</title>
<link>/tools/testtools.html</link>
<pubDate>Mon, 01 Jan 0001 00:00:00 +0000</pubDate>
<guid>/tools/testtools.html</guid>
<description>Ozone contains multiple test tools for load generation, partitioning test or acceptance tests.</description>
</item>
</channel>
</rss>