| <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> |
| <!-- |
| | Generated by Apache Maven Doxia at 2021-06-15 |
| | Rendered using Apache Maven Stylus Skin 1.5 |
| --> |
| <html xmlns="http://www.w3.org/1999/xhtml"> |
| <head> |
| <title>Apache Hadoop 3.3.1 – YARN Node Attributes</title> |
| <style type="text/css" media="all"> |
| @import url("./css/maven-base.css"); |
| @import url("./css/maven-theme.css"); |
| @import url("./css/site.css"); |
| </style> |
| <link rel="stylesheet" href="./css/print.css" type="text/css" media="print" /> |
| <meta name="Date-Revision-yyyymmdd" content="20210615" /> |
| <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> |
| </head> |
| <body class="composite"> |
| <div id="banner"> |
| <a href="http://hadoop.apache.org/" id="bannerLeft"> |
| <img src="http://hadoop.apache.org/images/hadoop-logo.jpg" alt="" /> |
| </a> |
| <a href="http://www.apache.org/" id="bannerRight"> |
| <img src="http://www.apache.org/images/asf_logo_wide.png" alt="" /> |
| </a> |
| <div class="clear"> |
| <hr/> |
| </div> |
| </div> |
| <div id="breadcrumbs"> |
| |
| <div class="xleft"> |
| <a href="http://www.apache.org/" class="externalLink">Apache</a> |
| > |
| <a href="http://hadoop.apache.org/" class="externalLink">Hadoop</a> |
| > |
| <a href="../index.html">Apache Hadoop YARN</a> |
| > |
| <a href="index.html">Apache Hadoop 3.3.1</a> |
| > |
| YARN Node Attributes |
| </div> |
| <div class="xright"> <a href="http://wiki.apache.org/hadoop" class="externalLink">Wiki</a> |
| | |
| <a href="https://gitbox.apache.org/repos/asf/hadoop.git" class="externalLink">git</a> |
| | |
| <a href="http://hadoop.apache.org/" class="externalLink">Apache Hadoop</a> |
| |
| | Last Published: 2021-06-15 |
| | Version: 3.3.1 |
| </div> |
| <div class="clear"> |
| <hr/> |
| </div> |
| </div> |
| <div id="leftColumn"> |
| <div id="navcolumn"> |
| |
| <h5>General</h5> |
| <ul> |
| <li class="none"> |
| <a href="../../index.html">Overview</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/SingleCluster.html">Single Node Setup</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/ClusterSetup.html">Cluster Setup</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/CommandsManual.html">Commands Reference</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/FileSystemShell.html">FileSystem Shell</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/Compatibility.html">Compatibility Specification</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/DownstreamDev.html">Downstream Developer's Guide</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/AdminCompatibilityGuide.html">Admin Compatibility Guide</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/InterfaceClassification.html">Interface Classification</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/filesystem/index.html">FileSystem Specification</a> |
| </li> |
| </ul> |
| <h5>Common</h5> |
| <ul> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/CLIMiniCluster.html">CLI Mini Cluster</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/FairCallQueue.html">Fair Call Queue</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/NativeLibraries.html">Native Libraries</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/Superusers.html">Proxy User</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/RackAwareness.html">Rack Awareness</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/SecureMode.html">Secure Mode</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/ServiceLevelAuth.html">Service Level Authorization</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/HttpAuthentication.html">HTTP Authentication</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/CredentialProviderAPI.html">Credential Provider API</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-kms/index.html">Hadoop KMS</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/Tracing.html">Tracing</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/UnixShellGuide.html">Unix Shell Guide</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/registry/index.html">Registry</a> |
| </li> |
| </ul> |
| <h5>HDFS</h5> |
| <ul> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HdfsDesign.html">Architecture</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HdfsUserGuide.html">User Guide</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HDFSCommands.html">Commands Reference</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html">NameNode HA With QJM</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html">NameNode HA With NFS</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/ObserverNameNode.html">Observer NameNode</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/Federation.html">Federation</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/ViewFs.html">ViewFs</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/ViewFsOverloadScheme.html">ViewFsOverloadScheme</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HdfsSnapshots.html">Snapshots</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HdfsEditsViewer.html">Edits Viewer</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HdfsImageViewer.html">Image Viewer</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html">Permissions and HDFS</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HdfsQuotaAdminGuide.html">Quotas and HDFS</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/LibHdfs.html">libhdfs (C API)</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/WebHDFS.html">WebHDFS (REST API)</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-hdfs-httpfs/index.html">HttpFS</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/ShortCircuitLocalReads.html">Short Circuit Local Reads</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/CentralizedCacheManagement.html">Centralized Cache Management</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HdfsNfsGateway.html">NFS Gateway</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HdfsRollingUpgrade.html">Rolling Upgrade</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/ExtendedAttributes.html">Extended Attributes</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/TransparentEncryption.html">Transparent Encryption</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html">Multihoming</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/ArchivalStorage.html">Storage Policies</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/MemoryStorage.html">Memory Storage Support</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/SLGUserGuide.html">Synthetic Load Generator</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HDFSErasureCoding.html">Erasure Coding</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HDFSDiskbalancer.html">Disk Balancer</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HdfsUpgradeDomain.html">Upgrade Domain</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HdfsDataNodeAdminGuide.html">DataNode Admin</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs-rbf/HDFSRouterFederation.html">Router Federation</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/HdfsProvidedStorage.html">Provided Storage</a> |
| </li> |
| </ul> |
| <h5>MapReduce</h5> |
| <ul> |
| <li class="none"> |
| <a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html">Tutorial</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapredCommands.html">Commands Reference</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduce_Compatibility_Hadoop1_Hadoop2.html">Compatibility with 1.x</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/EncryptedShuffle.html">Encrypted Shuffle</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/PluggableShuffleAndPluggableSort.html">Pluggable Shuffle/Sort</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistributedCacheDeploy.html">Distributed Cache Deploy</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/SharedCacheSupport.html">Support for YARN Shared Cache</a> |
| </li> |
| </ul> |
| <h5>MapReduce REST APIs</h5> |
| <ul> |
| <li class="none"> |
| <a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapredAppMasterRest.html">MR Application Master</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-hs/HistoryServerRest.html">MR History Server</a> |
| </li> |
| </ul> |
| <h5>YARN</h5> |
| <ul> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/YARN.html">Architecture</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/YarnCommands.html">Commands Reference</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html">Capacity Scheduler</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/FairScheduler.html">Fair Scheduler</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/ResourceManagerRestart.html">ResourceManager Restart</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html">ResourceManager HA</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/ResourceModel.html">Resource Model</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/NodeLabel.html">Node Labels</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/NodeAttributes.html">Node Attributes</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/WebApplicationProxy.html">Web Application Proxy</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/TimelineServer.html">Timeline Server</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/TimelineServiceV2.html">Timeline Service V.2</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html">Writing YARN Applications</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/YarnApplicationSecurity.html">YARN Application Security</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/NodeManager.html">NodeManager</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/DockerContainers.html">Running Applications in Docker Containers</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/RuncContainers.html">Running Applications in runC Containers</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/NodeManagerCgroups.html">Using CGroups</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/SecureContainer.html">Secure Containers</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/ReservationSystem.html">Reservation System</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/GracefulDecommission.html">Graceful Decommission</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/OpportunisticContainers.html">Opportunistic Containers</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/Federation.html">YARN Federation</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/SharedCache.html">Shared Cache</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/UsingGpus.html">Using GPU</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/UsingFPGA.html">Using FPGA</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/PlacementConstraints.html">Placement Constraints</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/YarnUI2.html">YARN UI2</a> |
| </li> |
| </ul> |
| <h5>YARN REST APIs</h5> |
| <ul> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/WebServicesIntro.html">Introduction</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html">Resource Manager</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/NodeManagerRest.html">Node Manager</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/TimelineServer.html#Timeline_Server_REST_API_v1">Timeline Server</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/TimelineServiceV2.html#Timeline_Service_v.2_REST_API">Timeline Service V.2</a> |
| </li> |
| </ul> |
| <h5>YARN Service</h5> |
| <ul> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/yarn-service/Overview.html">Overview</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/yarn-service/QuickStart.html">QuickStart</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/yarn-service/Concepts.html">Concepts</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/yarn-service/YarnServiceAPI.html">Yarn Service API</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/yarn-service/ServiceDiscovery.html">Service Discovery</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-site/yarn-service/SystemServices.html">System Services</a> |
| </li> |
| </ul> |
| <h5>Hadoop Compatible File Systems</h5> |
| <ul> |
| <li class="none"> |
| <a href="../../hadoop-aliyun/tools/hadoop-aliyun/index.html">Aliyun OSS</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-aws/tools/hadoop-aws/index.html">Amazon S3</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-azure/index.html">Azure Blob Storage</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-azure-datalake/index.html">Azure Data Lake Storage</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-openstack/index.html">OpenStack Swift</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-cos/cloud-storage/index.html">Tencent COS</a> |
| </li> |
| </ul> |
| <h5>Auth</h5> |
| <ul> |
| <li class="none"> |
| <a href="../../hadoop-auth/index.html">Overview</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-auth/Examples.html">Examples</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-auth/Configuration.html">Configuration</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-auth/BuildingIt.html">Building</a> |
| </li> |
| </ul> |
| <h5>Tools</h5> |
| <ul> |
| <li class="none"> |
| <a href="../../hadoop-streaming/HadoopStreaming.html">Hadoop Streaming</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-archives/HadoopArchives.html">Hadoop Archives</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-archive-logs/HadoopArchiveLogs.html">Hadoop Archive Logs</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-distcp/DistCp.html">DistCp</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-gridmix/GridMix.html">GridMix</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-rumen/Rumen.html">Rumen</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-resourceestimator/ResourceEstimator.html">Resource Estimator Service</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-sls/SchedulerLoadSimulator.html">Scheduler Load Simulator</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/Benchmarking.html">Hadoop Benchmarking</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-dynamometer/Dynamometer.html">Dynamometer</a> |
| </li> |
| </ul> |
| <h5>Reference</h5> |
| <ul> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/release/">Changelog and Release Notes</a> |
| </li> |
| <li class="none"> |
| <a href="../../api/index.html">Java API docs</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/UnixShellAPI.html">Unix Shell API</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/Metrics.html">Metrics</a> |
| </li> |
| </ul> |
| <h5>Configuration</h5> |
| <ul> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/core-default.xml">core-default.xml</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs/hdfs-default.xml">hdfs-default.xml</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-hdfs-rbf/hdfs-rbf-default.xml">hdfs-rbf-default.xml</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml">mapred-default.xml</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-yarn/hadoop-yarn-common/yarn-default.xml">yarn-default.xml</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-kms/kms-default.html">kms-default.xml</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-hdfs-httpfs/httpfs-default.html">httpfs-default.xml</a> |
| </li> |
| <li class="none"> |
| <a href="../../hadoop-project-dist/hadoop-common/DeprecatedProperties.html">Deprecated Properties</a> |
| </li> |
| </ul> |
| <a href="http://maven.apache.org/" title="Built by Maven" class="poweredBy"> |
| <img alt="Built by Maven" src="./images/logos/maven-feather.png"/> |
| </a> |
| |
| </div> |
| </div> |
| <div id="bodyColumn"> |
| <div id="contentBox"> |
| <!--- |
| Licensed under the Apache License, Version 2.0 (the "License"); |
| you may not use this file except in compliance with the License. |
| You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. See accompanying LICENSE file. |
| --> |
| <h1>YARN Node Attributes</h1> |
| <ul> |
| <li><a href="#Overview">Overview</a></li> |
| <li><a href="#Features">Features</a></li> |
| <li><a href="#Configuration">Configuration</a> |
| <ul> |
| <li><a href="#Setting_up_ResourceManager_for_Node_Attributes">Setting up ResourceManager for Node Attributes</a></li> |
| <li><a href="#Centralised_Node_Attributes_mapping.">Centralised Node Attributes mapping.</a></li> |
| <li><a href="#Distributed_Node_Attributes_mapping.">Distributed Node Attributes mapping.</a></li></ul></li> |
| <li><a href="#Specifying_node_attributes_for_application">Specifying node attributes for application</a></li> |
| <li><a href="#Monitoring">Monitoring</a> |
| <ul> |
| <li><a href="#Monitoring_through_REST">Monitoring through REST</a></li> |
| <li><a href="#Monitoring_through_web_UI">Monitoring through web UI</a></li> |
| <li><a href="#Monitoring_through_commandline">Monitoring through commandline</a></li></ul></li> |
| <li><a href="#Useful_links">Useful links</a></li></ul> |
| |
| <div class="section"> |
| <h2><a name="Overview"></a>Overview</h2> |
| <p>Node Attribute is a way to describe the attributes of a Node without resource guarantees. This could be used by applications to pick up the right nodes for their container to be placed based on expression of multitude of these attributes.</p></div> |
| <div class="section"> |
| <h2><a name="Features"></a>Features</h2> |
| <p>The salient features of <tt>Node Attributes</tt> is as follows:</p> |
| <ul> |
| |
| <li>A Node can be associated with multiple attributes.</li> |
| <li>Value can be associated with a attribute tagged to a node. String type values are only supported currently.</li> |
| <li>Unlike Node Labels, Node Attributes need not be specified explicitly at the cluster level, but there are API’s to list the attributes available at the cluster level.</li> |
| <li>As its non tangible resource, its not associated with any queue and thus queue resource planning and authorisation is not required for attributes.</li> |
| <li>Similar to the allocation tags, Applications will be able to request containers using expressions containing one or more of these attributes using <i>Placement Constraints</i>.</li> |
| <li>Equals (=) and Not Equals (!=) are the only supported operators in the expression. AND & OR can also be used as part of attribute expression.</li> |
| <li>Node attribute constraints are hard limits, that says the allocation can only be made if the node satisfies the node attribute constraint. In another word, the request keeps pending until it finds a valid node satisfying the constraint. There is no relax policy at present.</li> |
| <li>Operability |
| <ul> |
| |
| <li>Node Attributes and its mapping to nodes can be recovered across RM restart</li> |
| <li>Update node attributes - admin can add, remove and replace attributes on nodes when RM is running</li> |
| </ul> |
| </li> |
| <li>Mapping of NM to node attributes can be done in two ways, |
| <ul> |
| |
| <li><b>Centralised :</b> Node to attributes mapping can be done through RM exposed CLI or RPC (REST is yet to be supported).</li> |
| <li><b>Distributed :</b> Node to attributes mapping will be set by a configured Node Attributes Provider in NM. We have two different providers in YARN: <i>Script</i> based provider and <i>Configuration</i> based provider. In case of script, NM can be configured with a script path and the script can emit the attribute(s) of the node. In case of config, node Attributes can be directly configured in the NM’s yarn-site.xml. In both of these options dynamic refresh of the attribute mapping is supported.</li> |
| </ul> |
| </li> |
| <li> |
| |
| <p>Unlike labels, attributes can be mapped to a node from both Centralised and Distributed modes at the same time. There will be no clashes as attributes are identified with different prefix in different modes. In case of <b>Centralized</b> attributes are identified by prefix <i>“rm.yarn.io”</i> and in case of <b>Distributed</b> attributes are identified by prefix <i>“nm.yarn.io”</i>. This implies attributes are uniquely identified by <i>prefix</i> and <i>name</i>.</p> |
| </li> |
| </ul></div> |
| <div class="section"> |
| <h2><a name="Configuration"></a>Configuration</h2> |
| <div class="section"> |
| <h3><a name="Setting_up_ResourceManager_for_Node_Attributes"></a>Setting up ResourceManager for Node Attributes</h3> |
| <p>Unlike Node Labels, Node Attributes need not be explicitly enabled as it will always exist and would have no impact in terms of performance or compatibility even if feature is not used.</p> |
| <p>Setup following properties in <tt>yarn-site.xml</tt></p> |
| <table border="0" class="bodyTable"> |
| <thead> |
| |
| <tr class="a"> |
| <th>Property </th> |
| <th> Value </th> |
| <th> Default Value</th></tr> |
| </thead><tbody> |
| |
| <tr class="b"> |
| <td>yarn.node-attribute.fs-store.root-dir </td> |
| <td> path where centralized attribute mappings are stored </td> |
| <td> <a class="externalLink" href="file:///tmp/hadoop-yarn-${user}/node-attribute/">file:///tmp/hadoop-yarn-${user}/node-attribute/</a></td></tr> |
| <tr class="a"> |
| <td>yarn.node-attribute.fs-store.impl.class </td> |
| <td> Configured class needs to extend org.apache.hadoop.yarn.nodelabels.NodeAttributeStore </td> |
| <td> FileSystemNodeAttributeStore</td></tr> |
| </tbody> |
| </table> |
| <p>Notes:</p> |
| <ul> |
| |
| <li>Make sure <tt>yarn.node-attribute.fs-store.root-dir</tt> is created with resource manager process user and <tt>ResourceManager</tt> has permission to access it. (Typically from “yarn” user)</li> |
| <li>If user want to store node attributes to local file system of RM, paths like <tt>file:///home/yarn/node-attributes</tt> can be used else if in hdfs paths like <tt>hdfs://namenode:port/path/to/store/node-attributes/</tt> can be used.</li> |
| </ul></div> |
| <div class="section"> |
| <h3><a name="Centralised_Node_Attributes_mapping."></a>Centralised Node Attributes mapping.</h3> |
| <p>Three options are supported to map attributes to node in <b>Centralised</b> approach:</p> |
| <ul> |
| |
| <li> |
| |
| <p><b>add</b> Executing <tt>yarn nodeattributes -add “node1:attribute[(type)][=value],attribute2 node2:attribute2[=value],attribute3</tt> adds attributes to the nodes without impacting already existing mapping on the node(s).</p> |
| </li> |
| <li> |
| |
| <p><b>remove</b> Executing <tt>yarn nodeattributes -remove “node1:attribute,attribute1 node2:attribute2"</tt> removes attributes to the nodes without impacting already existing mapping on the node(s).</p> |
| </li> |
| <li> |
| |
| <p><b>replace</b> Executing <tt>yarn nodeattributes -replace “node1:attribute[(type)][=value],attribute1[=value],attribute2 node2:attribute2[=value],attribute3""</tt> replaces the existing attributes to the nodes with the one configured as part of this command.</p> |
| </li> |
| </ul> |
| <p>Notes:</p> |
| <ul> |
| |
| <li>Ports need <b>not</b> be mentioned, attributes are mapped to all the NM instances in the node.</li> |
| <li><i>Space</i> is the delimiter for multiple node-Attribute mapping pair</li> |
| <li><i>“,”</i> is used as delimiter for multiple attributes of a node.</li> |
| <li><i>“type”</i> defaults to string if not specified which is the only type currently supported.</li> |
| <li>All the above 3 operations can be performed only by admin user.</li> |
| </ul></div> |
| <div class="section"> |
| <h3><a name="Distributed_Node_Attributes_mapping."></a>Distributed Node Attributes mapping.</h3> |
| <p>Configuring attributes to nodes in <b>Distributed</b> mode</p> |
| <table border="0" class="bodyTable"> |
| <thead> |
| |
| <tr class="a"> |
| <th>Property </th> |
| <th> Value</th></tr> |
| </thead><tbody> |
| |
| <tr class="b"> |
| <td>yarn.nodemanager.node-attributes.provider </td> |
| <td> Administrators can configure the provider for the node attributes by configuring this parameter in NM. Administrators can configure <i>“config”</i>, <i>“script”</i> or the <i>class name</i> of the provider. Configured class needs to extend <i>org.apache.hadoop.yarn.server.nodemanager.nodelabels.NodeAttributesProvider</i>. If <i>“config”</i> is configured, then <i>“ConfigurationNodeAttributesProvider”</i> and if <i>“script”</i> is configured, then <i>“ScriptBasedNodeAttributesProvider”</i> will be used.</td></tr> |
| <tr class="a"> |
| <td>yarn.nodemanager.node-attributes.provider.fetch-interval-ms </td> |
| <td> When <i>“yarn.nodemanager.node-attributes.provider”</i> is configured with <i>“config”</i>, <i>“script”</i> or the <i>configured class</i> extends NodeAttributesProvider, then periodically node attributes are retrieved from the node attributes provider. This configuration is to define the interval period. If -1 is configured, then node attributes are retrieved from provider only during initialisation. Defaults to 10 mins.</td></tr> |
| <tr class="b"> |
| <td>yarn.nodemanager.node-attributes.provider.fetch-timeout-ms </td> |
| <td> When <i>“yarn.nodemanager.node-attributes.provider”</i> is configured with <i>“script”</i>, then this configuration provides the timeout period after which it will interrupt the script which queries the node attributes. Defaults to 20 mins.</td></tr> |
| <tr class="a"> |
| <td>yarn.nodemanager.node-attributes.provider.script.path </td> |
| <td> The node attribute script NM runs to collect node attributes. Lines in the script output starting with “NODE_ATTRIBUTE:” will be considered as a record of node attribute, attribute name, type and value should be delimited by comma. Each of such lines will be parsed to a node attribute.</td></tr> |
| <tr class="b"> |
| <td>yarn.nodemanager.node-attributes.provider.script.opts </td> |
| <td> The arguments to pass to the node attribute script.</td></tr> |
| <tr class="a"> |
| <td>yarn.nodemanager.node-attributes.provider.configured-node-attributes </td> |
| <td> When “yarn.nodemanager.node-attributes.provider” is configured with “config” then ConfigurationNodeAttributesProvider fetches node attributes from this parameter.</td></tr> |
| </tbody> |
| </table></div></div> |
| <div class="section"> |
| <h2><a name="Specifying_node_attributes_for_application"></a>Specifying node attributes for application</h2> |
| <p>Applications can use Placement Constraint APIs to specify node attribute request as mentioned in <a href="./PlacementConstraints.html">Placement Constraint documentation</a>.</p> |
| <p>Here is an example for creating a Scheduling Request object with NodeAttribute expression:</p> |
| |
| <div> |
| <div> |
| <pre class="source">//expression : AND(python!=3:java=1.8) |
| SchedulingRequest schedulingRequest = |
| SchedulingRequest.newBuilder().executionType( |
| ExecutionTypeRequest.newInstance(ExecutionType.GUARANTEED)) |
| .allocationRequestId(10L).priority(Priority.newInstance(1)) |
| .placementConstraintExpression( |
| PlacementConstraints.and( |
| PlacementConstraints |
| .targetNodeAttribute(PlacementConstraints.NODE, |
| NodeAttributeOpCode.NE, |
| PlacementConstraints.PlacementTargets |
| .nodeAttribute("python", "3")), |
| PlacementConstraints |
| .targetNodeAttribute(PlacementConstraints.NODE, |
| NodeAttributeOpCode.EQ, |
| PlacementConstraints.PlacementTargets |
| .nodeAttribute("java", "1.8"))) |
| .build()).resourceSizing( |
| ResourceSizing.newInstance(1, Resource.newInstance(1024, 1))) |
| .build(); |
| </pre></div></div> |
| |
| <p>The above SchedulingRequest requests for 1 container on nodes that must satisfy following constraints:</p> |
| <ol style="list-style-type: decimal"> |
| |
| <li> |
| |
| <p>Node attribute <i><tt>rm.yarn.io/python</tt></i> doesn’t exist on the node or it exist but its value is not equal to 3</p> |
| </li> |
| <li> |
| |
| <p>Node attribute <i><tt>rm.yarn.io/java</tt></i> must exist on the node and its value is equal to 1.8</p> |
| </li> |
| </ol></div> |
| <div class="section"> |
| <h2><a name="Monitoring"></a>Monitoring</h2> |
| <div class="section"> |
| <h3><a name="Monitoring_through_REST"></a>Monitoring through REST</h3> |
| <p>As part of <i><tt>http://rm-http-address:port/ws/v1/cluster/nodes/{nodeid}</tt></i> REST output attributes and its values mapped to the given node can be got.</p></div> |
| <div class="section"> |
| <h3><a name="Monitoring_through_web_UI"></a>Monitoring through web UI</h3> |
| <p>Yet to be supported</p></div> |
| <div class="section"> |
| <h3><a name="Monitoring_through_commandline"></a>Monitoring through commandline</h3> |
| <ul> |
| |
| <li>Use <tt>yarn cluster --list-node-attributes</tt> to get all the attributes in the cluster</li> |
| <li>Use <tt>yarn nodeattributes -list</tt> to get attributes in the cluster</li> |
| <li>Use <tt>yarn nodeattributes -attributestonodes -attributes <Attributes></tt> to list for each attribute, all the mapped nodes and the attribute value configured for each node. Optionally we can specify for the specified attributes using <i>-attributes</i>.</li> |
| <li>Use <tt>yarn nodeattributes -nodestoattributes -nodes <Host Names></tt> to list all the attributes and its value mapped to a node. Optionally we can specify for the specified node using <i>-nodes</i>.</li> |
| <li>Node status/detail got from <tt>yarn node -status</tt> will list all the attributes and its value associated with the node.</li> |
| </ul></div></div> |
| <div class="section"> |
| <h2><a name="Useful_links"></a>Useful links</h2> |
| <ul> |
| |
| <li><a href="./PlacementConstraints.html">Placement Constraint documentation</a>, if you need more understanding about how to configure Placement Constraints.</li> |
| </ul></div> |
| </div> |
| </div> |
| <div class="clear"> |
| <hr/> |
| </div> |
| <div id="footer"> |
| <div class="xright"> |
| © 2008-2021 |
| Apache Software Foundation |
| |
| - <a href="http://maven.apache.org/privacy-policy.html">Privacy Policy</a>. |
| Apache Maven, Maven, Apache, the Apache feather logo, and the Apache Maven project logos are trademarks of The Apache Software Foundation. |
| </div> |
| <div class="clear"> |
| <hr/> |
| </div> |
| </div> |
| </body> |
| </html> |
