YARN-4753. Use doxia macro to generate in-page TOC of YARN site documentation. (iwasakims)
This commit is contained in:
Masatake Iwasaki
parent
58acb676a2
commit
cba7e2c346
|
|
@ -15,19 +15,7 @@
|
|||
Hadoop: Capacity Scheduler
|
||||
==========================
|
||||
|
||||
* [Purpose](#Purpose)
|
||||
* [Overview](#Overview)
|
||||
* [Features](#Features)
|
||||
* [Configuration](#Configuration)
|
||||
* [Setting up `ResourceManager` to use `CapacityScheduler`](#Setting_up_ResourceManager_to_use_CapacityScheduler`)
|
||||
* [Setting up queues](#Setting_up_queues)
|
||||
* [Queue Properties](#Queue_Properties)
|
||||
* [Setup for application priority](#Setup_for_application_priority.)
|
||||
* [Capacity Scheduler container preemption](#Capacity_Scheduler_container_preemption)
|
||||
* [Configuring `ReservationSystem` with `CapacityScheduler`](#Configuring_ReservationSystem_with_CapacityScheduler)
|
||||
* [Other Properties](#Other_Properties)
|
||||
* [Reviewing the configuration of the CapacityScheduler](#Reviewing_the_configuration_of_the_CapacityScheduler)
|
||||
* [Changing Queue Configuration](#Changing_Queue_Configuration)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
|
||||
|
||||
Purpose
|
||||
-------
|
||||
|
|
|
|||
|
|
@ -15,12 +15,7 @@
|
|||
Docker Container Executor
|
||||
=========================
|
||||
|
||||
* [Overview](#Overview)
|
||||
* [Cluster Configuration](#Cluster_Configuration)
|
||||
* [Tips for connecting to a secure docker repository](#Tips_for_connecting_to_a_secure_docker_repository)
|
||||
* [Job Configuration](#Job_Configuration)
|
||||
* [Docker Image Requirements](#Docker_Image_Requirements)
|
||||
* [Working example of yarn launched docker containers](#Working_example_of_yarn_launched_docker_containers)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=1} -->
|
||||
|
||||
DEPRECATED
|
||||
----------
|
||||
|
|
|
|||
|
|
@ -15,13 +15,7 @@
|
|||
Launching Applications Using Docker Containers
|
||||
==============================================
|
||||
|
||||
* [Overview](#Overview)
|
||||
* [Cluster Configuration](#Cluster_Configuration)
|
||||
* [Docker Image Requirements](#Docker_Image_Requirements)
|
||||
* [Application Submission](#Application_Submission)
|
||||
* [Connecting to a Secure Docker Repository](#Connecting_to_a_Secure_Docker_Repository)
|
||||
* [Example: MapReduce](#Example:_MapReduce)
|
||||
* [Example: Spark](#Example:_Spark)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=1} -->
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
|
|
|||
|
|
@ -15,19 +15,7 @@
|
|||
Hadoop: Fair Scheduler
|
||||
======================
|
||||
|
||||
* [Purpose](#Purpose)
|
||||
* [Introduction](#Introduction)
|
||||
* [Hierarchical queues with pluggable policies](#Hierarchical_queues_with_pluggable_policies)
|
||||
* [Automatically placing applications in queues](#Automatically_placing_applications_in_queues)
|
||||
* [Installation](#Installation)
|
||||
* [Configuration](#Configuration)
|
||||
* [Properties that can be placed in yarn-site.xml](#Properties_that_can_be_placed_in_yarn-site.xml)
|
||||
* [Allocation file format](#Allocation_file_format)
|
||||
* [Queue Access Control Lists](#Queue_Access_Control_Lists)
|
||||
* [Administration](#Administration)
|
||||
* [Modifying configuration at runtime](#Modifying_configuration_at_runtime)
|
||||
* [Monitoring through web UI](#Monitoring_through_web_UI)
|
||||
* [Moving applications between queues](#Moving_applications_between_queues)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
|
||||
|
||||
##Purpose
|
||||
|
||||
|
|
|
|||
|
|
@ -15,18 +15,7 @@
|
|||
YARN Node Labels
|
||||
===============
|
||||
|
||||
* [Overview](#Overview)
|
||||
* [Features](#Features)
|
||||
* [Configuration](#Configuration)
|
||||
* [Setting up ResourceManager to enable Node Labels](#Setting_up_ResourceManager_to_enable_Node_Labels)
|
||||
* [Add/modify node labels list to YARN](#Add/modify_node_labels_list_to_YARN)
|
||||
* [Add/modify node-to-labels mapping to YARN](#Add/modify_node-to-labels_mapping_to_YARN)
|
||||
* [Configuration of Schedulers for node labels](#Configuration_of_Schedulers_for_node_labels)
|
||||
* [Specifying node label for application](#Specifying_node_label_for_application)
|
||||
* [Monitoring](#Monitoring)
|
||||
* [Monitoring through web UI](#Monitoring_through_web_UI)
|
||||
* [Monitoring through commandline](#Monitoring_through_commandline)
|
||||
* [Useful links](#Useful_links)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
|
|
|||
|
|
@ -15,13 +15,7 @@
|
|||
NodeManager
|
||||
===========
|
||||
|
||||
* [Overview](#Overview)
|
||||
* [Health Checker Service](#Health_checker_service)
|
||||
* [Disk Checker](#Disk_Checker)
|
||||
* [External Health Script](#External_Health_Script)
|
||||
* [NodeManager Restart](#NodeManager_Restart)
|
||||
* [Introduction](#Introduction)
|
||||
* [Enabling NM Restart](#Enabling_NM_Restart)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
|
|
|||
|
|
@ -15,8 +15,7 @@
|
|||
Using CGroups with YARN
|
||||
=======================
|
||||
|
||||
* [CGroups Configuration](#CGroups_configuration)
|
||||
* [CGroups and Security](#CGroups_and_security)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
|
||||
|
||||
CGroups is a mechanism for aggregating/partitioning sets of tasks, and all their future children, into hierarchical groups with specialized behaviour. CGroups is a Linux kernel feature and was merged into kernel version 2.6.24. From a YARN perspective, this allows containers to be limited in their resource usage. A good example of this is CPU usage. Without CGroups, it becomes hard to limit container CPU usage. Currently, CGroups is only used for limiting CPU usage.
|
||||
|
||||
|
|
|
|||
|
|
@ -15,13 +15,7 @@
|
|||
NodeManager REST API's
|
||||
=======================
|
||||
|
||||
* [Overview](#Overview)
|
||||
* [Enabling CORS support](#Enabling_CORS_support)
|
||||
* [NodeManager Information API](#NodeManager_Information_API)
|
||||
* [Applications API](#Applications_API)
|
||||
* [Application API](#Application_API)
|
||||
* [Containers API](#Containers_API)
|
||||
* [Container API](#Container_API)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=1} -->
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
|
|
|||
|
|
@ -15,22 +15,7 @@
|
|||
Opportunistic Containers
|
||||
========================
|
||||
|
||||
* [Purpose](#Purpose)
|
||||
* [Quick Guide](#Quick_Guide)
|
||||
* [Main Goal](#Main_Goal)
|
||||
* [Enabling Opportunistic Containers](#Enabling_Opportunistic_Containers)
|
||||
* [Running a Sample Job](#Running_a_Sample_Job)
|
||||
* [Opportunistic Containers in Web UI](#Opportunistic_Containers_in_Web_UI)
|
||||
* [Overview](#Overview)
|
||||
* [Container Execution Types](#Container_Execution_Types)
|
||||
* [Execution of Opportunistic Containers](#Execution_of_Opportunistic_Containers)
|
||||
* [Allocation of Opportunistic Containers](#Allocation_of_Opportunistic_Containers)
|
||||
* [Centralized Allocation](#Centralized_Allocation)
|
||||
* [Distributed Allocation](#Distributed_Allocation)
|
||||
* [Determining Nodes for Allocation](#Determining_Nodes_for_Allocation)
|
||||
* [Rebalancing Node Load](#Rebalancing_Node_Load)
|
||||
* [Advanced Configuration](#Advanced_Configuration)
|
||||
* [Items for Future Work](#Items_for_Future_Work)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
|
||||
|
||||
|
||||
<a name="Purpose"></a>Purpose
|
||||
|
|
|
|||
|
|
@ -15,11 +15,7 @@
|
|||
Reservation System
|
||||
==================
|
||||
|
||||
|
||||
* [Purpose](#Purpose)
|
||||
* [Overview](#Overview)
|
||||
* [Flow of a Reservation](#Flow_of_a_Reservation)
|
||||
* [Configuring the Reservation System](#Configuring_the_Reservation_System)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
|
||||
|
||||
Purpose
|
||||
-------
|
||||
|
|
|
|||
|
|
@ -15,15 +15,7 @@
|
|||
ResourceManager High Availability
|
||||
=================================
|
||||
|
||||
* [Introduction](#Introduction)
|
||||
* [Architecture](#Architecture)
|
||||
* [RM Failover](#RM_Failover)
|
||||
* [Recovering prevous active-RM's state](#Recovering_prevous_active-RMs_state)
|
||||
* [Deployment](#Deployment)
|
||||
* [Configurations](#Configurations)
|
||||
* [Admin commands](#Admin_commands)
|
||||
* [ResourceManager Web UI services](#ResourceManager_Web_UI_services)
|
||||
* [Web Services](#Web_Services)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=2} -->
|
||||
|
||||
Introduction
|
||||
------------
|
||||
|
|
|
|||
|
|
@ -15,32 +15,7 @@
|
|||
ResourceManager REST API's.
|
||||
===========================
|
||||
|
||||
* [Overview](#Overview)
|
||||
* [Enabling CORS support](#Enabling_CORS_support)
|
||||
* [Cluster Information API](#Cluster_Information_API)
|
||||
* [Cluster Metrics API](#Cluster_Metrics_API)
|
||||
* [Cluster Scheduler API](#Cluster_Scheduler_API)
|
||||
* [Cluster Applications API](#Cluster_Applications_API)
|
||||
* [Cluster Application Statistics API](#Cluster_Application_Statistics_API)
|
||||
* [Cluster Application API](#Cluster_Application_API)
|
||||
* [Cluster Application Attempts API](#Cluster_Application_Attempts_API)
|
||||
* [Cluster Nodes API](#Cluster_Nodes_API)
|
||||
* [Cluster Node API](#Cluster_Node_API)
|
||||
* [Cluster Writeable APIs](#Cluster_Writeable_APIs)
|
||||
* [Cluster New Application API](#Cluster_New_Application_API)
|
||||
* [Cluster Applications API(Submit Application)](#Cluster_Applications_APISubmit_Application)
|
||||
* [Cluster Application State API](#Cluster_Application_State_API)
|
||||
* [Cluster Application Queue API](#Cluster_Application_Queue_API)
|
||||
* [Cluster Application Priority API](#Cluster_Application_Priority_API)
|
||||
* [Cluster Delegation Tokens API](#Cluster_Delegation_Tokens_API)
|
||||
* [Cluster Reservation API List](#Cluster_Reservation_API_List)
|
||||
* [Cluster Reservation API Create](#Cluster_Reservation_API_Create)
|
||||
* [Cluster Reservation API Submit](#Cluster_Reservation_API_Submit)
|
||||
* [Cluster Reservation API Update](#Cluster_Reservation_API_Update)
|
||||
* [Cluster Reservation API Delete](#Cluster_Reservation_API_Delete)
|
||||
* [Cluster Application Timeouts API](#Cluster_Application_Timeouts_API)
|
||||
* [Cluster Application Timeout API](#Cluster_Application_Timeout_API)
|
||||
* [Cluster Application Timeout Update API](#Cluster_Application_Timeout_Update_API)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=1} -->
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
|
|
|||
|
|
@ -15,18 +15,7 @@
|
|||
ResourceManger Restart
|
||||
======================
|
||||
|
||||
* [Overview](#Overview)
|
||||
* [Feature](#Feature)
|
||||
* [Configurations](#Configurations)
|
||||
* [Enable RM Restart](#Enable_RM_Restart)
|
||||
* [Configure the state-store for persisting the RM state](#Configure_the_state-store_for_persisting_the_RM_state)
|
||||
* [How to choose the state-store implementation](#How_to_choose_the_state-store_implementation)
|
||||
* [Configurations for Hadoop FileSystem based state-store implementation](#Configurations_for_Hadoop_FileSystem_based_state-store_implementation)
|
||||
* [Configurations for ZooKeeper based state-store implementation](#Configurations_for_ZooKeeper_based_state-store_implementation)
|
||||
* [Configurations for LevelDB based state-store implementation](#Configurations_for_LevelDB_based_state-store_implementation)
|
||||
* [Configurations for work-preserving RM recovery](#Configurations_for_work-preserving_RM_recovery)
|
||||
* [Notes](#Notes)
|
||||
* [Sample Configurations](#Sample_Configurations)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@
|
|||
YARN Secure Containers
|
||||
======================
|
||||
|
||||
* [Overview](#Overview)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
|
|
|||
|
|
@ -15,21 +15,7 @@
|
|||
The YARN Timeline Server
|
||||
========================
|
||||
|
||||
* [Overview](#Overview)
|
||||
* [Introduction](#Introduction)
|
||||
* [Current Status](#Current_Status)
|
||||
* [Timeline Structure](#Timeline_Structure)
|
||||
* [Deployment](#Deployment)
|
||||
* [Configurations](#Configurations)
|
||||
* [Running the Timeline Server](#Running_Timeline_Server)
|
||||
* [Accessing generic-data via command-line](#Accessing_generic-data_via_command-line)
|
||||
* [Publishing of application specific data](#Publishing_of_application_specific_data)
|
||||
* [Timeline Server REST API](#Timeline_Server_REST_API_v1)
|
||||
* [Generic Data REST APIs](#GENERIC_DATA_REST_APIS)
|
||||
* [Timelnine Server Performance Test Tool](#TIMELINE_SERVER_PERFORMANCE_TEST_TOOL)
|
||||
* [Highlights](#HIGHLIGHTS)
|
||||
* [Usage](#USAGE)
|
||||
* [Sample Runs](#SAMPLE_RUNS)
|
||||
<!-- MACRO{toc|fromDepth=1|toDepth=1} -->
|
||||
|
||||
<a name="Overview"></a>Overview
|
||||
---------
|
||||
|
|
|
|||
|
|
@ -15,12 +15,7 @@
|
|||
Web Application Proxy
|
||||
=====================
|
||||
|
||||
* [Overview](#Overview)
|
||||
* [Introduction](#Introduction)
|
||||
* [Current Status](#Current_Status)
|
||||
* [Deployment](#Deployment)
|
||||
* [Configurations](#Configurations)
|
||||
* [Running Web Application Proxy](#Running_Web_Proxy)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
|
||||
|
||||
|
||||
Overview
|
||||
|
|
|
|||
|
|
@ -15,18 +15,7 @@
|
|||
Hadoop YARN - Introduction to the web services REST API's
|
||||
==========================================================
|
||||
|
||||
* [Overview](#Overview)
|
||||
* [URI's](#URIs)
|
||||
* [HTTP Requests](#HTTP_Requests)
|
||||
* [Summary of HTTP operations](#Summary_of_HTTP_operations)
|
||||
* [Security](#Security)
|
||||
* [Headers Supported](#Headers_Supported)
|
||||
* [HTTP Responses](#HTTP_Responses)
|
||||
* [Compression](#Compression)
|
||||
* [Response Formats](#Response_Formats)
|
||||
* [Response Errors](#Response_Errors)
|
||||
* [Response Examples](#Response_Examples)
|
||||
* [Sample Usage](#Sample_Usage)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=2} -->
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
|
|
|||
|
|
@ -15,19 +15,7 @@
|
|||
Hadoop: Writing YARN Applications
|
||||
=================================
|
||||
|
||||
* [Purpose](#Purpose)
|
||||
* [Concepts and Flow](#Concepts_and_Flow)
|
||||
* [Interfaces](#Interfaces)
|
||||
* [Writing a Simple Yarn Application](#Writing_a_Simple_Yarn_Application)
|
||||
* [Writing a simple Client](#Writing_a_simple_Client)
|
||||
* [Writing an ApplicationMaster (AM)](#Writing_an_ApplicationMaster_AM)
|
||||
* [FAQ](#FAQ)
|
||||
* [How can I distribute my application's jars to all of the nodes in the YARN cluster that need it?](#How_can_I_distribute_my_applications_jars_to_all_of_the_nodes_in_the_YARN_cluster_that_need_it)
|
||||
* [How do I get the ApplicationMaster's ApplicationAttemptId?](#How_do_I_get_the_ApplicationMasters_ApplicationAttemptId)
|
||||
* [Why my container is killed by the NodeManager?](#Why_my_container_is_killed_by_the_NodeManager)
|
||||
* [How do I include native libraries?](#How_do_I_include_native_libraries)
|
||||
* [Useful Links](#Useful_Links)
|
||||
* [Sample Code](#Sample_Code)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
|
||||
|
||||
Purpose
|
||||
-------
|
||||
|
|
|
|||
|
|
@ -14,6 +14,8 @@
|
|||
|
||||
# YARN Application Security
|
||||
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=1} -->
|
||||
|
||||
Anyone writing a YARN application needs to understand the process, in order
|
||||
to write short-lived applications or long-lived services. They also need to
|
||||
start testing on secure clusters during early development stages, in order
|
||||
|
|
|
|||
|
|
@ -15,26 +15,7 @@
|
|||
YARN Commands
|
||||
=============
|
||||
|
||||
* [Overview](#Overview)
|
||||
* [User Commands](#User_Commands)
|
||||
* [application](#application)
|
||||
* [applicationattempt](#applicationattempt)
|
||||
* [classpath](#classpath)
|
||||
* [container](#container)
|
||||
* [jar](#jar)
|
||||
* [logs](#logs)
|
||||
* [node](#node)
|
||||
* [queue](#queue)
|
||||
* [version](#version)
|
||||
* [Administration Commands](#Administration_Commands)
|
||||
* [daemonlog](#daemonlog)
|
||||
* [nodemanager](#nodemanager)
|
||||
* [proxyserver](#proxyserver)
|
||||
* [resourcemanager](#resourcemanager)
|
||||
* [rmadmin](#rmadmin)
|
||||
* [scmadmin](#scmadmin)
|
||||
* [sharedcachemanager](#sharedcachemanager)
|
||||
* [timelineserver](#timelineserver)
|
||||
<!-- MACRO{toc|fromDepth=0|toDepth=3} -->
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
|
|
|||
Loading…
Reference in New Issue