diff --git a/hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm b/hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm index 1e76bafec59..dd4eb0a3f6a 100644 --- a/hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm +++ b/hadoop-common-project/hadoop-common/src/site/apt/CommandsManual.apt.vm @@ -81,36 +81,15 @@ User Commands * <<>> - Creates a hadoop archive. More information can be found at Hadoop - Archives. - - Usage: <<* >>> - -*-------------------+-------------------------------------------------------+ -||COMMAND_OPTION || Description -*-------------------+-------------------------------------------------------+ -| -archiveName NAME | Name of the archive to be created. -*-------------------+-------------------------------------------------------+ -| src | Filesystem pathnames which work as usual with regular - | expressions. -*-------------------+-------------------------------------------------------+ -| dest | Destination directory which would contain the archive. -*-------------------+-------------------------------------------------------+ + Creates a hadoop archive. More information can be found at + {{{../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/HadoopArchives.html} + Hadoop Archives Guide}}. * <<>> Copy file or directories recursively. More information can be found at - Hadoop DistCp Guide. - - Usage: << >>> - -*-------------------+--------------------------------------------+ -||COMMAND_OPTION || Description -*-------------------+--------------------------------------------+ -| srcurl | Source Url -*-------------------+--------------------------------------------+ -| desturl | Destination Url -*-------------------+--------------------------------------------+ + {{{../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistCp.html} + Hadoop DistCp Guide}}. * <<>> @@ -142,103 +121,21 @@ User Commands * <<>> - Command to interact with Map Reduce Jobs. - - Usage: <<] | [-status ] | [-counter ] | [-kill ] | [-events <#-of-events>] | [-history [all] ] | [-list [all]] | [-kill-task ] | [-fail-task ] | [-set-priority ]>>> - -*------------------------------+---------------------------------------------+ -|| COMMAND_OPTION || Description -*------------------------------+---------------------------------------------+ -| -submit | Submits the job. -*------------------------------+---------------------------------------------+ -| -status | Prints the map and reduce completion - | percentage and all job counters. -*------------------------------+---------------------------------------------+ -| -counter | Prints the counter value. -*------------------------------+---------------------------------------------+ -| -kill | Kills the job. -*------------------------------+---------------------------------------------+ -| -events <#-of-events> | Prints the events' details - | received by jobtracker for the given range. -*------------------------------+---------------------------------------------+ -| -history [all] | Prints job details, failed and killed tip - | details. More details about the job such as - | successful tasks and task attempts made for - | each task can be viewed by specifying the [all] - | option. -*------------------------------+---------------------------------------------+ -| -list [all] | Displays jobs which are yet to complete. - | <<<-list all>>> displays all jobs. -*------------------------------+---------------------------------------------+ -| -kill-task | Kills the task. Killed tasks are NOT counted - | against failed attempts. -*------------------------------+---------------------------------------------+ -| -fail-task | Fails the task. Failed tasks are counted - | against failed attempts. -*------------------------------+---------------------------------------------+ -| -set-priority | Changes the priority of the job. Allowed - | priority values are VERY_HIGH, HIGH, NORMAL, - | LOW, VERY_LOW -*------------------------------+---------------------------------------------+ + Deprecated. Use + {{{../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapredCommands.html#job} + <<>>}} instead. * <<>> - Runs a pipes job. - - Usage: <<] [-jobconf , , - ...] [-input ] [-output ] [-jar ] [-inputformat - ] [-map ] [-partitioner ] [-reduce ] [-writer - ] [-program ] [-reduces ]>>> - -*----------------------------------------+------------------------------------+ -|| COMMAND_OPTION || Description -*----------------------------------------+------------------------------------+ -| -conf | Configuration for job -*----------------------------------------+------------------------------------+ -| -jobconf , , ... | Add/override configuration for job -*----------------------------------------+------------------------------------+ -| -input | Input directory -*----------------------------------------+------------------------------------+ -| -output | Output directory -*----------------------------------------+------------------------------------+ -| -jar | Jar filename -*----------------------------------------+------------------------------------+ -| -inputformat | InputFormat class -*----------------------------------------+------------------------------------+ -| -map | Java Map class -*----------------------------------------+------------------------------------+ -| -partitioner | Java Partitioner -*----------------------------------------+------------------------------------+ -| -reduce | Java Reduce class -*----------------------------------------+------------------------------------+ -| -writer | Java RecordWriter -*----------------------------------------+------------------------------------+ -| -program | Executable URI -*----------------------------------------+------------------------------------+ -| -reduces | Number of reduces -*----------------------------------------+------------------------------------+ + Deprecated. Use + {{{../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapredCommands.html#pipes} + <<>>}} instead. * <<>> - command to interact and view Job Queue information - - Usage: << [-showJobs]] | [-showacls]>>> - -*-----------------+-----------------------------------------------------------+ -|| COMMAND_OPTION || Description -*-----------------+-----------------------------------------------------------+ -| -list | Gets list of Job Queues configured in the system. - | Along with scheduling information associated with the job queues. -*-----------------+-----------------------------------------------------------+ -| -info [-showJobs] | Displays the job queue information and - | associated scheduling information of particular job queue. - | If <<<-showJobs>>> options is present a list of jobs - | submitted to the particular job queue is displayed. -*-----------------+-----------------------------------------------------------+ -| -showacls | Displays the queue name and associated queue operations - | allowed for the current user. The list consists of only - | those queues to which the user has access. -*-----------------+-----------------------------------------------------------+ + Deprecated. Use + {{{../../hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapredCommands.html#queue} + <<>>}} instead. * <<>> diff --git a/hadoop-mapreduce-project/CHANGES.txt b/hadoop-mapreduce-project/CHANGES.txt index 6f8afd913f5..fc0ee321484 100644 --- a/hadoop-mapreduce-project/CHANGES.txt +++ b/hadoop-mapreduce-project/CHANGES.txt @@ -44,6 +44,9 @@ Release 2.6.0 - UNRELEASED MAPREDUCE-5944. Remove MRv1 commands from CommandsManual.apt.vm (Akira AJISAKA via aw) + MAPREDUCE-5943. Separate mapred commands from CommandManual.apt.vm + (Akira AJISAKA via aw) + Release 2.5.0 - UNRELEASED INCOMPATIBLE CHANGES diff --git a/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/MapredCommands.apt.vm b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/MapredCommands.apt.vm new file mode 100644 index 00000000000..795eb344e64 --- /dev/null +++ b/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/MapredCommands.apt.vm @@ -0,0 +1,224 @@ +~~ Licensed to the Apache Software Foundation (ASF) under one or more +~~ contributor license agreements. See the NOTICE file distributed with +~~ this work for additional information regarding copyright ownership. +~~ The ASF licenses this file to You under the Apache License, Version 2.0 +~~ (the "License"); you may not use this file except in compliance with +~~ the License. You may obtain a copy of the License at +~~ +~~ http://www.apache.org/licenses/LICENSE-2.0 +~~ +~~ Unless required by applicable law or agreed to in writing, software +~~ distributed under the License is distributed on an "AS IS" BASIS, +~~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +~~ See the License for the specific language governing permissions and +~~ limitations under the License. + + --- + MapReduce Commands Guide + --- + --- + ${maven.build.timestamp} + +MapReduce Commands Guide + +%{toc|section=1|fromDepth=2|toDepth=4} + +* Overview + + MapReduce commands are invoked by the <<>> script. Running the + script without any arguments prints the description for all commands. + + Usage: <<>> + + MapReduce has an option parsing framework that employs parsing generic + options as well as running classes. + +*-------------------------+---------------------------------------------------+ +|| COMMAND_OPTIONS || Description | +*-------------------------+---------------------------------------------------+ +| --config confdir | Overwrites the default Configuration directory. Default +| | is $\{HADOOP_PREFIX\}/conf. +*-------------------------+---------------------------------------------------+ +| COMMAND COMMAND_OPTIONS | Various commands with their options are described +| | in the following sections. The commands have been +| | grouped into {{User Commands}} and +| | {{Administration Commands}}. +*-------------------------+---------------------------------------------------+ + +* User Commands + + Commands useful for users of a hadoop cluster. + +** <<>> + + Runs a pipes job. + + Usage: <<] [-jobconf , , + ...] [-input ] [-output ] [-jar ] [-inputformat + ] [-map ] [-partitioner ] [-reduce ] [-writer + ] [-program ] [-reduces ]>>> + +*----------------------------------------+------------------------------------+ +|| COMMAND_OPTION || Description +*----------------------------------------+------------------------------------+ +| -conf | Configuration for job +*----------------------------------------+------------------------------------+ +| -jobconf , , ... | Add/override configuration for job +*----------------------------------------+------------------------------------+ +| -input | Input directory +*----------------------------------------+------------------------------------+ +| -output | Output directory +*----------------------------------------+------------------------------------+ +| -jar | Jar filename +*----------------------------------------+------------------------------------+ +| -inputformat | InputFormat class +*----------------------------------------+------------------------------------+ +| -map | Java Map class +*----------------------------------------+------------------------------------+ +| -partitioner | Java Partitioner +*----------------------------------------+------------------------------------+ +| -reduce | Java Reduce class +*----------------------------------------+------------------------------------+ +| -writer | Java RecordWriter +*----------------------------------------+------------------------------------+ +| -program | Executable URI +*----------------------------------------+------------------------------------+ +| -reduces | Number of reduces +*----------------------------------------+------------------------------------+ + +** <<>> + + Command to interact with Map Reduce Jobs. + + Usage: <<] + | [-status ] + | [-counter ] + | [-kill ] + | [-events <#-of-events>] + | [-history [all] ] | [-list [all]] + | [-kill-task ] | [-fail-task ] + | [-set-priority ]>>> + +*------------------------------+---------------------------------------------+ +|| COMMAND_OPTION || Description +*------------------------------+---------------------------------------------+ +| -submit | Submits the job. +*------------------------------+---------------------------------------------+ +| -status | Prints the map and reduce completion + | percentage and all job counters. +*------------------------------+---------------------------------------------+ +| -counter | Prints the counter value. +*------------------------------+---------------------------------------------+ +| -kill | Kills the job. +*------------------------------+---------------------------------------------+ +| -events <#-of-events> | Prints the events' details + | received by jobtracker for the given range. +*------------------------------+---------------------------------------------+ +| -history [all] | Prints job details, failed and killed tip + | details. More details about the job such as + | successful tasks and task attempts made for + | each task can be viewed by specifying the + | [all] option. +*------------------------------+---------------------------------------------+ +| -list [all] | Displays jobs which are yet to complete. + | <<<-list all>>> displays all jobs. +*------------------------------+---------------------------------------------+ +| -kill-task | Kills the task. Killed tasks are NOT counted + | against failed attempts. +*------------------------------+---------------------------------------------+ +| -fail-task | Fails the task. Failed tasks are counted + | against failed attempts. +*------------------------------+---------------------------------------------+ +| -set-priority | Changes the priority of the job. Allowed + | priority values are VERY_HIGH, HIGH, NORMAL, + | LOW, VERY_LOW +*------------------------------+---------------------------------------------+ + +** <<>> + + command to interact and view Job Queue information + + Usage: << [-showJobs]] + | [-showacls]>>> + +*-----------------+-----------------------------------------------------------+ +|| COMMAND_OPTION || Description +*-----------------+-----------------------------------------------------------+ +| -list | Gets list of Job Queues configured in the system. + | Along with scheduling information associated with the job + | queues. +*-----------------+-----------------------------------------------------------+ +| -info [-showJobs] | Displays the job queue information and + | associated scheduling information of particular job queue. + | If <<<-showJobs>>> options is present a list of jobs + | submitted to the particular job queue is displayed. +*-----------------+-----------------------------------------------------------+ +| -showacls | Displays the queue name and associated queue operations + | allowed for the current user. The list consists of only + | those queues to which the user has access. +*-----------------+-----------------------------------------------------------+ + +** <<>> + + Prints the class path needed to get the Hadoop jar and the required + libraries. + + Usage: <<>> + +** <<>> + + Copy file or directories recursively. More information can be found at + {{{./DistCp.html}Hadoop DistCp Guide}}. + +** <<>> + + Creates a hadoop archive. More information can be found at + {{{./HadoopArchives.html}Hadoop Archives Guide}}. + +* Administration Commands + + Commands useful for administrators of a hadoop cluster. + +** <<>> + + Start JobHistoryServer. + + Usage: <<>> + +** <<>> + + Runs a MapReduce hsadmin client for execute JobHistoryServer administrative + commands. + + Usage: <<>> + +*-----------------+-----------------------------------------------------------+ +|| COMMAND_OPTION || Description +*-----------------+-----------------------------------------------------------+ +| -refreshUserToGroupsMappings | Refresh user-to-groups mappings +*-----------------+-----------------------------------------------------------+ +| -refreshSuperUserGroupsConfiguration| Refresh superuser proxy groups mappings +*-----------------+-----------------------------------------------------------+ +| -refreshAdminAcls | Refresh acls for administration of Job history server +*-----------------+-----------------------------------------------------------+ +| -refreshLoadedJobCache | Refresh loaded job cache of Job history server +*-----------------+-----------------------------------------------------------+ +| -refreshJobRetentionSettings|Refresh job history period, job cleaner settings +*-----------------+-----------------------------------------------------------+ +| -refreshLogRetentionSettings | Refresh log retention period and log retention +| | check interval +*-----------------+-----------------------------------------------------------+ +| -getGroups [username] | Get the groups which given user belongs to +*-----------------+-----------------------------------------------------------+ +| -help [cmd] | Displays help for the given command or all commands if none is +| | specified. +*-----------------+-----------------------------------------------------------+ diff --git a/hadoop-project/src/site/site.xml b/hadoop-project/src/site/site.xml index 512d98d5a8b..42183ab6a43 100644 --- a/hadoop-project/src/site/site.xml +++ b/hadoop-project/src/site/site.xml @@ -94,6 +94,7 @@ +