Apache
/
hbase
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HBASE-4317 book.xml - minor cleanup of MR examples 

git-svn-id: https://svn.apache.org/repos/asf/hbase/trunk@1163866 13f79535-47bb-0310-9956-ffa450edef68
This commit is contained in:
Doug Meil 2011-08-31 23:31:05 +00:00
parent 1fd899d9d7
commit 7b9a82d8df
1 changed files with 11 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
src/docbkx/book.xml
View File

@ -126,7 +126,7 @@ TableMapReduceUtil.initTableMapperJob(
job.setOutputFormatClass(NullOutputFormat.class); // because we aren't emitting anything from mapper job.setOutputFormatClass(NullOutputFormat.class); // because we aren't emitting anything from mapper
boolean b = job.waitForCompletion(true); boolean b = job.waitForCompletion(true);
if ( b == false) { if (!b) {
throw new IOException("error with job!"); throw new IOException("error with job!");
} }
</programlisting> </programlisting>
@ -169,14 +169,14 @@ TableMapReduceUtil.initTableReducerJob(
job.setNumReduceTasks(0); job.setNumReduceTasks(0);
boolean b = job.waitForCompletion(true); boolean b = job.waitForCompletion(true);
if ( b == false) { if (!b) {
throw new IOException("error with job!"); throw new IOException("error with job!");
} }
</programlisting> </programlisting>
An explanation is required of what <classname>TableMapReduceUtil</classname> is doing, especially with the reducer. An explanation is required of what <classname>TableMapReduceUtil</classname> is doing, especially with the reducer.
<link xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.html">TableOutputFormat</link> is being used <link xlink:href="http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/TableOutputFormat.html">TableOutputFormat</link> is being used
as the outputFormat class, and several parameters are being set on the config (e.g., TableOutputFormat.OUTPUT_TABLE), as as the outputFormat class, and several parameters are being set on the config (e.g., TableOutputFormat.OUTPUT_TABLE), as
well as setting the reducer output keys to <classname>ImmutableBytesWritable</classname> and <classname>Writable</classname>. well as setting the reducer output key to <classname>ImmutableBytesWritable</classname> and reducer value to <classname>Writable</classname>.
These could be set by the programmer on the job and conf, but <classname>TableMapReduceUtil</classname> tries to make things easier. These could be set by the programmer on the job and conf, but <classname>TableMapReduceUtil</classname> tries to make things easier.
<para>The following is the example mapper, which will create a <classname>Put</classname> and matching the input <classname>Result</classname> <para>The following is the example mapper, which will create a <classname>Put</classname> and matching the input <classname>Result</classname>
and emit it. Note: this is what the CopyTable utility does. and emit it. Note: this is what the CopyTable utility does.
@ -189,13 +189,12 @@ public static class MyMapper extends TableMapper&lt;ImmutableBytesWritable, Put&
context.write(row, resultToPut(row,value)); context.write(row, resultToPut(row,value));
} }
private static Put resultToPut(ImmutableBytesWritable key, Result result) private static Put resultToPut(ImmutableBytesWritable key, Result result) throws IOException {
throws IOException { Put put = new Put(key.get());
Put put = new Put(key.get()); for (KeyValue kv : result.raw()) {
for (KeyValue kv : result.raw()) { put.add(kv);
put.add(kv); }
} return put;
return put;
} }
} }
</programlisting> </programlisting>
@ -235,7 +234,7 @@ TableMapReduceUtil.initTableReducerJob(
job.setNumReduceTasks(1); // at least one, adjust as required job.setNumReduceTasks(1); // at least one, adjust as required
boolean b = job.waitForCompletion(true); boolean b = job.waitForCompletion(true);
if ( b == false) { if (!b) {
throw new IOException("error with job!"); throw new IOException("error with job!");
} }
</programlisting> </programlisting>