HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<?xml version="1.0" encoding="UTF-8"?>
|
2019-05-21 15:34:25 -04:00
|
|
|
<project xmlns="https://maven.apache.org/POM/4.0.0" xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<!--
|
|
|
|
/**
|
|
|
|
* Licensed to the Apache Software Foundation (ASF) under one
|
|
|
|
* or more contributor license agreements. See the NOTICE file
|
|
|
|
* distributed with this work for additional information
|
|
|
|
* regarding copyright ownership. The ASF licenses this file
|
|
|
|
* to you under the Apache License, Version 2.0 (the
|
|
|
|
* "License"); you may not use this file except in compliance
|
|
|
|
* with the License. You may obtain a copy of the License at
|
|
|
|
*
|
|
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
*
|
|
|
|
* Unless required by applicable law or agreed to in writing, software
|
|
|
|
* distributed under the License is distributed on an "AS IS" BASIS,
|
|
|
|
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
|
|
* See the License for the specific language governing permissions and
|
|
|
|
* limitations under the License.
|
|
|
|
*/
|
|
|
|
-->
|
|
|
|
<modelVersion>4.0.0</modelVersion>
|
|
|
|
<parent>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
2017-08-19 04:23:52 -04:00
|
|
|
<artifactId>hbase-build-configuration</artifactId>
|
2022-10-24 00:15:18 -04:00
|
|
|
<version>${revision}</version>
|
2017-08-19 04:23:52 -04:00
|
|
|
<relativePath>../hbase-build-configuration</relativePath>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
</parent>
|
|
|
|
<artifactId>hbase-mapreduce</artifactId>
|
|
|
|
<name>Apache HBase - MapReduce</name>
|
|
|
|
<description>This module contains implementations of InputFormat, OutputFormat, Mapper, Reducer, etc which
|
|
|
|
are needed for running MR jobs on tables, WALs, HFiles and other HBase specific constructs.
|
|
|
|
It also contains a bunch of tools: RowCounter, ImportTsv, Import, Export, CompactionTool,
|
2021-04-27 11:41:15 -04:00
|
|
|
ExportSnapshot, WALPlayer, etc</description>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
|
|
|
|
<dependencies>
|
2017-08-31 11:03:22 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase.thirdparty</groupId>
|
|
|
|
<artifactId>hbase-shaded-miscellaneous</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase.thirdparty</groupId>
|
|
|
|
<artifactId>hbase-shaded-netty</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase.thirdparty</groupId>
|
|
|
|
<artifactId>hbase-shaded-protobuf</artifactId>
|
|
|
|
</dependency>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<!-- Intra-project dependencies -->
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-annotations</artifactId>
|
|
|
|
<type>test-jar</type>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
2020-05-13 05:59:21 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-logging</artifactId>
|
|
|
|
<type>test-jar</type>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-common</artifactId>
|
|
|
|
</dependency>
|
2017-11-27 20:40:42 -05:00
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-zookeeper</artifactId>
|
|
|
|
</dependency>
|
2017-12-06 03:38:34 -05:00
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-zookeeper</artifactId>
|
|
|
|
<type>test-jar</type>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
2017-09-02 16:14:09 -04:00
|
|
|
<dependency>
|
|
|
|
<!--Needed by ExportSnapshot. It is reading
|
|
|
|
Snapshot protos. TODO: Move to internal types.-->
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-protocol</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<!--Needed by ExportSnapshot. It is reading
|
|
|
|
Snapshot protos. TODO: Move to internal types.-->
|
|
|
|
<groupId>com.google.protobuf</groupId>
|
|
|
|
<artifactId>protobuf-java</artifactId>
|
|
|
|
</dependency>
|
2017-08-31 11:03:22 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-protocol-shaded</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-metrics</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-metrics-api</artifactId>
|
|
|
|
</dependency>
|
2020-05-09 00:00:45 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-asyncfs</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-asyncfs</artifactId>
|
|
|
|
<type>test-jar</type>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
2017-09-02 16:14:09 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>io.dropwizard.metrics</groupId>
|
|
|
|
<artifactId>metrics-core</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<!--This is not used by hbase directly. Used by thrift,
|
|
|
|
dropwizard and zk.-->
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.slf4j</groupId>
|
|
|
|
<artifactId>slf4j-api</artifactId>
|
|
|
|
</dependency>
|
2017-08-31 11:03:22 -04:00
|
|
|
<dependency>
|
2021-07-29 12:15:10 -04:00
|
|
|
<groupId>io.opentelemetry</groupId>
|
|
|
|
<artifactId>opentelemetry-api</artifactId>
|
2017-08-31 11:03:22 -04:00
|
|
|
</dependency>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-common</artifactId>
|
|
|
|
<type>test-jar</type>
|
2017-12-06 03:38:34 -05:00
|
|
|
<scope>test</scope>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-client</artifactId>
|
|
|
|
</dependency>
|
2017-09-02 16:14:09 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-hadoop-compat</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>${compat.module}</artifactId>
|
|
|
|
</dependency>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-hadoop-compat</artifactId>
|
|
|
|
<type>test-jar</type>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
2017-09-02 16:14:09 -04:00
|
|
|
<!--Do not removed. maven dependency:analyze says not needed but it
|
|
|
|
is; perhaps it comes in via reflection so maven can't figure it-->
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<groupId>org.apache.hbase</groupId>
|
2017-09-02 16:14:09 -04:00
|
|
|
<artifactId>${compat.module}</artifactId>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<type>test-jar</type>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-server</artifactId>
|
2018-04-09 14:37:44 -04:00
|
|
|
<exclusions>
|
|
|
|
<!-- commons-logging is only used by hbase-http's HttpRequestLog and hbase-server's
|
|
|
|
HBaseTestingUtil. We don't need either of those here, so execlude it.
|
|
|
|
-->
|
|
|
|
<exclusion>
|
|
|
|
<groupId>commons-logging</groupId>
|
|
|
|
<artifactId>commons-logging</artifactId>
|
|
|
|
</exclusion>
|
|
|
|
</exclusions>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-server</artifactId>
|
|
|
|
<type>test-jar</type>
|
2017-12-06 03:38:34 -05:00
|
|
|
<scope>test</scope>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
</dependency>
|
2017-08-31 11:03:22 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hbase</groupId>
|
|
|
|
<artifactId>hbase-replication</artifactId>
|
|
|
|
</dependency>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<!-- General dependencies -->
|
2020-03-11 22:18:09 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>com.github.stephenc.findbugs</groupId>
|
|
|
|
<artifactId>findbugs-annotations</artifactId>
|
|
|
|
<scope>compile</scope>
|
|
|
|
<optional>true</optional>
|
|
|
|
</dependency>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>org.mockito</groupId>
|
2017-10-03 01:53:34 -04:00
|
|
|
<artifactId>mockito-core</artifactId>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
2017-08-31 11:03:22 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>commons-io</groupId>
|
|
|
|
<artifactId>commons-io</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
2017-08-24 14:10:17 -04:00
|
|
|
<groupId>org.apache.commons</groupId>
|
|
|
|
<artifactId>commons-lang3</artifactId>
|
2017-08-31 11:03:22 -04:00
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.zookeeper</groupId>
|
|
|
|
<artifactId>zookeeper</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>junit</groupId>
|
|
|
|
<artifactId>junit</artifactId>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
2020-05-13 05:59:21 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>org.slf4j</groupId>
|
|
|
|
<artifactId>jcl-over-slf4j</artifactId>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.slf4j</groupId>
|
|
|
|
<artifactId>jul-to-slf4j</artifactId>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
2022-03-11 14:17:43 -05:00
|
|
|
<groupId>org.apache.logging.log4j</groupId>
|
|
|
|
<artifactId>log4j-api</artifactId>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.logging.log4j</groupId>
|
|
|
|
<artifactId>log4j-core</artifactId>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.logging.log4j</groupId>
|
|
|
|
<artifactId>log4j-slf4j-impl</artifactId>
|
2020-05-13 05:59:21 -04:00
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
2022-03-11 14:17:43 -05:00
|
|
|
<groupId>org.apache.logging.log4j</groupId>
|
|
|
|
<artifactId>log4j-1.2-api</artifactId>
|
2020-05-13 05:59:21 -04:00
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
HBASE-27340 Artifacts with resolved profiles (#4740)
Make it so our published poms carry the minimum needed to run
an hbase; the published pom has no profiles -- the profiles
specified at build time are resolved, their dependencies inlined,
and then they are stripped -- and no build-time, or plugins
dependencies or properties, etc. Resultant poms have explicit
hadoop lib versions baked in -- no more being able to choose
hbase with hadoop2 or haddop3 at downstream build time by setting
a '-Dhadoop.profile=X.0'.
Pattern is to add profiles when none in sub-modules when
the flatten plugin complains it can't resolve an hadoop
dependency's 'version' (e.g. hadoop-common, hadoop-hdfs).
Adding the hadoop-2.0 and hadoop-3.0 profiles in the sub-module
make it so the flatten plugin can figure 'hadoop.version'
definitively.
Another spin on the above happens when profiles already exist
in submodule but the flatten plugin is complaining it can't
figure figure version on an hadoop dependency NOT under
profiles. Below, we move the delinquent hadoop dependency under
existing profiles (minikdc was the usual dependency outside
profiles in sub-modules that flatten complained about).
Sometimes, moving an hadoop dependency under a profile, there
would be excludes on the local dependency. If the parent pom
excludes section was missing the local excludes, we added them
up to the parent module so all excluding is done up there in
the parent profile dependencyManagement section.
* hbase-asyncfs/pom.xml
* hbase-endpoint/pom.xml
* hbase-examples/pom.xml
* hbase-http/pom.xml
* hbase-rest/pom.xml
* hbase-server/pom.xml
Move the minikdc under profiles so it picks up appropriate hadoop version
when the flatten plugin runs.
* hbase-hadoop2-compat/pom.xml
Add hadoop2 and hadoop3 profiles and move hadoop-common, etc.
under them so we pick up appropriate hadoop version when flatten
plugin runs.
* hbase-mapreduce/pom.xml
Move hadoop dependencies under profiles so right version is
available when the flatten plugin runs.
* hbase-shaded/hbase-shaded-testing-util/pom.xml
Add profiles for hadoop-2.0 and hadoop-3.0 and move the
hadoop dependencies under them.
pom.xml
Add the flatten plugin with the flatten profiles enabled.
Add a few excludes on hadoop profiles picked up from sub-modules.
E.g. exclude bouncycastle bcprov-jdk15 when we include minikdc.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-02 19:43:29 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>org.bouncycastle</groupId>
|
|
|
|
<artifactId>bcprov-jdk15on</artifactId>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
</dependencies>
|
2022-05-01 10:52:40 -04:00
|
|
|
|
|
|
|
<build>
|
2019-03-21 12:34:12 -04:00
|
|
|
<plugins>
|
2022-05-01 10:52:40 -04:00
|
|
|
<plugin>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<!--Make it so assembly:single does nothing in here-->
|
|
|
|
<artifactId>maven-assembly-plugin</artifactId>
|
2019-03-21 12:34:12 -04:00
|
|
|
<configuration>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<skipAssembly>true</skipAssembly>
|
2019-03-21 12:34:12 -04:00
|
|
|
</configuration>
|
|
|
|
</plugin>
|
2022-05-01 10:52:40 -04:00
|
|
|
<plugin>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<groupId>org.apache.maven.plugins</groupId>
|
2017-08-29 01:14:49 -04:00
|
|
|
<artifactId>maven-jar-plugin</artifactId>
|
2019-03-21 12:34:12 -04:00
|
|
|
<configuration>
|
2017-08-29 01:14:49 -04:00
|
|
|
<archive>
|
|
|
|
<manifest>
|
2021-04-27 11:41:15 -04:00
|
|
|
<!--Include the Driver class as the 'main'. Executing the jar will then show a
|
|
|
|
list of the basic MR jobs.-->
|
2017-08-29 01:14:49 -04:00
|
|
|
<mainClass>org/apache/hadoop/hbase/mapreduce/Driver</mainClass>
|
|
|
|
</manifest>
|
|
|
|
</archive>
|
2019-03-21 12:34:12 -04:00
|
|
|
</configuration>
|
|
|
|
</plugin>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<!-- Make a jar and put the sources in the jar -->
|
2022-05-01 10:52:40 -04:00
|
|
|
<plugin>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<groupId>org.apache.maven.plugins</groupId>
|
|
|
|
<artifactId>maven-source-plugin</artifactId>
|
2019-03-21 12:34:12 -04:00
|
|
|
</plugin>
|
2022-05-01 10:52:40 -04:00
|
|
|
<plugin>
|
2018-03-21 21:24:14 -04:00
|
|
|
<groupId>net.revelc.code</groupId>
|
|
|
|
<artifactId>warbucks-maven-plugin</artifactId>
|
2019-03-21 12:34:12 -04:00
|
|
|
</plugin>
|
|
|
|
</plugins>
|
2022-05-01 10:52:40 -04:00
|
|
|
</build>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
<profiles>
|
|
|
|
<!-- Skip the tests in this module -->
|
|
|
|
<profile>
|
|
|
|
<id>skipMapReduceTests</id>
|
|
|
|
<activation>
|
|
|
|
<property>
|
|
|
|
<name>skipMapReduceTests</name>
|
|
|
|
</property>
|
|
|
|
</activation>
|
|
|
|
<properties>
|
|
|
|
<surefire.skipFirstPart>true</surefire.skipFirstPart>
|
|
|
|
<surefire.skipSecondPart>true</surefire.skipSecondPart>
|
|
|
|
</properties>
|
|
|
|
</profile>
|
|
|
|
<!-- profile against Hadoop 2.x: This is the default. -->
|
|
|
|
<profile>
|
|
|
|
<id>hadoop-2.0</id>
|
|
|
|
<activation>
|
|
|
|
<property>
|
|
|
|
<!--Below formatting for dev-support/generate-hadoopX-poms.sh-->
|
|
|
|
<!--h2-->
|
|
|
|
<name>!hadoop.profile</name>
|
|
|
|
</property>
|
|
|
|
</activation>
|
|
|
|
<dependencies>
|
2017-12-14 10:19:34 -05:00
|
|
|
<!-- Hadoop needs Netty 3.x at test scope for the minicluster -->
|
|
|
|
<dependency>
|
|
|
|
<groupId>io.netty</groupId>
|
|
|
|
<artifactId>netty</artifactId>
|
|
|
|
<version>${netty.hadoop.version}</version>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
HBASE-27340 Artifacts with resolved profiles (#4740)
Make it so our published poms carry the minimum needed to run
an hbase; the published pom has no profiles -- the profiles
specified at build time are resolved, their dependencies inlined,
and then they are stripped -- and no build-time, or plugins
dependencies or properties, etc. Resultant poms have explicit
hadoop lib versions baked in -- no more being able to choose
hbase with hadoop2 or haddop3 at downstream build time by setting
a '-Dhadoop.profile=X.0'.
Pattern is to add profiles when none in sub-modules when
the flatten plugin complains it can't resolve an hadoop
dependency's 'version' (e.g. hadoop-common, hadoop-hdfs).
Adding the hadoop-2.0 and hadoop-3.0 profiles in the sub-module
make it so the flatten plugin can figure 'hadoop.version'
definitively.
Another spin on the above happens when profiles already exist
in submodule but the flatten plugin is complaining it can't
figure figure version on an hadoop dependency NOT under
profiles. Below, we move the delinquent hadoop dependency under
existing profiles (minikdc was the usual dependency outside
profiles in sub-modules that flatten complained about).
Sometimes, moving an hadoop dependency under a profile, there
would be excludes on the local dependency. If the parent pom
excludes section was missing the local excludes, we added them
up to the parent module so all excluding is done up there in
the parent profile dependencyManagement section.
* hbase-asyncfs/pom.xml
* hbase-endpoint/pom.xml
* hbase-examples/pom.xml
* hbase-http/pom.xml
* hbase-rest/pom.xml
* hbase-server/pom.xml
Move the minikdc under profiles so it picks up appropriate hadoop version
when the flatten plugin runs.
* hbase-hadoop2-compat/pom.xml
Add hadoop2 and hadoop3 profiles and move hadoop-common, etc.
under them so we pick up appropriate hadoop version when flatten
plugin runs.
* hbase-mapreduce/pom.xml
Move hadoop dependencies under profiles so right version is
available when the flatten plugin runs.
* hbase-shaded/hbase-shaded-testing-util/pom.xml
Add profiles for hadoop-2.0 and hadoop-3.0 and move the
hadoop dependencies under them.
pom.xml
Add the flatten plugin with the flatten profiles enabled.
Add a few excludes on hadoop profiles picked up from sub-modules.
E.g. exclude bouncycastle bcprov-jdk15 when we include minikdc.
Signed-off-by: Andrew Purtell <apurtell@apache.org>
Signed-off-by: Duo Zhang <zhangduo@apache.org>
2022-09-02 19:43:29 -04:00
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hadoop</groupId>
|
|
|
|
<artifactId>hadoop-minikdc</artifactId>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hadoop</groupId>
|
|
|
|
<artifactId>hadoop-common</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hadoop</groupId>
|
|
|
|
<artifactId>hadoop-hdfs</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hadoop</groupId>
|
|
|
|
<artifactId>hadoop-hdfs</artifactId>
|
|
|
|
<type>test-jar</type>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<!--maven dependency:analyze says not needed but tests fail w/o-->
|
|
|
|
<groupId>org.apache.hadoop</groupId>
|
|
|
|
<artifactId>hadoop-minicluster</artifactId>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hadoop</groupId>
|
|
|
|
<artifactId>hadoop-mapreduce-client-core</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hadoop</groupId>
|
|
|
|
<artifactId>hadoop-mapreduce-client-jobclient</artifactId>
|
|
|
|
<type>test-jar</type>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
|
|
|
</dependencies>
|
|
|
|
</profile>
|
|
|
|
<!--
|
|
|
|
profile for building against Hadoop 3.x. Activate using:
|
|
|
|
mvn -Dhadoop.profile=3.0
|
|
|
|
-->
|
|
|
|
<profile>
|
|
|
|
<id>hadoop-3.0</id>
|
|
|
|
<activation>
|
|
|
|
<property>
|
|
|
|
<name>hadoop.profile</name>
|
|
|
|
<value>3.0</value>
|
|
|
|
</property>
|
|
|
|
</activation>
|
|
|
|
<dependencies>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hadoop</groupId>
|
|
|
|
<artifactId>hadoop-minikdc</artifactId>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hadoop</groupId>
|
|
|
|
<artifactId>hadoop-common</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hadoop</groupId>
|
|
|
|
<artifactId>hadoop-hdfs</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hadoop</groupId>
|
|
|
|
<artifactId>hadoop-hdfs</artifactId>
|
|
|
|
<type>test-jar</type>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<!--maven dependency:analyze says not needed but tests fail w/o-->
|
|
|
|
<groupId>org.apache.hadoop</groupId>
|
|
|
|
<artifactId>hadoop-minicluster</artifactId>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hadoop</groupId>
|
|
|
|
<artifactId>hadoop-mapreduce-client-core</artifactId>
|
|
|
|
</dependency>
|
|
|
|
<dependency>
|
|
|
|
<groupId>org.apache.hadoop</groupId>
|
|
|
|
<artifactId>hadoop-mapreduce-client-jobclient</artifactId>
|
|
|
|
<type>test-jar</type>
|
|
|
|
<scope>test</scope>
|
|
|
|
</dependency>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
</dependencies>
|
|
|
|
</profile>
|
2019-03-21 12:34:12 -04:00
|
|
|
<profile>
|
|
|
|
<id>eclipse-specific</id>
|
|
|
|
<activation>
|
|
|
|
<property>
|
|
|
|
<name>m2e.version</name>
|
|
|
|
</property>
|
|
|
|
</activation>
|
|
|
|
<build>
|
|
|
|
<pluginManagement>
|
|
|
|
<plugins>
|
|
|
|
<!--This plugin's configuration is used to store Eclipse m2e settings
|
|
|
|
only. It has no influence on the Maven build itself.-->
|
|
|
|
<plugin>
|
|
|
|
<groupId>org.eclipse.m2e</groupId>
|
|
|
|
<artifactId>lifecycle-mapping</artifactId>
|
|
|
|
<configuration>
|
|
|
|
<lifecycleMappingMetadata>
|
|
|
|
<pluginExecutions/>
|
|
|
|
</lifecycleMappingMetadata>
|
|
|
|
</configuration>
|
|
|
|
</plugin>
|
|
|
|
</plugins>
|
|
|
|
</pluginManagement>
|
|
|
|
</build>
|
|
|
|
</profile>
|
HBASE-18640 Move mapreduce out of hbase-server into separate module.
- Moves out o.a.h.h.{mapred, mapreduce} to new hbase-mapreduce module which depends
on hbase-server because of classes like *Snapshot{Input,Output}Format.java, WALs, replication, etc
- hbase-backup depends on it for WALPlayer and MR job stuff
- A bunch of tools needed to be pulled into hbase-mapreduce becuase of their dependencies on MR.
These are: CompactionTool, LoadTestTool, PerformanceEvaluation, ExportSnapshot
This is better place of them than hbase-server. But ideal place would be in separate hbase-tools module.
- There were some tests in hbase-server which were digging into these tools for static util funtions or
confs. Moved these to better/easily shared place. For eg. security related stuff to HBaseKerberosUtils.
- Note that hbase-mapreduce has secondPartExecution tests. On my machine they took like 20 min, so maybe
more on apache jenkins. That's basically equal reduction of runtime of hbase-server tests, which is a
big win!
Change-Id: Ieeb7235014717ca83ee5cb13b2a27fddfa6838e8
2017-08-20 17:34:16 -04:00
|
|
|
</profiles>
|
|
|
|
</project>
|