d55d76e1e2
HADOOP-18371. S3A FS init to log at debug when fs.s3a.create.storage.class is unset ( #4730 )
...
Contributed By: Viraj Jasani
2022-08-16 03:55:58 +05:30
622ca0d51f
HDFS-16705. RBF supports healthMonitor timeout configurable and caching NN and client proxy in NamenodeHeartbeatService ( #4662 )
2022-08-15 13:55:16 -07:00
eff3b8c59a
YARN-10885. Make FederationStateStoreFacade#getApplicationHomeSubCluster use JCache. ( #4701 )
2022-08-15 13:46:40 -07:00
b1d4af2492
HDFS-16704. Datanode return empty response instead of NPE for GetVolumeInfo during restarting ( #4661 ). Contributed by ZanderXu.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-08-15 20:57:27 +08:00
86cc96c493
Revert "HDFS-16689. NameNode may crash when transitioning to Active with in-progress tailer if there are some abnormal JNs. ( #4628 )" ( #4743 )
2022-08-15 19:48:05 +08:00
906ae5138e
HADOOP-18402. S3A committer NPE in spark job abort ( #4735 )
...
JobID.toString() and TaskID.toString() to only be called
when the IDs are not null.
This doesn't surface in MapReduce, but Spark SQL can trigger
in job abort, where it may invok abortJob() with an
incomplete TaskContext.
This patch MUST be applied to branches containing
HADOOP-17833. "Improve Magic Committer Performance."
Contributed by Steve Loughran.
2022-08-15 11:20:59 +01:00
eee59a8372
Revert "HADOOP-18402. S3A committer NPE in spark job abort ( #4735 )"
...
(managed to commit through the github ui before I'd got the message done)
This reverts commit ad83e95046
2022-08-15 11:20:36 +01:00
ad83e95046
HADOOP-18402. S3A committer NPE in spark job abort ( #4735 )
...
jobId.toString() to only be called when the ID isn't null.
this doesn't surface in MR, but spark seems to manage it
Change-Id: I06692ef30a4af510c660d7222292932a8d4b5147
2022-08-15 11:18:47 +01:00
ab88e4b65d
YARN-11223. [Federation] Add getAppPriority, updateApplicationPriority REST APIs for Router. ( #4733 )
2022-08-14 19:22:16 -07:00
d0fdb1d6e0
HADOOP-18404. Fix broken link to wiki help page in org.apache.hadoop.util.Shell ( #4718 ). Contributed by Paul King.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-08-14 19:28:22 +05:30
d383cc4525
YARN-11236. Implement FederationReservationHomeSubClusterStore With MemoryStore. ( #4711 )
2022-08-13 10:37:20 -07:00
9f6bbc90a8
YARN-11148. In federation and security mode, nm recover may fail. ( #4308 )
2022-08-13 10:33:16 -07:00
b737869e01
HADOOP-18383. Codecs with @DoNotPool annotation are not closed causing memory leak ( #4585 )
2022-08-12 16:05:13 -07:00
e0c8c6eed4
HDFS-16678. RBF should supports disable getNodeUsage() in RBFMetrics ( #4606 )
2022-08-12 12:01:58 -07:00
521e65acfe
HDFS-16723. Replace incorrect SafeModeException with StandbyException in RouterRpcServer.class ( #4716 ). Contributed by ZanderXu.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-08-13 00:21:11 +05:30
8c9533a0f8
HADOOP-18397. Shutdown AWSSecurityTokenService when its resources are no longer in use ( #4722 )
...
Contributed by Viraj Jasani.
2022-08-12 11:59:15 +01:00
59619ad247
HDFS-16689. NameNode may crash when transitioning to Active with in-progress tailer if there are some abnormal JNs. ( #4628 )
2022-08-12 12:19:28 +08:00
2005582a28
HDFS-16702. MiniDFSCluster should report cause of exception in assert error ( #4680 )
...
When the MiniDFSClsuter detects that an exception caused an exit, it should include that exception as the cause for the AssertionError that it throws. The current AssertError simply reports the message "Test resulted in an unexpected exit" and provides a stack trace to the location of the check for an exit exception. This patch adds the original exception as the cause of the AssertError.
2022-08-11 13:52:39 -07:00
6ca2d3f848
YARN-6539. Create SecureLogin inside Router. ( #4712 )
2022-08-11 13:25:51 -07:00
09cabaad68
HDFS-13274. RBF: Extend RouterRpcClient to use multiple sockets ( #4531 )
2022-08-11 13:23:32 -07:00
b28e4c6904
HADOOP-18392. Propagate vectored s3a input stream stats to file system stats. ( #4704 )
...
part of HADOOP-18103.
Contributed By: Mukund Thakur
2022-08-12 01:42:00 +05:30
e9509ac467
HADOOP-18340. deleteOnExit does not work with S3AFileSystem ( #4608 )
...
Contributed by Huaxiang Sun
2022-08-11 20:25:13 +01:00
c0bbdca97e
HADOOP-18398. Prevent AvroRecord*.class from being included non-test jar ( #4727 )
...
Contributed by Yubi Lee.
2022-08-11 20:12:41 +01:00
133e8aabf0
YARN-11227. [Federation] Add getAppTimeout, getAppTimeouts, updateApplicationTimeout REST APIs for Router. ( #4715 )
2022-08-10 14:53:46 -07:00
ffa9ed93a4
YARN-6572. Refactoring Router services to use common util classes for pipeline creations. ( #4594 )
2022-08-09 14:44:29 -07:00
92abd99450
YARN-11237. Fix Bug while disabling proxy failover with Federation ( #4658 )
2022-08-08 13:29:27 -07:00
977f4b6165
MAPREDUCE-7385. impove JobEndNotifier#httpNotification With recommended methods. ( #4403 ). Contributed by fanshilun.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-08-09 00:59:03 +05:30
895f7c51fd
HDFS-16709. Remove redundant cast in FSEditLogOp.class ( #4667 ). Contributed by ZanderXu.
...
Reviewed-by: Inigo Goiri <inigoiri@apache.org>
Reviewed-by: He Xiaoqiao <hexiaoqiao@apache.org>
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-08-09 00:54:01 +05:30
06f0f7db79
HADOOP-18373. IOStatisticsContext tuning ( #4705 )
...
The name of the option to enable/disable thread level statistics is
"fs.iostatistics.thread.level.enabled";
There is also an enabled() probe in IOStatisticsContext which can
be used to see if the thread level statistics is active.
Contributed by Viraj Jasani
2022-08-08 10:42:57 +01:00
d8d3325d2f
HADOOP-18387. Fix incorrect placeholder in hadoop-common ( #4679 ). Contributed by fanshilun.
2022-08-08 02:35:39 +05:30
1cda2dcb6e
YARN-10793. Upgrade Junit from 4 to 5 in hadoop-yarn-server-applicationhistoryservice ( #4603 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-08-07 16:15:47 +09:00
52c2d99889
YARN-11228. [Federation] Add getAppAttempts, getAppAttempt REST APIs for Router. ( #4695 )
2022-08-06 09:36:26 -07:00
25ccdc77af
HDFS-16648. Add isDebugEnabled check for debug blockLogs in some classes ( #4529 )
2022-08-06 21:34:01 +08:00
bd0f9a46e1
HADOOP-18390. Fix out of sync import for HADOOP-18321 ( #4694 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-08-06 21:51:23 +09:00
b5642c5638
HADOOP-18366. ITestS3Select.testSelectSeekFullLandsat is timing out. ( #4702 )
...
Reduces size of data read to 1 MB
Contributed by Ahmar Suhail
2022-08-05 14:13:04 +01:00
62dbefd8f2
HADOOP-18305. Release Hadoop 3.3.4: upstream changelog and jdiff files
...
Add the r3.3.4 changelog, release notes and jdiff xml files.
2022-08-05 14:06:22 +01:00
080e67039d
HADOOP-17234. Addendum. Add .asf.yaml to allow github and jira integration. ( #4686 ). Contributed by Ayush Saxena.
...
Reviewed-by: Mingliang Liu <liuml07@apache.org>
2022-08-05 08:34:56 +05:30
6f7c4c74ea
YARN-11235. Refactor Policy Code and Define getReservationHomeSubcluster ( #4656 )
2022-08-04 10:16:08 -07:00
0aa08ef543
HADOOP-18363. Fix bug preventing hadoop-metrics2 from emitting metrics to > 1 Ganglia servers ( #4627 )
...
* HADOOP-18363. Fix bug preventing hadoop-metrics2 from emitting metrics to > 1 Ganglia servers
2022-08-04 18:26:38 +05:30
dbf73e16b1
HADOOP-18364. All method metrics related to the RPC protocol should be initialized. ( #4624 ). Contributed by Shuyan Zhang.
...
Reviewed-by: Erik Krogen <xkrogen@apache.org>
Reviewed-by: Chao Sun <sunchao@apache.org>
Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
2022-08-04 13:00:37 +08:00
8eebf40b1a
HDFS-16642. Moving the selecting inputstream from journalnode in EditLogTailer outof FSNLock ( #4497 )
2022-08-04 11:04:28 +08:00
66dec9d322
HADOOP-18355. Update previous index properly while validating overlapping ranges. ( #4647 )
...
part of HADOOP-18103.
Contributed By: Mukund Thakur
2022-08-04 04:08:04 +05:30
c5ec727435
YARN-11230. [Federation] Add getContainer, signalToContainer REST APIs for Router. ( #4689 )
2022-08-03 11:21:48 -07:00
6463f86f83
YARN-11029. Refactor AMRMProxy Service code and Added Some Metrics. ( #4650 )
2022-08-03 09:38:00 -07:00
c5eba323bc
YARN-6972. Adding RM ClusterId in AppInfo. ( #4673 )
2022-08-03 09:35:40 -07:00
0fc7dd8228
HDFS-16687. RouterFsckServlet replicates code from DfsServlet base class ( #4681 )
2022-08-03 09:14:11 -07:00
0f36539d60
HDFS-16712. Fix incorrect placeholder in DataNode.java ( #4672 ). Contributed by ZanderXu.
...
Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
2022-08-03 13:01:41 +05:30
69f6fdb757
HADOOP-18301. Upgrade commons-io to 2.11.0 ( #4455 )
...
Co-authored-by: Ashutosh Gupta <ashugpt@amazon.com>
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2022-08-03 10:44:39 +09:00
1f0a71a92b
YARN-8973. [Router] Add missing methods in RMWebProtocol. ( #4664 )
2022-08-02 14:07:09 -07:00
57da4bb0a1
YARN-11220. [Federation] Add getLabelsToNodes, getClusterNodeLabels, getLabelsOnNode REST APIs for Router ( #4657 )
2022-08-02 12:09:55 -07:00
Viraj Jasani
xuzq
slfan1989
Hui Fei
Steve Loughran
Paul King
zhengchenyu
kevins-29
Steve Vaughan
Mukund Thakur
huaxiangsun
Yubi Lee
Ashutosh Gupta
ahmarsuhail
Ayush Saxena
zhangshuyan0