91320b4461
HDFS-14859. Prevent unnecessary evaluation of costly operation getNumLiveDataNodes when dfs.namenode.safemode.min.datanodes is not zero. Contributed by Srinivasu Majeti.
2019-10-08 14:14:14 +05:30
761594549e
HDFS-14814. RBF: RouterQuotaUpdateService supports inherited rule. Contributed by Jinglun.
2019-10-08 14:01:44 +05:30
4fdf016358
HDDS-2245. Use dynamic ports for SCM in TestSecureOzoneCluster
...
Contributed by kevin su.
2019-10-07 15:41:42 -07:00
012d897e5b
HDDS-2262. SLEEP_SECONDS: command not found
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-10-07 14:38:54 -07:00
aaa94c3da6
HDDS-2259. Container Data Scrubber computes wrong checksum
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-10-07 14:30:23 -07:00
cfba6ac951
HDDS-2264. Improve output of TestOzoneContainer
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-10-07 14:10:57 -07:00
1877312440
HDDS-2238. Container Data Scrubber spams log in empty cluster
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-10-07 13:59:49 -07:00
382967be51
HDFS-14373. EC : Decoding is failing when block group last incomplete cell fall in to AlignedStripe. Contributed by Surendra Singh Lilhore.
2019-10-08 00:14:30 +05:30
9685a6c0e5
HDDS-2239. Fix TestOzoneFsHAUrls ( #1600 )
2019-10-07 10:17:25 -07:00
1a77a15fe4
HADOOP-16587. Make ABFS AAD endpoints configurable.
...
Contributed by Bilahari T H.
This also addresses HADOOP-16498: AzureADAuthenticator cannot authenticate
in China.
Change-Id: I2441dd48b50b59b912b0242f7f5a4418cf94a87c
2019-10-07 13:07:46 +01:00
7f332ebf8b
HDDS-2252. Enable gdpr robot test in daily build
...
Closes #1602
2019-10-07 11:35:39 +02:00
14cd969b6e
HADOOP-16512. [hadoop-tools] Fix order of actual and expected expression in assert statements
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-10-07 16:38:08 +09:00
022fe5f5b2
HDDS-2169. Avoid buffer copies while submitting client requests in Ratis. Contributed by Tsz-wo Sze( #1517 ).
2019-10-07 09:34:36 +05:30
55c5436f39
Revert "HADOOP-16579. Upgrade to Apache Curator 4.2.0 excluding ZK ( #1531 ). Contributed by Norbert Kalmár."
...
This reverts commit 10bdc592d6
2019-10-05 11:16:11 -07:00
b8086bf54d
HADOOP-16626. S3A ITestRestrictedReadAccess fails without S3Guard.
...
Contributed by Steve Loughran.
Change-Id: Ife730b80057ddd43e919438cb5b2abbda990e636
2019-10-05 12:52:42 +01:00
579dc87015
HDDS-2251. Add an option to customize unit.sh and integration.sh parameters
...
Closes #1598
2019-10-05 13:40:02 +02:00
fb1ecff6a2
Revert "YARN-9873. Mutation API Config Change updates Version Number. Contributed by Prabhu Joseph"
...
This reverts commit 4510970e2f
2019-10-05 09:15:17 +05:30
f209722a19
HDDS-2257. Fix checkstyle issues in ChecksumByteBuffer ( #1603 )
2019-10-04 16:36:32 -07:00
a3cf54ccdc
HDDS-2250. Generated configs missing from ozone-filesystem-lib jars
...
Closes #1597
2019-10-04 23:31:22 +02:00
8de4374427
HDDS-2158. Fixing Json Injection Issue in JsonUtils. ( #1486 )
2019-10-04 12:52:29 -07:00
f3eaa84f9d
HDDS-2164 : om.db.checkpoints is getting filling up fast. ( #1536 )
2019-10-04 12:44:21 -07:00
10bdc592d6
HADOOP-16579. Upgrade to Apache Curator 4.2.0 excluding ZK ( #1531 ). Contributed by Norbert Kalmár.
2019-10-04 10:57:07 -07:00
6574f27fa3
HADOOP-16570. S3A committers encounter scale issues.
...
Contributed by Steve Loughran.
This addresses two scale issues which has surfaced in large scale benchmarks
of the S3A Committers.
* Thread pools are not cleaned up.
This now happens, with tests.
* OOM on job commit for jobs with many thousands of tasks,
each generating tens of (very large) files.
Instead of loading all pending commits into memory as a single list, the list
of files to load is the sole list which is passed around; .pendingset files are
loaded and processed in isolation -and reloaded if necessary for any
abort/rollback operation.
The parallel commit/abort/revert operations now work at the .pendingset level,
rather than that of individual pending commit files. The existing parallelized
Tasks API is still used to commit those files, but with a null thread pool, so
as to serialize the operations.
Change-Id: I5c8240cd31800eaa83d112358770ca0eb2bca797
2019-10-04 18:54:22 +01:00
aa24add8f0
HDFS-14890. Fixed namenode and journalnode startup on Windows.
...
Contributed by Siddharth Wagle
2019-10-04 13:13:10 -04:00
3f166512af
HDDS-2237. KeyDeletingService throws NPE if it's started too early ( #1584 )
2019-10-04 09:48:49 -07:00
4510970e2f
YARN-9873. Mutation API Config Change updates Version Number. Contributed by Prabhu Joseph
2019-10-04 21:49:07 +05:30
f826420d2b
HDDS-2230. Invalid entries in ozonesecure-mr config. (Addendum)
2019-10-04 17:37:51 +02:00
531cc938fe
HDDS-2222. Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C. ( #1595 )
2019-10-04 21:16:28 +08:00
f44abc3e11
HADOOP-16207 Improved S3A MR tests.
...
Contributed by Steve Loughran.
Replaces the committer-specific terasort and MR test jobs with parameterization
of the (now single tests) and use of file:// over hdfs:// as the cluster FS.
The parameterization ensures that only one of the specific committer tests
run at a time -overloads of the test machines are less likely, and so the
suites can be pulled back into the parallel phase.
There's also more detailed validation of the stage outputs of the terasorting;
if one test fails the rest are all skipped. This and the fact that job
output is stored under target/yarn-${timestamp} means failures should
be more debuggable.
Change-Id: Iefa370ba73c6419496e6e69dd6673d00f37ff095
2019-10-04 14:12:31 +01:00
bca014b0e0
HDDS-2216. Rename HADOOP_RUNNER_VERSION to OZONE_RUNNER_VERSION in compose .env files.
...
Closes #1570 .
2019-10-04 14:57:33 +02:00
6171a41b4c
HDDS-2199. In SCMNodeManager dnsToUuidMap cannot track multiple DNs on the same host
...
Closes #1551
2019-10-04 14:00:06 +02:00
d061c8469f
HDDS-2140. Add robot test for GDPR feature
...
Closes #1542
2019-10-04 13:51:57 +02:00
bffcd33085
HDDS-2230. Invalid entries in ozonesecure-mr config
...
Doroszlai, Attila <adoroszlai@apache.org>
2019-10-04 13:24:42 +02:00
a9849f65ba
Revert "HDDS-2222 ( #1578 )" ( #1594 )
...
This reverts commit 4cf0b3660f
2019-10-04 19:19:45 +08:00
4cf0b3660f
HDDS-2222 ( #1578 )
...
Thanks @jnp for reviewing this. Merging now.
2019-10-04 17:50:21 +08:00
2478cbafe6
YARN-9782. Avoid DNS resolution while running SLS. Contributed by Abhishek Modi.
2019-10-04 14:45:10 +05:30
b23bdaf085
HDFS-14879. Header was wrong in Snapshot web UI. Contributed by hemanthboyina
2019-10-04 16:47:06 +09:00
ec8f691201
HDDS-2225. SCM fails to start in most unsecure environments due to leftover secure config
...
Closes #1568
2019-10-04 08:50:13 +02:00
c99a12167f
HDFS-14637. Namenode may not replicate blocks to meet the policy after enabling upgradeDomain. Contributed by Stephen O'Donnell.
...
Reviewed-by: Ayush Saxena <ayushsaxena@apache.org>
Signed-off-by: Wei-Chiu Chuang <weichiu@apache.org>
2019-10-03 22:13:50 -07:00
844b766da5
HDFS-14889. Ability to check if a block has a replica on provided storage. Contributed by Ashvin Agrawal. ( #1573 )"
2019-10-03 21:32:48 -07:00
9700e2003a
HDDS-2223. Support ReadWrite lock in LockManager. ( #1564 )
2019-10-04 08:32:43 +05:30
cdaa480dbf
HDDS-2198. SCM should not consider containers in CLOSING state to come out of safemode. ( #1540 )
2019-10-04 08:20:58 +05:30
1dde3efb91
HADOOP-16624. Upgrade hugo to the latest version in Dockerfile
...
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>
2019-10-04 10:01:07 +09:00
b7cb8fe07c
HDDS-2200 : Recon does not handle the NULL snapshot from OM DB cleanly.
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-10-03 15:38:19 -07:00
76605f17dd
HDDS-1720 : Add ability to configure RocksDB logs for Ozone Manager.
...
Signed-off-by: Anu Engineer <aengineer@apache.org>
2019-10-03 15:17:09 -07:00
944668674b
HDDS-2231. test-single.sh cannot copy results ( #1575 )
2019-10-03 14:50:04 -07:00
47d721d7dd
HDDS-2234. rat.sh fails due to ozone-recon-web/build files ( #1580 )
2019-10-03 14:46:50 -07:00
51eaecab20
HDDS-2211. Collect docker logs if env fails to start ( #1553 )
2019-10-03 12:44:04 -07:00
a3fe404244
HDFS-14881. Safemode 'forceExit' option, doesn’t shown in help message. Contributed by Renukaprasad C.
2019-10-03 22:34:55 +05:30
d59bcbfa0f
HDDS-2226. S3 Secrets should use a strong RNG. ( #1572 )
2019-10-03 09:28:41 -07:00
Ayush Saxena
Anu Engineer
Doroszlai, Attila
Surendra Singh Lilhore
Doroszlai, Attila
bilaharith
dchitlangia
pingsutw
Shashikant Banerjee
Wei-Chiu Chuang
Steve Loughran
Márton Elek
Sunil G
Vivek Ratnavel Subramanian
Hanisha Koneru
avijayanhwx
Norbert Kalmar
Eric Yang
Elek, Márton
Tsz-Wo Nicholas Sze
cxorm
S O'Donnell
Abhishek Modi
Takanobu Asanuma
Virajith Jalaparti
Nanda kumar
Aravindan Vijayan