honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
OpenSearch/test/build.gradle

46 lines
1.6 KiB
Groovy
Raw Normal View History

[CORE] Intorduce shards level locks to prevent concurrent shard modifications Today it's possible that the data directory for a single shard is used by more than on IndexShard->Store instances. While one shard is already closed but has a concurrent recovery running and a new shard is creating it's engine files can conflict and data can potentially be lost. We also remove shards data without checking if there are still users of the files or if files are still open which can cause pending writes / flushes or the delete operation to fail. If the latter is the case the index might be treated as a dangeling index and is brought back to life at a later point in time. This commit introduces a shard level lock that prevents modifications to the shard data while it's still in use. Locks are created per shard and maintined in NodeEnvironment.java. In contrast to most java concurrency primitives those locks are not reentrant. This commit also adds infrastructure that checks if all shard locks are released after tests.
2014-11-16 08:24:29 -05:00
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
Build: Add fixture capabilities to integ tests This change adds a Fixture class for use by gradle. A Fixture is an external process that integration tests will use. It can be added as a dependsOn for integTest, and will automatically be shutdown upon success or failure, as well as relevant information dumped on failure. There is also an example fixture in this change.
2015-12-15 23:14:37 -05:00
import org.elasticsearch.gradle.precommit.PrecommitTasks
Build: Move test framework under a "test" top level dir This allows adding more test projects, eg integ test fixtures that will be coming soon.
2015-12-17 19:57:39 -05:00
subprojects {
Add authentication to reindex-from-remote The tests for authentication extend ESIntegTestCase and use a mock authentication plugin. This way the clients don't have to worry about running it. Sadly, that means we don't really have good coverage on the REST portion of the authentication. This also adds ElasticsearchStatusException, and exception on which you can set an explicit status. The nice thing about it is that you can set the RestStatus that it returns to whatever arbitrary status you like based on the status that comes back from the remote system. reindex-from-remote then uses it to wrap all remote failures, preserving the status from the remote Elasticsearch or whatever proxy is between us and the remove Elasticsearch.
2016-07-06 15:47:45 -04:00
// fixtures is just intermediate parent project
Build: Add fixture capabilities to integ tests This change adds a Fixture class for use by gradle. A Fixture is an external process that integration tests will use. It can be added as a dependsOn for integTest, and will automatically be shutdown upon success or failure, as well as relevant information dumped on failure. There is also an example fixture in this change.
2015-12-15 23:14:37 -05:00
if (name == 'fixtures') return
Build: Move test framework under a "test" top level dir This allows adding more test projects, eg integ test fixtures that will be coming soon.
2015-12-17 19:57:39 -05:00
group = 'org.elasticsearch.test'
Build: Add fixture capabilities to integ tests This change adds a Fixture class for use by gradle. A Fixture is an external process that integration tests will use. It can be added as a dependsOn for integTest, and will automatically be shutdown upon success or failure, as well as relevant information dumped on failure. There is also an example fixture in this change.
2015-12-15 23:14:37 -05:00
apply plugin: 'elasticsearch.build'
Build: Add pom generation to assemble task In preparation for a unified release process, we need to be able to generate the pom files independently of trying to actually publish. This change adds back the maven-publish plugin just for that purpose. The nexus plugin still exists for now, so that we do not break snapshots, but that can be removed at a later time once snapshots are happenign through the unified tools. Note I also changed the dir jars are written into so that all our artifacts are under build/distributions.
2016-05-05 20:53:01 -04:00
apply plugin: 'nebula.maven-base-publish'
apply plugin: 'nebula.maven-scm'
Build: Add fixture capabilities to integ tests This change adds a Fixture class for use by gradle. A Fixture is an external process that integration tests will use. It can be added as a dependsOn for integTest, and will automatically be shutdown upon success or failure, as well as relevant information dumped on failure. There is also an example fixture in this change.
2015-12-15 23:14:37 -05:00
Add authentication to reindex-from-remote The tests for authentication extend ESIntegTestCase and use a mock authentication plugin. This way the clients don't have to worry about running it. Sadly, that means we don't really have good coverage on the REST portion of the authentication. This also adds ElasticsearchStatusException, and exception on which you can set an explicit status. The nice thing about it is that you can set the RestStatus that it returns to whatever arbitrary status you like based on the status that comes back from the remote system. reindex-from-remote then uses it to wrap all remote failures, preserving the status from the remote Elasticsearch or whatever proxy is between us and the remove Elasticsearch.
2016-07-06 15:47:45 -04:00
Fix typos in comments/strings of `test` module.
2016-03-09 13:01:15 -05:00
// the main files are actually test files, so use the appropriate forbidden api sigs
Build: Add fixture capabilities to integ tests This change adds a Fixture class for use by gradle. A Fixture is an external process that integration tests will use. It can be added as a dependsOn for integTest, and will automatically be shutdown upon success or failure, as well as relevant information dumped on failure. There is also an example fixture in this change.
2015-12-15 23:14:37 -05:00
forbiddenApisMain {
Check that number of placeholders in log message matches number of parameters
2016-02-16 18:32:21 -05:00
signaturesURLs = [PrecommitTasks.getResource('/forbidden/jdk-signatures.txt'),
PrecommitTasks.getResource('/forbidden/es-signatures.txt'),
PrecommitTasks.getResource('/forbidden/es-test-signatures.txt')]
Build: Add fixture capabilities to integ tests This change adds a Fixture class for use by gradle. A Fixture is an external process that integration tests will use. It can be added as a dependsOn for integTest, and will automatically be shutdown upon success or failure, as well as relevant information dumped on failure. There is also an example fixture in this change.
2015-12-15 23:14:37 -05:00
}
// TODO: should we have licenses for our test deps?
dependencyLicenses.enabled = false
// TODO: why is the test framework pulled in...
forbiddenApisMain.enabled = false
jarHell.enabled = false
[CORE] Intorduce shards level locks to prevent concurrent shard modifications Today it's possible that the data directory for a single shard is used by more than on IndexShard->Store instances. While one shard is already closed but has a concurrent recovery running and a new shard is creating it's engine files can conflict and data can potentially be lost. We also remove shards data without checking if there are still users of the files or if files are still open which can cause pending writes / flushes or the delete operation to fail. If the latter is the case the index might be treated as a dangeling index and is brought back to life at a later point in time. This commit introduces a shard level lock that prevents modifications to the shard data while it's still in use. Locks are created per shard and maintined in NodeEnvironment.java. In contrast to most java concurrency primitives those locks are not reentrant. This commit also adds infrastructure that checks if all shard locks are released after tests.
2014-11-16 08:24:29 -05:00
}