honeymoose
/
OpenSearch
mirror of https://github.com/honeymoose/OpenSearch.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
OpenSearch/qa/smoke-test-tribe-node/build.gradle

66 lines
2.9 KiB
Groovy
Raw Normal View History

Add socket permissions for tribe nodes Today when a node starts, we create dynamic socket permissions based on the configured HTTP ports and transport ports. If no ports are configured, we use the default port ranges. When a tribe node starts, a tribe node creates an internal node client for connecting to each remote cluster. If neither an explicit HTTP port nor transport ports were specified, the default port ranges are large enough for the tribe node and its internal node clients. If an explicit HTTP port or transport port was specified for the tribe node, then socket permissions for those ports will be created, but not for the internal node clients. Whether the internal node clients have explicit ports specified, or attempt to bind within the default range, socket permissions for these will not have been created and the internal node clients will hit a permissions issue when attempting to bind. This commit addresses this issue by also accounting for tribe nodes when creating the dynamic socket permissions. Additionally, we add our first real integration test for tribe nodes.
2016-11-13 17:27:18 -05:00
/*
* Licensed to Elasticsearch under one or more contributor
* license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright
* ownership. Elasticsearch licenses this file to you under
* the Apache License, Version 2.0 (the "License"); you may
* not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
import org.elasticsearch.gradle.test.ClusterConfiguration
import org.elasticsearch.gradle.test.ClusterFormationTasks
import org.elasticsearch.gradle.test.NodeInfo
Switch from standalone-test to standalone-rest-test standalone-rest-test doesn't configure unit tests and for these integ test only tests, that is what we want.
2017-01-04 14:26:55 -05:00
apply plugin: 'elasticsearch.standalone-rest-test'
Add socket permissions for tribe nodes Today when a node starts, we create dynamic socket permissions based on the configured HTTP ports and transport ports. If no ports are configured, we use the default port ranges. When a tribe node starts, a tribe node creates an internal node client for connecting to each remote cluster. If neither an explicit HTTP port nor transport ports were specified, the default port ranges are large enough for the tribe node and its internal node clients. If an explicit HTTP port or transport port was specified for the tribe node, then socket permissions for those ports will be created, but not for the internal node clients. Whether the internal node clients have explicit ports specified, or attempt to bind within the default range, socket permissions for these will not have been created and the internal node clients will hit a permissions issue when attempting to bind. This commit addresses this issue by also accounting for tribe nodes when creating the dynamic socket permissions. Additionally, we add our first real integration test for tribe nodes.
2016-11-13 17:27:18 -05:00
apply plugin: 'elasticsearch.rest-test'
Build: Rework integ test setup and shutdown to ensure stop runs when desired (#23304) Gradle's finalizedBy on tasks only ensures one task runs after another, but not immediately after. This is problematic for our integration tests since it allows multiple project's integ test clusters to be simultaneously. While this has not been a problem thus far (gradle 2.13 happened to keep the finalizedBy tasks close enough that no clusters were running in parallel), with gradle 3.3 the task graph generation has changed, and numerous clusters may be running simultaneously, causing memory pressure, and thus generally slower tests, or even failure if the system has a limited amount of memory (eg in a vagrant host). This commit reworks how integ tests are configured. It adds an `integTestCluster` extension to gradle which is equivalent to the current `integTest.cluster` and moves the rest test runner task to `integTestRunner`. The `integTest` task is then just a dummy task, which depends on the cluster runner task, as well as the cluster stop task. This means running `integTest` in one project will both run the rest tests, and shut down the cluster, before running `integTest` in another project.
2017-02-22 15:43:15 -05:00
ClusterConfiguration configOne = new ClusterConfiguration(project)
configOne.clusterName = 'one'
configOne.setting('node.name', 'one')
List<NodeInfo> oneNodes = ClusterFormationTasks.setup(project, 'clusterOne', integTestRunner, configOne)
Add socket permissions for tribe nodes Today when a node starts, we create dynamic socket permissions based on the configured HTTP ports and transport ports. If no ports are configured, we use the default port ranges. When a tribe node starts, a tribe node creates an internal node client for connecting to each remote cluster. If neither an explicit HTTP port nor transport ports were specified, the default port ranges are large enough for the tribe node and its internal node clients. If an explicit HTTP port or transport port was specified for the tribe node, then socket permissions for those ports will be created, but not for the internal node clients. Whether the internal node clients have explicit ports specified, or attempt to bind within the default range, socket permissions for these will not have been created and the internal node clients will hit a permissions issue when attempting to bind. This commit addresses this issue by also accounting for tribe nodes when creating the dynamic socket permissions. Additionally, we add our first real integration test for tribe nodes.
2016-11-13 17:27:18 -05:00
Build: Rework integ test setup and shutdown to ensure stop runs when desired (#23304) Gradle's finalizedBy on tasks only ensures one task runs after another, but not immediately after. This is problematic for our integration tests since it allows multiple project's integ test clusters to be simultaneously. While this has not been a problem thus far (gradle 2.13 happened to keep the finalizedBy tasks close enough that no clusters were running in parallel), with gradle 3.3 the task graph generation has changed, and numerous clusters may be running simultaneously, causing memory pressure, and thus generally slower tests, or even failure if the system has a limited amount of memory (eg in a vagrant host). This commit reworks how integ tests are configured. It adds an `integTestCluster` extension to gradle which is equivalent to the current `integTest.cluster` and moves the rest test runner task to `integTestRunner`. The `integTest` task is then just a dummy task, which depends on the cluster runner task, as well as the cluster stop task. This means running `integTest` in one project will both run the rest tests, and shut down the cluster, before running `integTest` in another project.
2017-02-22 15:43:15 -05:00
ClusterConfiguration configTwo = new ClusterConfiguration(project)
configTwo.clusterName = 'two'
configTwo.setting('node.name', 'two')
List<NodeInfo> twoNodes = ClusterFormationTasks.setup(project, 'clusterTwo', integTestRunner, configTwo)
Add socket permissions for tribe nodes Today when a node starts, we create dynamic socket permissions based on the configured HTTP ports and transport ports. If no ports are configured, we use the default port ranges. When a tribe node starts, a tribe node creates an internal node client for connecting to each remote cluster. If neither an explicit HTTP port nor transport ports were specified, the default port ranges are large enough for the tribe node and its internal node clients. If an explicit HTTP port or transport port was specified for the tribe node, then socket permissions for those ports will be created, but not for the internal node clients. Whether the internal node clients have explicit ports specified, or attempt to bind within the default range, socket permissions for these will not have been created and the internal node clients will hit a permissions issue when attempting to bind. This commit addresses this issue by also accounting for tribe nodes when creating the dynamic socket permissions. Additionally, we add our first real integration test for tribe nodes.
2016-11-13 17:27:18 -05:00
Build: Rework integ test setup and shutdown to ensure stop runs when desired (#23304) Gradle's finalizedBy on tasks only ensures one task runs after another, but not immediately after. This is problematic for our integration tests since it allows multiple project's integ test clusters to be simultaneously. While this has not been a problem thus far (gradle 2.13 happened to keep the finalizedBy tasks close enough that no clusters were running in parallel), with gradle 3.3 the task graph generation has changed, and numerous clusters may be running simultaneously, causing memory pressure, and thus generally slower tests, or even failure if the system has a limited amount of memory (eg in a vagrant host). This commit reworks how integ tests are configured. It adds an `integTestCluster` extension to gradle which is equivalent to the current `integTest.cluster` and moves the rest test runner task to `integTestRunner`. The `integTest` task is then just a dummy task, which depends on the cluster runner task, as well as the cluster stop task. This means running `integTest` in one project will both run the rest tests, and shut down the cluster, before running `integTest` in another project.
2017-02-22 15:43:15 -05:00
integTestCluster {
// tribe nodes had a bug where if explicit ports was specified for the tribe node, the dynamic socket permissions that were applied
// would not account for the fact that the internal node client needed to bind to sockets too; thus, we use explicit port ranges to
// ensure that the code that fixes this bug is exercised
setting 'http.port', '40200-40249'
setting 'transport.tcp.port', '40300-40349'
setting 'node.name', 'quest'
setting 'tribe.one.cluster.name', 'one'
setting 'tribe.one.discovery.zen.ping.unicast.hosts', "'${-> oneNodes.get(0).transportUri()}'"
setting 'tribe.one.http.enabled', 'true'
setting 'tribe.one.http.port', '40250-40299'
setting 'tribe.one.transport.tcp.port', '40350-40399'
setting 'tribe.two.cluster.name', 'two'
setting 'tribe.two.discovery.zen.ping.unicast.hosts', "'${-> twoNodes.get(0).transportUri()}'"
setting 'tribe.two.http.enabled', 'true'
setting 'tribe.two.http.port', '40250-40299'
setting 'tribe.two.transport.tcp.port', '40250-40399'
Test: Write node ports file before starting tribe service (#24351) The tribe service can take a while to initialize, depending on how many cluster it needs to connect to. This change moves writing the ports file used by tests to before the tribe service is started.
2017-04-27 03:59:54 -04:00
waitCondition = { node, ant ->
File tmpFile = new File(node.cwd, 'wait.success')
// 5 nodes: tribe + clusterOne (1 node + tribe internal node) + clusterTwo (1 node + tribe internal node)
ant.get(src: "http://${node.httpUri()}/_cluster/health?wait_for_nodes=>=5&wait_for_status=yellow",
dest: tmpFile.toString(),
ignoreerrors: true,
retries: 10)
return tmpFile.exists()
}
Add socket permissions for tribe nodes Today when a node starts, we create dynamic socket permissions based on the configured HTTP ports and transport ports. If no ports are configured, we use the default port ranges. When a tribe node starts, a tribe node creates an internal node client for connecting to each remote cluster. If neither an explicit HTTP port nor transport ports were specified, the default port ranges are large enough for the tribe node and its internal node clients. If an explicit HTTP port or transport port was specified for the tribe node, then socket permissions for those ports will be created, but not for the internal node clients. Whether the internal node clients have explicit ports specified, or attempt to bind within the default range, socket permissions for these will not have been created and the internal node clients will hit a permissions issue when attempting to bind. This commit addresses this issue by also accounting for tribe nodes when creating the dynamic socket permissions. Additionally, we add our first real integration test for tribe nodes.
2016-11-13 17:27:18 -05:00
}