In your daily work, you might received the request of setting up multi-region Elasticsearch cluster that serves content globally, the cluster have to meet these requirements:
Data Consistency.
High Availability.
Privacy & Security.
Single Endpoint & Easy to use.
Please check below recommended solution.
TL;DR:
Setup VPC & VPC Peering
Setup AWS EC2 & Elasticsearch Installation
Implement ALB for every regions
Configure Route53, Health Check and Geo Location
Step 1: Setup VPC & VPC Peering
Setup VPC
First of all, we need to setup VPC with non-overlapping CIDR as on diagram:
Region us-east-1: 172.16.1.0/24
Region eu-central-1: 172.17.1.0/24
Region ap-southeast-1: 172.18.1.0/24
CIDR have to be non-overlap due to set up VPC Peering and Routing Table
Initiate VPC Peering between the regions, which make mesh network.
IMPORTANT: The Elasticsearch cluster need mesh connections between nodes due to failover & synchronization seemlessly.
Setup VPC Peering
Follow this instruction from official AWS.
There are 3 VPC Peering Connections we have to setup is:
Connection 1: between us-east-1 and eu-central-1
Connection 2: between us-east-1 and ap-southeast-1
Connection 3: between eu-central-1 and ap-southeast-1
If you implement Elasticsearch cluster on 4 regions, you must create 6 VPC Peering Connections. Such as: 4 regions contain us-east-1, eu-central-1, ap-southeast-1, sa-east-1
Connection 1: between us-east-1 and eu-central-1
Connection 2: between us-east-1 and ap-southeast-1
Connection 3: between eu-central-1 and ap-southeast-1
Connection 4: between us-east-1 and sa-east-1
Connection 5: between eu-central-1 and sa-east-1
Connection 6: between ap-souteast-1 and sa-east-1
The fomular is:
Number of VPC Peering Connection Needed = <number_of_regions> Combinations of 2
After this step, please make sure all VPC Peering work properly 100%.
Step 2: Setup AWS EC2 & Elasticsearch Installation
Setup AWS EC2
Follow this instruction from official AWS.
Create these rules in your security group that allow Elasticsearch services running within EC2 can communicate each others for data transfer & synchronization.
Allow from us-east-1 VPC CIDR 172.16.1.0/24 access to port 9200 and 9300 on EC2.
Allow from eu-central-1 VPC CIDR 172.17.1.0/24 acess to port 9200 and 9300 on EC2.
Allow from ap-southeast-1 VPC CIDR 172.18.1.0/24 acess to port 9200 and 9300 on EC2.
Allow from any IP from local subnet of launched EC2 for Application Load Balancer to access to port 9200 on EC2.
Elasticsearch Installation
Before setting up Elasticsearch cluster, we should calculate how many master eligible nodes for cluster, the common fomular is: (N/2)+1
Elasticsearch cluster election process based on quorum-based, so we need +1 to create enough votes due to quick decision in election.
Let see the sample elasticsearch.yml configuration file:
[...]
cluster.name: sample_cluster
network.host: <node_ip>
node.roles: [master, data, voting_only] # it's up to you
[...]
discovery.seed_hosts: ["172.16.1.5:9300", "172.16.1.6:9300", "172.17.1.4:9300", "172.17.1.5:9300", "172.18.1.4:9300", "172.18.1.5:9300"]
cluster.initial_master_nodes: ["172.16.1.5:9300", "172.16.1.6:9300", "172.17.1.4:9300", "172.17.1.5:9300", "172.18.1.4:9300", "172.18.1.5:9300"]Step 3: Implement ALB for every regions
Follow this instruction from official AWS.
Create AWS ALB as above instruction with target is Elasticsearch nodes for every region, you will get 3 dns names, like this:
us-east-1: ALB-US-ES-1056884759.us-east-1.elb.amazonaws.com
us-central-1: ALB-EU-ES-1056884759.us-east-1.elb.amazonaws.com
ap-southeast-1: ALB-AP-ES-1056884759.us-east-1.elb.amazonaws.com
Step 4: Configure Route53, heath check & geolocation
Configure Route53 records
Configure health check for Route53, follow this instruction with these steps:
Create an Cloudwatch alarm that fires when healthy targets of ALB below two.
Configure Route53 health check based on Cloudwatch Alarm.
Now configure Route53 for your hosted zone with following configuration:
us-east-1:
Record type: CNAME
Record name: us.es.airwallet365.com
Routing policy: Failover
Health check: Cloudwatch Alarm US Elasticsearch
eu-central-1:
Record type: CNAME
Record name: eu.es.airwallet365.com
Routing policy: Failover
Health check: Cloudwatch Alarm EU Elasticsearch
ap-southeast-1:
Record type: CNAME
Record name: ap.es.airwallet365.com
Routing policy: Failover
Health check: Cloudwatch Alarm AP Elasticsearch
Now you have 3 endpoints in 3 regions. After having 3 endpoints, configure Route53 geolocation with health check, we recommend as below:
Record type: CNAME
Alias: enabled
Routing Policy: Geolocation
Location: Default, North America, South America
Alias: us.es.airwallet365.com
Location: Europe, Africa
Alias: eu.es.airwallet365.com
Location: Asia, China, Japan, Oceania, Hong Kong
Alias: ap.es.airwallet365.com
You can modify DNS records anytime you want.
Comments