Part 1 Series of Elasticsearch Cluster on Spot Instances with zero downtime in App?

Elasticsearch is an open source ,distributable,scalable and enterprise grade search engine.

It can powers up your web-app search mechanism with n-number of queries executing and
fetching
results for your end user within milliseconds.

Use case of Elasticsearch:

1.Full Text search

2.Log Analysis in ELK stack

3.Scraping and Combining Public Data

4.Event Data and Metrics(Time series is good option)

Will Start with Discussion for Elasticsearch Master node Setup:

We will be creating a three master node cluster in order to keep quorum so that there will not be any
split brain problem.

Split brain problem usually occurs in case of two master nodes cluster setup and one node goes
down then second node will treat itself to be a candidate for master node form a cluster and
suddenly first node comes up and itself treat as a master and form a new es cluster.So two nodes
for two cluster which will not be rejoin until one is restarted again.

The above problem can be resolved if we set minimum of number of master nodes by following
the below rules:

(Number of master nodes)/2+1=(3/2)+1=2 so there will be at-least 2 candidate for checking as
if which will be right candidate to  be selected as master when one node goes down.

Note:

We have to make sure elasticsearch master should be launched as on-demand instances as
they can be launched with t2.small instances and price is low.

elasticsearch.yml for master node1:

cluster.name: my-application-1

node.name: nb-master-es1

path.data: /data/elasticsearch-1-2

path.logs: /var/log/elasticsearch

http.port: 9200

node.master: true

node.data: false

cluster.routing.allocation.awareness.attributes: rack_id

discovery.zen.ping.unicast.hosts: ["nb-master-es1.testing.com",
"nb-master-es2.testing.com",”nb-master-es3.testing.com”]

Similarly all three master node will have same elasticsearch.yml and 3 master
node setup will be good to go.

Further this blog will be continued with more understanding of Data nodes,
Replica,Primary shards in upcoming blog Series..