Part 1 Series of Elasticsearch Cluster on Spot Instances with zero downtime in App?


Elasticsearch is an open source ,distributable,scalable and enterprise grade search engine.
It can powers up your web-app search mechanism with n-number of queries executing and
fetching
results for your end user within milliseconds.
Use case of Elasticsearch:
1.Full Text search
2.Log Analysis in ELK stack
3.Scraping and Combining Public Data
4.Event Data and Metrics(Time series is good option)
Will Start with Discussion for Elasticsearch Master node Setup:
We will be creating a three master node cluster in order to keep quorum so that there will not be any
split brain problem.
Split brain problem usually occurs in case of two master nodes cluster setup and one node goes
down then second node will treat itself to be a candidate for master node form a cluster and
suddenly first node comes up and itself treat as a master and form a new es cluster.So two nodes
for two cluster which will not be rejoin until one is restarted again.

The above problem can be resolved if we set minimum of number of master nodes by following
the below rules:
(Number of master nodes)/2+1=(3/2)+1=2 so there will be at-least 2 candidate for checking as
if which will be right candidate to  be selected as master when one node goes down.
Note:
We have to make sure elasticsearch master should be launched as on-demand instances as
they can be launched with t2.small instances and price is low.
elasticsearch.yml for master node1:
cluster.name: my-application-1
node.name: nb-master-es1
path.data: /data/elasticsearch-1-2
path.logs: /var/log/elasticsearch
http.port: 9200
node.master: true
node.data: false
cluster.routing.allocation.awareness.attributes: rack_id
discovery.zen.ping.unicast.hosts: ["nb-master-es1.testing.com",
"nb-master-es2.testing.com",”nb-master-es3.testing.com”]

Similarly all three master node will have same elasticsearch.yml and 3 master
node setup will be good to go.

Further this blog will be continued with more understanding of Data nodes,
Replica,Primary shards in upcoming blog Series..

Comments

Post a Comment

Popular posts from this blog

What is a Devops ??

What is Devops and its culture to adopt? Devops is simply a means of collaboration between developers and operations team working together over a common goal of making a commit change or complete source code live with just click of an instance i.e How operations team with the help of their coding skill sets automate their infrastructure in an cost efficient and effective manner in order to save time to market the product in fastest possible way. Devops Best Practices:(AWS Specific)    1.Do not stick with any single tool or code for solving a problem whether it can be Backup and restore or monitoring,cloud infrastructure setup ,first solve the problem then provide the best way of doing it with a purpose like for monitoring we can use any tool like nagios or zabbix but both tools has its own advantages and disadvantages so before moving with any tools it better to understand its pros and cons. 2.Always Keep your App server,DB server in private subnet so no public acces...
Read more

Part 3 Series of Elasticsearch Cluster on Spot Instances with zero downtime in App?

Image
Elasticsearch Note for cluster State: Red state: There is no primary shard in the cluster for indexing ,writing and searching Yellow State: There is primary shard for indexing and searching but not replica shards available of primary shards. Green state: Primary shard and replica shard are in good state for indexing and searching. Question:What happen when we increase the number of replicas: curl -XPUT 'localhost:9200/testing/_settings?pretty' -H 'Content-Type: application/json' -d' { "index" : {    "number_of_replicas" : 2 } } '               As you can see above the replica 2’s for primary shards will be unassigned as we have 2 data node cluster so let’s increase the data node in our setup. So from our conclusion we can see that by increasing the number of replicas in our case we can handle 2 node failur...
Read more