DevOPS Learn and Share

Elasticsearch Note for cluster State: Red state: There is no primary shard in the cluster for indexing ,writing and searching Yellow State: There is primary shard for indexing and searching but not replica shards available of primary shards. Green state: Primary shard and replica shard are in good state for indexing and searching. Question:What happen when we increase the number of replicas: curl -XPUT 'localhost:9200/testing/_settings?pretty' -H 'Content-Type: application/json' -d' { "index" : {    "number_of_replicas" : 2 } } '               As you can see above the replica 2’s for primary shards will be unassigned as we have 2 data node cluster so let’s increase the data node in our setup. So from our conclusion we can see that by increasing the number of replicas in our case we can handle 2 node failur...

Elasticsearch Data node setup : Data node are the nodes/instance where our index data will reside.There is two things to consider for data nodes while creating an index i.e 1.Number of shards: How to divide our index data into number of shards so that data can be retrieved in fast paced manner. It help in sharding data to different nodes so that we can scale it well. 2.Number of Replicas: Number of Copies of primary shards are called number of replicas Let’s take an example to understand it’s concept : How many node failure can be possible if we have 3 node data cluster with 5 shards and 1 replica. Elasticsearch notes: Primary shards and its replica will not be on same node. If we delete one node from our cluster then : Our cluster will re-balance our lost replicas and move to existing nodes ,for time-being the cluster becomes yellow as you can see below for unassigned shards. Once rebalancing is completed our cluster back to green state...

Elasticsearch is an open source ,distributable,scalable and enterprise grade search engine. It can powers up your web-app search mechanism with n-number of queries executing and fetching results for your end user within milliseconds. Use case of Elasticsearch: 1.Full Text search 2.Log Analysis in ELK stack 3.Scraping and Combining Public Data 4.Event Data and Metrics(Time series is good option) Will Start with Discussion for Elasticsearch Master node Setup : We will be creating a three master node cluster in order to keep quorum so that there will not be any split brain problem. Split brain problem usually occurs in case of two master nodes cluster setup and one node goes down then second node will treat itself to be a candidate for master node form a cluster and suddenly first node comes up and itself treat as a master and form a new es cluster.So two nodes for two cluster which will not be rejoin until one is restarted again. The above pr...