21 Sep 2018

Backing up and restoring Elasticsearch data with Elasticdump

Elasticsearch is a type NoSQL database which allows you to store unstructured data. It is highly performant and scalable. It allows you to implement full text searches and aggregations. This is very useful when dealing with large amounts of data.

Elasticdump is a NodeJS based package. This allows you to backup and restore Elasticsearch data. Its command line interface is quite intuitive to use.

Installation

npm install -g elasticdump

Features

  • Backing up ES index

    elasticdump --input=http://elastic-search.mydomain.com:9200/my_index --output=/backups/my_index.json --limit=1000 --fileSize=1gb
    
  • Restoring ES index from a file

    elasticdump --input=/backups/my_index.json --output=http://elastic-search.mydomain.com:9200/my_index --limit=1000
    

Issues

Data too big

Under the hood, elasticdump uses JSON.stringify to convert data objects into actual strings to be stored into a file but this causes some issues if the data is just too big. To overcome this, I have created a python implementation of elasticdump. This is still a work in progress, I would love if anyone can help in the project.


Tags: