If you have ever worked on a big project with RDBS as a data store then managing database changes is a huge task if you are not using the right set of tools. Changing the state of the database should be tracked in the revision system and it makes it easy to migrate from one state of the DB to another. It generally also helps if you want to spin up the new environment from the same codebase. I basically worked a lot with RDBMS and their DB Migration tools are a must to work with. I mainly worked with Alembic and DB-migrate and they both are super awesome.

Generally, it’s a human tendency that whenever you start something different you always try to compare it with what you were using before. I recently started working on Elasticsearch to manage big text-based data and while developing I got into a situation where I couldn’t find any Migration tool for elasticsearch or you can say couldn’t find any popular or stable tool for it. In the RDBMS world, there is plenty to use but here it was not the case. So to solve this issue, I tried to create a library that is basically inspired by Alembic.

Chalan

Chalan is a migration tool designed and developed for Elasticsearch and inspired by Alembic. With Chalan, you can create revisions or migration steps that will be used to upgrade or downgrade from a certain state to another state.

Installation

This tool is created by a python developer so basically, we love our pip 😁. So you can use pip to install the tool for now

pip install chalan

Initialization

To set up the migration folder and some basic configuration, you have to initialize the Chalan tool first using the below command

chalan init

It will create some files and folders to maintain all the revisions files into them and chalan.ini file to maintain the configurations

my-app/
├─ chalan.ini
├─ es_migrations/
│  ├─ versions/
├─ README.md

Here versions/ is the folder that will be having all the revision files

Before continuing you have to change the configuration in chalan.ini according to your configs.

The config file looks like this

directory is basically the name of the folder where all the revisions will be stored
es_host is the hostname of the elasticsearch
es_user is the username of the elasticsearch
es_pass is the password of the elasticsearch
migration_index is the index used internally to store the current state of the migration in Elasticsearch

Create First Revision

Let’s create the first revision using Chalan by simply creating a random index in elasticsearch. For that you have to use chalan revision command

chalan revision -m"first_index_creation"

This will create one revision file under es_migrations/versions

Additionally, it will create an index in elasticsearch to maintain the revisions also

Let’s checkout the file that is autogenerated by the above command. The fille look something like this

It has the revision number, down_revision that points to the previous version and in this case its None because it’s the first revision.

It has two functions,upgrade and downgrade that is responsible to migrate from one state to another. Both functions are having the es object that you can use to write some commands to elasticsearch to change the state.

Let’s change some states for this revision. We are now going to create some random index now.

So with this change, we are creating first_index and in downgrade we are deleting the index to come back to the same old state.

Upgrade

Command used to upgrade the migration level to head(current level). Or you can also specify a specific version to which you want to upgrade the version.

chalan upgrade
# or
chalan upgrade <specific version>

It will upgrade the revision to version —>1 and will create first_index in elasticsearch let’s crosscheck the same.

First index is created in Elasticsearch let’s crosscheck chalan_versions also to check the current revision of the tool.

You can the version is upgraded to 1.

Downgrade

Command used to downgrade the migration level to base(initial version). Or specify the level you want to downgrade to.

chalan downgrade # Downgrade 1 level down
# or
chalan downgrade --version base # Downgrade to base version

This will change the state back to the initial state so there will be no first_index and the version in the migration table will also become None. Let’s crosscheck

Conclusion

Please let me know in the comments or in GitHub if you face any issues or if you want some new features in the tool. And also please give the GitHub repo a Star it will motivate me to contribute more to the tool