If you have ever worked on a big project with RDBS as a data store then managing database changes is a huge task if you are not using the right set of tools. Changing the state of the database should be tracked in the revision system and it makes it easy to migrate from one state of the DB to another. It generally also helps if you want to spin up the new environment from the same codebase. I basically worked a lot with RDBMS and their DB Migration tools are a must to work with. I mainly worked with Alembic and DB-migrate and they both are super awesome.
Generally, it’s a human tendency that whenever you start something different you always try to compare it with what you were using before. I recently started working on Elasticsearch to manage big text-based data and while developing I got into a situation where I couldn’t find any Migration tool for elasticsearch or you can say couldn’t find any popular or stable tool for it. In the RDBMS world, there is plenty to use but here it was not the case. So to solve this issue, I tried to create a library that is basically inspired by Alembic.
Chalan
Chalan is a migration tool designed and developed for Elasticsearch and inspired by Alembic. With Chalan, you can create revisions or migration steps that will be used to upgrade or downgrade from a certain state to another state.
Installation
This tool is created by a python developer so basically, we love our pip 😁. So you can use pip to install the tool for now
pip install chalan
Initialization
To set up the migration folder and some basic configuration, you have to initialize the Chalan tool first using the below command
chalan init
It will create some files and folders to maintain all the revisions files into them and chalan.ini
file to maintain the configurations
my-app/
├─ chalan.ini
├─ es_migrations/
│ ├─ versions/
├─ README.md
Here versions/
is the folder that will be having all the revision files
Before continuing you have to change the configuration in chalan.ini according to your configs.
The config file looks like this
directory
is basically the name of the folder where all the revisions will be storedes_host
is the hostname of the elasticsearches_user
is the username of the elasticsearches_pass
is the password of the elasticsearchmigration_index
is the index used internally to store the current state of the migration in Elasticsearch
Create First Revision
Let’s create the first revision using Chalan by simply creating a random index in elasticsearch. For that you have to use chalan revision
command
chalan revision -m"first_index_creation"
This will create one revision file under es_migrations/versions
Additionally, it will create an index in elasticsearch to maintain the revisions also
Let’s checkout the file that is autogenerated by the above command. The fille look something like this
It has the revision number, down_revision
that points to the previous version and in this case its None
because it’s the first revision.
It has two functions,upgrade
and downgrade
that is responsible to migrate from one state to another. Both functions are having the es object that you can use to write some commands to elasticsearch to change the state.
Let’s change some states for this revision. We are now going to create some random index now.
So with this change, we are creating first_index
and in downgrade we are deleting the index to come back to the same old state.
Upgrade
Command used to upgrade the migration level to head(current level). Or you can also specify a specific version to which you want to upgrade the version.
chalan upgrade
# or
chalan upgrade <specific version>
It will upgrade the revision to version —>1 and will create first_index
in elasticsearch let’s crosscheck the same.
First index is created in Elasticsearch let’s crosscheck chalan_versions also to check the current revision of the tool.
You can the version is upgraded to 1.
Downgrade
Command used to downgrade the migration level to base(initial version). Or specify the level you want to downgrade to.
chalan downgrade # Downgrade 1 level down
# or
chalan downgrade --version base # Downgrade to base version
This will change the state back to the initial state so there will be no first_index
and the version in the migration table will also become None. Let’s crosscheck
Conclusion
Please let me know in the comments or in GitHub if you face any issues or if you want some new features in the tool. And also please give the GitHub repo a Star it will motivate me to contribute more to the tool