From MySQL To NoSQL! How To Migrate Your MySQL Data To MongoDB Using Mongify Utility

http://linuxpitstop.com/migrate-mysql-to-mongodb-using-mongify-utility-linux

Welcome again. Big data is here and therefore there needs to be a solution to store such kind of data in a database which is independent of the boundaries of normalization and relationships. RDBMS is no longer a great solution for storing big data. And that is why noSQL databases are now needed everywhere. Today, I am going to explain how the mongify utility can be used to migrate a database from MySQL to MongoDB. But before we jump into it, let me share with you little background information:

Introduction to MySQL

MySQL is an open source relational database management system (RDBMS) which uses the Structured Query Language (SQL) as a mechanism for dealing and interacting with the data. Although MySQL is one of the widely used and well known database management systems and is considered as reliable, scalable and efficient database management system, It is NOT well suited for handling big data and especially with HUGE insertion rates.

Introduction to MongoDB

MongoDB server is an opensource document database which stores data in JSON (which is a key:value) format. It has no db schemas filled with joins and relationships and is highly recommended as backend for web applications where huge volume of data is inserted and processed in real time.

When to Use MongoDB and When Not?

If you need a flexible database solution with no strict schema and expect a very high insert rate into your database also if reliability and security is of less concern for you then you can go for MongoDB. While on the other hand when security and reliability is of prime concern and you do not expect very huge write transactions into your database then you may use MySQL or any other RDBMS.

Introduction to Mongify

Mongify is a utility (or a ruby gem ) written in the ruby language and is used to migrate databases from SQL to mongodb. Further detailed information about ruby language and ruby gems can be found on their corresponding websites. Mongify utility migrates databases while not caring about primary keys and foreign keys as in case of an RDBMS. It supports data migration from MySQL, SQLite and other relational databases however this article only focuses on migrating data from MySQL to MongoDB.

Install Ruby if not already installed

As mentioned earlier, the mongify utility is based on ruby language therefore we need to install ruby if it is not already present on the sytem.
The following command can be used to install ruby on Ubuntu systems:

 apt-get install ruby

Below screen displays a typical output of this command:

Install ‘gem’ Package

Once ruby has been installed successfully, the next step is to install the ‘gem’ package which itself is the ruby gem manager. We will use the below command to achieve this:

apt-get install gem

The output for this command should be something list below:

Install Other Dependencies If Not Already Installed

Once these packages are installed, we need to complete a few more prerequisite packages to install and run mongify. These package dependencies are mentioned as below:

1. ruby-dev
2. mongodb
3. libmysqlclient-dev

Besides these packages there are a few ‘gems’ needed as run time dependencies. These runtime dependencies include (at least):

1. activerecord
2. activesupport
3. bson
4. bson_ext
5. highline
6. mongo

Once all these dependencies are met, we are good to go for installing the mongify gem.

Install ‘mongify’ gem

The below command can be used to install the mongify utility:

sudo gem install mongify

The output for this command may look like something below:

Create a database.config file

Next, we need to create a database configuration file. This configuration file will contain the details and credentials for MySQL database and the MongoDB. Here we need to make sure that the correct database name, username and password are used for the MySQL database that we need to migrate.
The contents of the database.config may look similar to as shown in the following screenshot:

Check if Database Config is Correct

Next, we can check if the newly created database.config file is correct. We can use below command:

 mongify check database.config

If everything is alright, the output for this command can be something like this:

Create a Database Translation File

Now if the configuration file is correct, we can proceed to the next step which is to create a translation file.
We will use the below command to create a translation file:

mongify translation database.config >> translation.rb

The output for this command should be something like below:

We are almost done! But wait, one more step is needed and that is the actual step which will migrate the database for us.

Process the Translation File

This will be the step which will process the translation file and will create a new database in Mongodb for us. We will use below command :

mongify process database.config translation.rb

And the output should be something like below:

Congratulations! We have successfully migrated our database named ‘cloud’ from MySQL to Mongodb. This can be confirmed within the mongo shell by running below command:

$ mongo
>> db.stats()

The output for this command should be something like this:

In the above screenshot the details about our newly migrated database are displayed. It contains the database name, total number of tables (collections) and other details.

Conclusion

In this article we demonstrated how can we use the mongify utility to migrate an existing MySQL database to MongoDB. If you like this article or if you have any queries regarding the procedure, you are most welcome to share your comments and feedback here. We will come back with a new topic soon. Happy reading!