Cloud Migration Tool
Migrating data from bare metal servers to the cloud with minimal downtime
arangosync-migration tool allows you to easily move from on-premises to
the cloud while ensuring a smooth transition with minimal downtime.
Start the cloud migration, let the tool do the job and, at the same time,
keep your local cluster up and running.
Some of the key benefits of the cloud migration tool include:
- Safety comes first - pre-checks and potential failures are carefully handled.
- Your data is secure and fully encrypted.
- Ease-of-use with a live migration while your local cluster is still in use.
- Get access to what a cloud-based fully managed service has to offer: high availability and reliability, elastic scalability, and much more.
arangosync-migration tool is available to download for the following
macOS / Darwin
For macOS as well as other Unix-based operating systems, run the following command to make sure you can execute the binary:
chmod 755 ./arangosync-migration
Before getting started, make sure the following prerequisites are in place:
Go to the ArangoGraph Insights Platform and sign in. If you don’t have an account yet, sign-up to create one.
Generate an ArangoGraph API key and API secret. See a detailed guide on how to create an API key.
Continue by creating a new ArangoGraph deployment or choose an existing one.
The target deployment in ArangoGraph requires specific configuration rules to be set up before the migration can start:
- Configuration settings: The target deployment must be compatible with the source data cluster. This includes the ArangoDB version that is being used, the DB-Servers count, and disk space.
- Deployment region and cloud provider: Choose the closest region to your data cluster. This factor can speed up your migration to the cloud.
After setting up your ArangoGraph deployment, wait for a few minutes for it to become fully operational.
arangosync-migration tool provides a set of commands that allow you to:
- start the migration process
- check whether your source and target clusters are fully compatible
- get the current status of the migration process
- stop or abort the migration process
- switch the local cluster to read-only mode
To start the migration process, run the following command:
start command runs some pre-checks. Among other things, it measures
the disk space which is occupied by your ArangoDB cluster. If you are using the
same data volume for ArangoDB servers and other data as well, the measurements
can be incorrect. Provide the
--source.ignore-metrics option to overcome this.
You also have the option of doing a
--check-only without starting the actual
migration. If specified, this checks if your local cluster and target deployment
are compatible without sending any data to ArangoGraph.
Once the migration starts, the local cluster enters into monitoring mode and the synchronization status is displayed in real-time. If you don’t want to see the status you can terminate this process, as the underlying agent process continues to work. If something goes wrong, restarting the same command restores the replication state.
To restart the migration, first
stop --abort the migration. Then,
start it again using the
The following takes place during an active migration:
- The source data cluster remains usable.
- The target deployment in ArangoGraph is switched to read-only mode.
- Your root user password is not copied to the target deployment in ArangoGraph. To get your root password, select the target deployment from the ArangoGraph Dashboard and go to the Overview tab. All other users are fully synchronized.
./arangosync-migration start \ --source.endpoint=$COORDINATOR_ENDPOINT \ --source.jwt-secret=/path-to/jwt-secret.file \ --arango-graph.api-key=$ARANGO_GRAPH_API_KEY \ --arango-graph.api-secret=$ARANGO_GRAPH_API_SECRET \ --arango-graph.deployment-id=$ARANGO_GRAPH_DEPLOYMENT_ID
The total time required to complete the migration depends on how much data you have and how often write operations are executed during the process.
You can also track the progress by checking the Migration status section of your target deployment in ArangoGraph dashboard.
To print the current status of the migration, run the following command:
./arangosync-migration status \ --arango-graph.api-key=$ARANGO_GRAPH_API_KEY \ --arango-graph.api-secret=$ARANGO_GRAPH_API_SECRET \ --arango-graph.deployment-id=$ARANGO_GRAPH_DEPLOYMENT_ID
You can also add the
--watch option to start monitoring the status in real-time.
arangosync-migration stop command stops the migration and terminates
the migration agent process.
If replication is running normally, the command waits until all shards are
in sync. The local cluster is then switched into read-only mode.
After all shards are in-sync and the migration stopped, the target deployment
is switched into the mode specified in
--source.server-mode option. If no
option is specified, it defaults to the read/write mode.
./arangosync-migration stop \ --arango-graph.api-key=$ARANGO_GRAPH_API_KEY \ --arango-graph.api-secret=$ARANGO_GRAPH_API_SECRET \ --arango-graph.deployment-id=$ARANGO_GRAPH_DEPLOYMENT_ID
--abort option is supported. If specified, the
will not check anymore if both deployments are in-sync and stops all
migration-related processes as soon as possible.
arangosync-migration set-server-mode command allows switching
for your local cluster on and off.
In a read-only mode, all write operations are going to fail with an error code
Creating or dropping databases and collections are also going to fail with
./arangosync-migration set-server-mode \ --source.endpoint=$COORDINATOR_ENDPOINT \ --source.jwt-secret=/path-to/jwt-secret.file \ --source.server-mode=readonly
--source.server-mode option allows you to specify the desired server mode.
Allowed values are
arangosync-migration tool supports the following environment variables:
Using these environment variables is highly recommended to ensure a secure way of providing sensitive data to the application.
When running the migration, ensure that your target deployment has the same (or bigger) amount of resources (CPU, RAM) than your cluster. Otherwise, the migration process might get stuck or require manual intervention. This is closely connected to the type of data you have and how it is distributed between shards and collections.
In general, the most important parameters are:
- Total number of leader shards
- The amount of data in bytes per collection
Both parameters can be retrieved from the ArangoDB Web Interface.
arangosync-migration tool supports migrating large datasets of up to
5 TB of data and 3800 leader shards, as well as collections as big as 250 GB.
In case you have any questions, please reach out to us .
Download and start the
arangosync-migrationtool. The target deployment is switched into read-only mode automatically.
Wait until all shards are in sync. You can use the
startcommand with the same parameters to track that.
Optionally, when all shards are in-sync, you can switch your applications to use the endpoint of the ArangoGraph deployment, but note that it stays in read-only mode until the migration process is fully completed.
Stop the migration using the
stopsubcommand. The following steps are executed:
If you switched the source data cluster into read-only mode, you can switch it back to default (read/write) mode using the
- The source data cluster is switched into read-only mode.
- It waits until all shards are synchronized.
- The target deployment is switched into default read/write mode.