Understanding the Backup Processes

Starter and Business plans of Scalingo for OpenSearch® include automated and managed backups so you don’t have to worry about them.

We use Scheduled backups to create the automated ones. Manual backups are also available for these plans.

Both Scheduled and Manual backups capture a point-in-time state of one or more indexes, including settings, mappings, and data. They are stored in a remote object storage only accessible by Scalingo.

When dealing with backups, please keep in mind that OpenSearch®’s Snapshot APIs are not available.

Scheduled Backups

Creating a Scheduled Backup

Scheduled backups are automatically enabled and created by the platform when using a Starter or a Business plan.

Configuring Scheduled Backups

By default, Scheduled backups are done around 1:00 AM Central European Time (CET or UTC+0100). This time can be modified.

Using the Database Dashboard

1. From your web browser, open your database dashboard
2. Click the Backups tab
3. Locate the Backup schedule block
4. Click the Schedule button
5. Make sure to check the I want to enable scheduled backups checkbox
6. Pick an hour (timezone is UTC)
7. Validate by clicking the Update button

Using the Command Line

1. Make sure you have correctly setup the Scalingo command line tool
2. From the command line, configure the time of backup:

   scalingo --app my-app --addon opensearch backups-config --schedule-at "<hour> <timezone>"
   

   With hour being an integer between 0-23 (specifying minutes is currently not supported) and timezone being optional (default is UTC).
   In this example, we ask the platform to create the backup at ~06:00 Europe/Paris:

   scalingo --app my-app --addon opensearch backups-config --schedule-at "6 Europe/Paris"
   

   The output should look like this:

   -----> Periodic backups will be done daily at 6:00 CEST
   

Downloading a Scheduled Backup

Manual Backups

Creating a Manual Backup

Using the Database Dashboard

1. From your web browser, open your database dashboard
2. Click the Backups tab
3. Locate the Backups block
4. Click the Trigger manual backup button

Using the Command Line

1. Make sure you have correctly setup the Scalingo command line tool
2. Ask the platform to backup the database:

   scalingo --app my-app --addon opensearch backups-create
   

   After a while, the output should look like this:

   -----> Backup successfully finished
   

Downloading a Manual Backup

Dumping Indexes

OpenSearch® does not provide any official tool to dump the content of a database. However, the open source community has created very useful tools to fill this gap. The elasticsearch-dump project provides two of them: elasticdump and multielasticdump. We usually advise to rely on them to dump indexes from an OpenSearch® database to a the workstation local storage or to another OpenSearch® database.

For more comprehensive help about these tools, please refer to the official documentation.

From Your Workstation

1. Make sure you have successfully installed elasticsearch-dump on your workstation
2. Open a DB tunnel so you can access your database from your workstation
3. Create a local variable to store the local connection URI:

   export SCALINGO_DB_URL="http://<user>:<password>@127.0.0.1:<port>"
   

   With user and password from your original connection URI and port depending on what you did (default is 10000)

4. Run the following commands to create a dump of a single index, including analyzers, mappings, and data:

   # Dump analyzers:
   elasticdump \
       --input="${SCALINGO_DB_URL}/<index>" \
       --output=<index>-analyzers.json \
       --type=analyzer
   
   # Dump mappings:
   elasticdump \
       --input="${SCALINGO_DB_URL}/<index>" \
       --output=<index>-mappings.json \
       --type=mapping
   
   # Dump data:
   elasticdump \
       --input="${SCALINGO_DB_URL}/<index>" \
       --output=<index>-data.json \
       --type=data
   

   With index being the name of the index you want to dump.

   Please adjust the above commands to fit your needs. For instance, you may be interested in dumping only the data, or in dumping the settings, aliases or templates of the index (please refer to the official documentation to get an exhaustive list of values to use with the --type flag).

   It’s also possible to dump an index directly to another accessible OpenSearch® database. To do so, set the --output flag value to the connection URI of the destination OpenSearch® database, like so:

   elasticdump \
       --input="${SCALINGO_DB_URL}/<index>" \
       --output=<destination_db_uri>/<index> \
       --type=data
   

   With index being the name of the index to dump and destination_db_uri being the connection URI of the destination OpenSearch® database.

5. The elasticsearch-dump project also provides a convenient tool called multielasticdump to create a dump of multiple indexes at once:

   multielasticdump \
       --direction=dump\
       --input="${SCALINGO_DB_URL}" \
       --output=<dump_dir> \
       --match=<index_regexp> \
       --includeType=<types>
   

   With dump_dir being the destination directory, where all files are stored, index_regexp being a regular expression to filter which indexes should be dumped and types being a comma-separated list of types to dump.