Migration API in Octopus

Octopus includes a migration API that provides the ability to back-up and restore parts of an Octopus Deploy instance remotely.

The API currently includes support for both the partial-export and import commands. The API uses the same Migrator.exe command line tool that you’d typically use to migrate data manually, but the API gives you some additional parameters to orchestrate the process between remote servers.

How it works

When you trigger a migration via the API, your Octopus Server will queue up a migration task that you can view from your Tasks screen. During execution of this task, your Octopus Server will go into maintenance mode to try and minimize any data mutations during the migration. When the task is completed, it will be taken out of maintenance mode.

The typical process for migrating projects between a source and destination server is as follows:

1. Create an API key for your source server (the server you’re exporting from).
2. Create an API key for your destination server (the server you’re importing to).
3. Get a list of project names that you wish to export from your source server.
4. Call the partial-export migration API against your source server, telling it the destination server URL, API key, password for your migration package, and the list of project names you want to export (You’ll receive a 200 response from the API telling you the TaskId that has been queued to do the actual work).
5. At this point, your source server’s task queue will then execute the partial-export command using Migrator.exe, package up the contents of your export and push it to your destination server’s package feed.
6. Watch your source server’s migration task in the Octopus UI to know when this operation is complete … Or if you’re really keen, you could write a script that queries the task API and it will let you know when the migration task is complete (as seen in the Octopus.Clients example below).
7. Call the import migration API against your destination server, telling it the package and password to import from (You’ll receive a 200 response from the API telling you the TaskId that has been queued to do the actual work).
8. Your destination server’s task queue will then execute an import command using Migrator.exe.
9. At this point, your destination server’s task queue will then execute the import command using Migrator.exe.

Partial export API

Using the partial-export API, we can export one or more of our projects and choose to send the package to the destination Octopus Server’s package feed.

Partial Export API parameters:

Import API

The import API lets you import a migration package from your Octopus Server’s built-in package feed (which is where packages are pushed to when using the partial-export API).

Import API parameters:

Examples

Raw request

You can trigger a request however you prefer, using curl, Fiddler, or your tool of choice…

Partial export

Request Method: POST
Request URL: https://YOUR_SOURCE_OCTOPUS_SERVER/api/migrations/partialexport
Request Headers:
- Content-Type: application/json
- X-Octopus-ApiKey: API-YOUR-SOURCE-KEY
Request Body:
{
    "PackageId": "MyAwesomeOctopusMigration",
    "PackageVersion": "1.0.0",
    "Password": "Demo1234",
    "Projects": ["First Project", "Second Project"],
    "EncryptPackage": true,
    "IncludeTaskLogs": true,
    "DestinationApiKey": "API-YOUR-DESTINATION-KEY",
    "DestinationPackageFeed": "https://YOUR_DESTINATION_OCTOPUS_SERVER"
}

Import

Request Method: POST
Request URL: https://YOUR_DESTINATION_OCTOPUS_SERVER/api/migrations/import
Request Headers:
- Content-Type: application/json
- X-Octopus-ApiKey: API-YOUR-DESTINATION-KEY
Request Body:
{
    "PackageId": "MyAwesomeOctopusMigration",
    "PackageVersion": "1.0.0",
    "Password": "Demo1234",
    "IsDryRun": "true", // Only set this to false when you've reviewed the dry run and are happy to proceed with the migration for real.
    "IsEncryptedPackage": true,
}

Spaces

If you are using the Spaces feature of Octopus Deploy on the source server, you should supply the appropriate SpaceID values in the example below. The destination Space cannot be specified; it will match the Source. There are two values to supply:

1. The location to look for the Projects in the source Space.
2. The Space that has the feed we’ll be pushing the exported package to.

If you are not using the Spaces feature, you do not need to supply the SpaceID values.

Octopus.Clients example

The Octopus.Clients library can also help you run a migration.

Here’s an example showing you how that might look, performing a partial-export from a source server and sending it to a destination server, then automatically running the associated import on the destination server:

Add-Type -Path 'YOUR_LOCAL_PATH\Octopus.Client.dll'

$sourceOctopusURI = 'https://SOURCE_OCTOPUS_SERVER'
$sourceApikey = 'API-YOUR-SOURCE-KEY'
$destinationOctopusURI = 'https://DESTINATION_OCTOPUS_SERVER'
$destinationApikey = 'API-YOUR-DESTINATION-KEY'

# Spaces related
$sourceSpaceId = 'Spaces-1'
$destinationPackageFeedSpaceId = 'Spaces-1'

$migrationPackageId = 'MyAwesomeOctopusMigration'
$migrationPackageVersion = '1.0.0'
$migrationPassword = 'Demo1234'
$isDryRun = $true # Only set this to false when you've reviewed the dry run and are happy to proceed with the migration for real.

$sourceEndpoint = New-Object Octopus.Client.OctopusServerEndpoint $sourceOctopusURI,$sourceApikey
$sourceRepository = New-Object Octopus.Client.OctopusRepository $sourceEndpoint

$migrationExportResource = new-object Octopus.Client.Model.Migrations.MigrationPartialExportResource
$migrationExportResource.PackageId = $migrationPackageId
$migrationExportResource.PackageVersion = $migrationPackageVersion
$migrationExportResource.Password = $migrationPassword
$migrationExportResource.Projects = @('First Project', 'Second Project')
$migrationExportResource.IgnoreCertificates = $false
$migrationExportResource.IgnoreMachines = $false
$migrationExportResource.IgnoreDeployments = $false
$migrationExportResource.IgnoreTenants = $false
$migrationExportResource.IncludeTaskLogs = $true
$migrationExportResource.EncryptPackage = $true
$migrationExportResource.DestinationPackageFeed = $destinationOctopusURI
$migrationExportResource.DestinationApiKey = $destinationApikey

$migrationExportResource.SpaceId = $sourceSpaceId
$migrationExportResource.DestinationPackageFeedSpaceId = $destinationPackageFeedSpaceId


$migrationExportResource = $sourceRepository.Migrations.PartialExport($migrationExportResource)

Write-Host("Export task queued: $($migrationExportResource.TaskId)")
$migrationExportTask = $sourceRepository.Tasks.Get($migrationExportResource.TaskId);
if ($migrationExportTask -eq $null) {
    Write-Host ("Export failed. You'll need to investigate.")
    Exit
}

# Now we can poll this migration task to know when our export is complete #ScriptingFTW
Write-Host("Export task: $($migrationExportTask.Id)")
while($migrationExportTask -and ($migrationExportTask.State -eq 'Queued' -or $migrationExportTask.State -eq 'Executing' -or $migrationExportTask.State -eq 'Cancelling')){
    Write-Host("Export task status: $($migrationExportTask.State). Re-checking in 5 seconds...")
    start-sleep -s 5
    $migrationExportTask = $sourceRepository.Tasks.Get($migrationExportTask.Id);
}
Write-Host("Export task completed: $($migrationExportTask.State)")

if ($migrationExportTask.State -ne 'Success' -or $migrationExportTask.HasWarningsOrErrors -eq $true) {
    Write-Host ("Export failed or has warnings/errors. You'll need to investigate.")
    Exit
}

# From here, we can proceed with an import on our destination server.
$destinationEndpoint = New-Object Octopus.Client.OctopusServerEndpoint $destinationOctopusURI,$destinationApikey
$destinationRepository = New-Object Octopus.Client.OctopusRepository $destinationEndpoint

$migrationImportResource = new-object Octopus.Client.Model.Migrations.MigrationImportResource
$migrationImportResource.PackageId = $migrationPackageId
$migrationImportResource.PackageVersion = $migrationPackageVersion
$migrationImportResource.Password = $migrationPassword
$migrationImportResource.IsDryRun = $isDryRun
$migrationImportResource.IsEncryptedPackage = $true
$migrationImportResource.DeletePackageOnCompletion = $true # May as well clean up after ourselves.

$migrationImportResource = $destinationRepository.Migrations.Import($migrationImportResource)

Write-Host("Import task queued: $($migrationExportResource.TaskId)")
$migrationImportTask = $destinationRepository.Tasks.Get($migrationImportResource.TaskId);
if ($migrationImportTask -eq $null) {
    Write-Host ("Import failed. You'll need to investigate.")
    Exit
}

# Now we can poll this migration task to know when our import is complete.
Write-Host("Import task: $($migrationImportTask.Id)")
while($migrationImportTask -and ($migrationImportTask.State -eq 'Queued' -or $migrationImportTask.State -eq 'Executing' -or $migrationImportTask.State -eq 'Cancelling')){
    Write-Host("Import task status: $($migrationImportTask.State). Re-checking in 5 seconds...")
    start-sleep -s 5
    $migrationImportTask = $destinationRepository.Tasks.Get($migrationImportTask.Id);
}
Write-Host("Import task completed: $($migrationImportTask.State)")

if ($migrationImportTask.State -ne 'Success' -or $migrationImportTask.HasWarningsOrErrors -eq $true) {
    Write-Host ("Import failed or has warnings/errors. You'll need to investigate.")
    Exit
}

Write-Host ("Migration complete, #GreatSuccess")

Troubleshooting

We do our best to log information and warnings to your task logs during a migration. An API migration follows the same path as a manual migration using Migrator.exe command line tools behind the scenes, so if you are having difficulty running migrations, be sure to check your task logs for information that might help.

Help us continuously improve

Please let us know if you have any feedback about this page.

Send feedback

Page updated on Tuesday, June 25, 2024