Octopus Deploy High Availability is our offering for customers where Octopus Deploy has become a critical piece of infrastructure. In addition to the benefits of extra reliability, High Availability provides performance gains over a single Octopus Server. Our aim was to provide a linear performance increase as you add nodes to your High Availability cluster. Enter Tentacle Army.
We tested High Availability on Amazon EC2 with the following configuration:
- Octopus Servers on m3.2xlarge (8 vCPU, 30GB memory)
- Octopus Tentacles on m3.large (2 vCPU, 7.5GB memory)
- SQL Server 2014 Express on m3.large
- Samba file share on m3.large
- Elastic Load Balancer pinging /api
We queued 150 deployments of packages ranging from 10MB to 1000MB, some with scripts and some with artifacts. Total size of deployed packages was just over 50GB with a lot of delta compression being computed to stress the CPU and file share.
The results:
A single Octopus Server completed all of the deployments in 35 minutes, compared to 18 minutes for two Octopus Servers and 11 minutes for three Octopus Servers. Two Octopus Servers are twice as fast as one and three servers are three times faster than one. That is the linear performance increase we were hoping to achieve.
Bottlenecks
While conducting these tests we ran into some bottlenecks that impact the speed at which the cluster could perform deployments. Here are some of the bottlenecks that we identified:
- Using an external NuGet feed has a big impact on performance as all Octopus Servers in the node acquired packages from the feed at once. Using the Octopus Server built-in package repository removed this bottleneck.
- Having multiple servers deploy to the same Tentacle can limit speed. In these tests there was no Tentacle overlap.
- The limit on concurrent running tasks per Octopus Server node can impact performance. We used the default value of 5.
We have some final testing and work to do, and then we'll be making High Availability available for trial in a few weeks. We hope that you will enjoy it!
