Add-ons Operations Guide

When you release an add-on publicly on the Atlassian Marketplace, administrators of an Atlassian OnDemand application can install it in their instance, meaning they rely on your service to deliver content to their users. Therefore, ensuring consistent operation of your add-on is critical. You can avoid many potential service disruptions by planning carefully. There are some important considerations to take into account:

• Our customers are all around the world, covering all timezones.
• While some instances have a handful of users, others have thousands of users depending on our products to run their business.
• We have designed our cloud-based products to be both secure and reliable, boasting 99.9% uptime and 24x7x365 support.

Below, we discuss strategies for running your Atlassian Connect add-on as scalable, reliable software as a service. Some of these aspects need to be addressed very early in the design, as implementing them after the fact can be really difficult.

1. Defining your Service Level Agreement (SLA)

You should define your service level targets, which you can validate during performance testing, use as a basis to monitor your add-ons at runtime, and guarantee by scaling your deployment. The following table lists some examples of indicators you could track:

2. Managing the Performance of your Add-ons

Scalability

There are two ways to design your add-ons to scale with a growing number of installations and users:

• Vertical scaling: you scale by adding more resources (e.g. CPU, memory) to existing nodes
• Horizontal scaling: you scale by adding more nodes (e.g. servers)

It may be difficult to predict exactly the resources your add-ons will need. For this reason, and because your add-ons will operate in a cloud environment targeting thousands of customers, we encourage you to design your add-ons to scale horizontally.

Existing cloud providers can help you scale your implementations. One example of such providers is Heroku, a cloud application platform that can host applications developed in Java, Node.js, Pyton, Ruby, Scala or Clojure. Heroku leverages Amazon AWS (Amazon Web Services) technology, and mostly supports horizontal scaling. Other examples or world-class platforms include the Google Cloud Platform and Salesforce One.

Performance testing

We recommend you run performance tests for your add-ons. This will help you define the resources your add-ons require when you first deploy them, and understand how new versions of your add-ons impact resource utilization. The following classes of tests are particularly useful:

You should run performance tests for your add-ons:

• In isolation, using mock implementations of Atlassian products REST APIs. This helps identify any issues (memory leaks, etc.) limited to your implementation.
• Using a real-life deployment environment for end-to-end performance tests. For this we recommend you use a performance testing environment that is as close as possible to the production environment. You should set up instances of Atlassian OnDemand products for this purpose.

There are a number of tools to help you design and run performance tests for your add-ons. Examples of Load Testing Frameworks include The Grinder and Locust. They help you run distributed tests using many load injector machines.

Monitoring your SLA

You should have tools to monitor your add-on performance at runtime, and procedures in place to scale resources once specific thresholds are met. At a minimum, you should monitor utilization of resources by your add-ons (CPU, memory, disk space, etc.). When using a cloud provider, you can look at strategies to automatically scale the resources allocated to your add-ons based on load.

3. Maintaining your Add-ons

Versioning and Upgrading

We automatically detect updates to add-ons with a polling service. This way, you can easily release fixes and new features without having to manually create new version entries in the Marketplace. For more information on how to upgrade your add-ons and manage versions, you should read the Upgrading your Add-on section.

Maintenance Windows

Since your add-on and Atlassian products are decoupled, you can decide when to upgrade your add-ons independently from the Atlassian OnDemand Maintenance Windows. Ideally, your solution should be architected in a way that ensures maintenance is transparent to end-users. If this is not possible, make sure you publish your maintenance windows online, and provide a meaningful error message to users trying to access your add-ons at this time.

4. Addressing Business Continuity Planning

You should address the following aspects when looking at potential major outages:

• Data backups: you should have a data backup strategy that ensures your RPO (Recovery Point Objective) is met. For example, for a RPO of 24h, you should do a full backup of all add-on data overnight, keeping the backups on a different site to the one that is running the add-on.
• Recovery procedures: you should have procedures in place to restore your add-ons in the case of a major outage, and we suggest you do a few dry runs. Ideally, you should be testing your disaster recovery procedures regularly. Hope for the best, plan for the worst!

Note that using an world-class cloud provider minimizes the risk of a major outage impacting the users of your add-ons. For example when using Heroku with Heroku Postgres, the platform automatically backs up deployed applications and data, and automatically brings the application back online in case of a data center outage, with minimum data loss.

5. Providing Support

First, check out the Atlassian Support Offerings. We are well known for our great support! Here is what we recommend you focus on: