Alerts and incident escalation

Prev Next

Overview

Alerts tell you when something in a tenant needs attention. Tenant Manager checks each alert on a schedule, raises an incident when the condition is met, and notifies people through an escalation policy until someone responds.

Tenant Manager has five alert types. You set them up per tenant from the Alerts page.

Business value

The five alert types cover money, capacity, and connection problems. Waste alerts show licenses that cost money without being used, capacity alerts warn you before seats or licenses run out, and a connection alert warns you before Tenant Manager loses access to a tenant.

Escalation policies make sure an incident reaches the right people in the right order, and move to the next level when nobody responds.

How it works

Tenant Manager evaluates alerts against the stored data from the last sync. It does not read Microsoft 365 when an alert runs. This section covers the alert types, the incidents they raise, and how escalation and notifications work.

Alert types

Each alert type watches for one condition. The types are grouped as Waste, Capacity, and Connection.

Alert type Category What it tells you
Disabled user with license Waste The account is disabled but still has licenses. Removing them affects no one, because the user cannot sign in.
Inactive user with license Waste A user has a license but has not used it. Users who have never signed in, meaning no sign-in of either kind (interactive or background), are included, so they do not need a separate alert.
License seat shortage Capacity A license has no seats left, or almost none. When no seats are available, the next new user cannot get one.
License expiry Capacity A license is close to its renewal date. When it lapses, everyone holding a seat on it loses the service at once.
Client secret expiry Connection The client secret that lets Turbo360 read this tenant is about to expire. If it expires, nothing is updated for that tenant, and no other alert works for it either.

Each alert type can be switched on or off for a tenant, and each has its own trigger, check frequency, and escalation policy. The trigger is a number that is set for the alert, such as a number of days or seats. An alert cannot be switched on until it has a notification channel or a recipient email address (recipient email addresses are not available on private hosting). For the details, see Alert types and thresholds.

An alert checks the data from the last sync, so checking it more often than the tenant syncs finds nothing new. License expiry and Client secret expiry are checked once a day at 00:00 in the Background activity time zone from General settings. For the other types, you choose the frequency. An alert with nothing to measure, such as a tenant with no paid licenses, is not applicable and never fires.

Each incident has a severity of Critical or Warning. For License seat shortage, an incident becomes Critical when available seats fall to the Error line that you set. For License expiry, an incident becomes Critical when a license purchase has lapsed.

Incidents

When a check finds the condition, Tenant Manager raises an incident. An incident is the record of one occurrence of the problem, and it has the status Open, Acknowledged, or Closed.

Event What happens
Raised Each due check that finds the condition raises its own incident, even when earlier incidents for the same alert are still open.
Acknowledged A person marks the incident as in hand. It stops escalating and is not sent again until it is reopened. It stays acknowledged until someone closes it. Acknowledging does not stop the next due check from raising a new incident.
Closed by a person The incident closes. If the condition is still met, the next check raises a new incident.
Condition clears When a check finds the condition no longer met, open incidents stay open and stop escalating. The log notes that the condition is no longer met. A person has to acknowledge or close them.
Reopened A person reopens an incident, and it is sent again on the alert's schedule.
Alert disabled Tenant Manager stops checking the alert and stops escalating. Open incidents stay open, and escalation resumes from the step it reached when you switch the alert on again.

Incidents are listed in each alert's Alert history. The list shows the newest 200 incidents for that alert, with filters for status and date. Each incident has a details view, a log of user actions with comments, and the notification details. For every alert type except Client secret expiry, you can download the affected records as an Excel file. For the details, see Managing alert incidents.

Anyone who can read alerts can view the history. Enabling, disabling, configuring, acknowledging, closing, and reopening need Manage on Alerts.

Escalation and notifications

An escalation policy decides who is notified, in what order, and after what delay. Policies are account level, so one policy can serve alerts on many tenants. They are managed under Escalation policies. For the details, see Managing escalation policies. A Default Escalation Policy exists, and you can view it but not edit or delete it.

A policy has up to five escalation rules. Each rule names the notification channels to notify and, on SaaS, the recipient email addresses, an email template, and an email subject. On private hosting, a rule names notification channels only. Each rule except the last (and the last too when the policy repeats) has an escalates after time, in days, hours, and minutes, that sets the wait before the next rule. You can also repeat the whole policy from 1 to 5 times when nobody acknowledges the incident.

The channels in the first rule are notified immediately. When an incident is neither acknowledged nor closed, Tenant Manager notifies the channels in the next rule after the escalates after time. An acknowledged incident is not escalated. If you change the escalation policy of an alert, an open incident raised under the old policy stays open but is not escalated further. At least one Tenant Manager escalation policy must exist before an alert can be saved, and deleting a policy assigns the Default Escalation Policy to the alerts that used it. Open incidents raised under the deleted policy stay open but are no longer escalated.

Notifications go to email and to the notification channels set up for your account. An email shows the account, tenant, severity, and time of the alert, the most urgent affected records, and a View incident button. It also shows a Download records link when the option Download records from notifications is on. The email never attaches the file. If nobody can be reached, Tenant Manager still raises the incident, and View notification details shows the reason.

FAQs

  1. Why did an alert fire but nobody was notified?
    The alert may have no notification channel or recipient, outbound email may not be configured, or a channel may have been deleted. Open Alert history, select the incident, and use View notification details to see the reason.

  2. Why do I see several open incidents for one alert?
    Each due check that finds the condition raises its own incident. Acknowledge or close them once the problem is dealt with. They are not closed automatically.

  3. What happens to open incidents when I disable an alert?
    They stay open. Disabling an alert stops it from being checked completely, which is different from muting it. Escalation resumes from the step it reached when you switch the alert on again.

Related articles