> For the complete documentation index, see [llms.txt](https://docs.pharos.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.pharos.ai/platform-overview/pharos-platform-architecture/alerting.md).

# Alerting

Learn how Pharos alerting system delivers context-aware notifications to Slack, Microsoft Teams, and PagerDuty through flexible rule-based automation.

The alerting functionality in Pharos ensures that the right people know about problems at the right time, transforming error detection into awareness without causing alert fatigue.

{% hint style="success" %}
**Smart Notifications:** Pharos alerts on Issues (grouped errors), not individual logs—reducing 500 errors into 1 actionable notification with complete context.
{% endhint %}

## Why Alerting Matters

In observability systems, collecting data is only half the equation—teams need to be **notified when issues arise** before they escalate into business incidents. However, poorly configured alerting creates its own problems:

**The Alert Fatigue Problem:**

* **Too many alerts:** Teams become numb to notifications and start ignoring them
* **Duplicate alerts:** Same error triggers 50 notifications instead of one grouped alert
* **Lack of context:** Alerts say "something broke" without providing actionable information
* **Wrong audience:** Developers get paged for data quality issues; Admins get API timeouts

**The Business Impact:**

* Critical issues get lost in notification noise
* Teams spend hours debugging to gather context the alert should have provided
* On-call burnout from non-actionable alerts
* Delayed response to revenue-impacting failures

Pharos alerting is designed to solve these problems through:

1. **De-duplication:** Alert on Issues (grouped errors), not individual log entries
2. **Context-rich notifications:** Include impact metrics and direct links to debugging info
3. **Rule-based targeting:** Route alerts based on error type, severity, and business area
4. **Integration flexibility:** Meet teams where they work (Slack, Teams, PagerDuty)

## Alerting Philosophy

Pharos alerting operates on two core principles:

### 1. Alert on Issues, Not Logs

**The Problem:**\
If the same error occurs 500 times in an hour, traditional systems send 500 notifications. This is overwhelming and unhelpful.

**Pharos Approach:**\
Pharos groups identical errors into a single **Issue** via fingerprinting. Alerts are triggered at the **Issue level**, not the log level.

**Outcome:**

* First occurrence of an error → Issue created → Notification sent ✅
* Second occurrence of same error → Existing Issue updated → No new notification ✅
* Different error → New Issue created → New notification sent ✅

**Result:** 500 errors, 1 notification. Team investigates once, fixes the root cause.

### 2. Context-Aware Routing

Not all errors are equal. Alerting rules should account for:

* **Severity:** Critical vs. informational
* **Business impact:** User-facing vs. backend processing
* **Functional area:** Sales vs. Service vs. Marketing
* **Time sensitivity:** Payment failures need immediate attention; reporting errors don't

Pharos notification rules provide granular control to route the right alerts to the right teams at the right time.

## In This Section

This section covers alerting end to end, from strategy to channel setup.

<table data-view="cards"><thead><tr><th></th><th></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><strong>💡 Alerting Use Cases</strong></td><td>Different use cases for leveraging notifications and recommended alert types for various situations.</td><td><a href="https://docs.pharos.ai/platform-overview/pharos-platform-architecture/alerting/alerting-use-cases.md">Alerting Use Cases</a></td></tr><tr><td><strong>📢 Notification Channels</strong></td><td>Main channels of alerting and their capabilities—Slack, Teams, PagerDuty, Email, and Webhooks.</td><td><a href="https://docs.pharos.ai/platform-overview/pharos-platform-architecture/alerting/notification-channels.md">Notification Channels</a></td></tr><tr><td><strong>📝 Log-level Alerts</strong></td><td>Individual log occurrence alerts and when to use them with caution.</td><td><a href="https://docs.pharos.ai/platform-overview/pharos-platform-architecture/alerting/log-level-alerts.md">Log-level Alerts</a></td></tr><tr><td><strong>🎯 Issue-level Alerts</strong></td><td>Most common notification pattern with create-based and update-based triggering.</td><td><a href="https://docs.pharos.ai/platform-overview/pharos-platform-architecture/alerting/issue-level-alerts.md">Issue-level Alerts</a></td></tr><tr><td><strong>📊 Order of Execution</strong></td><td>Rule ordering to avoid excessive noise with ladder system notifications.</td><td><a href="https://docs.pharos.ai/platform-overview/pharos-platform-architecture/alerting/order-of-execution.md">Order of Execution</a></td></tr></tbody></table>

## Best Practices

Effective alerting is iterative—start narrow and expand as you learn your error patterns.

### Start Conservative, Refine Over Time

**Phase 1: Basic Alerting**

* Single rule: All production Issues → One Slack channel
* Monitor for 1-2 weeks
* Identify noise and patterns

**Phase 2: Categorization**

* Split by functional area (Sales, Service, Marketing)
* Add user impact thresholds
* Introduce PagerDuty for critical only

**Phase 3: Optimization**

* Fine-tune thresholds based on historical data
* Add suppression rules for known non-issues
* Implement escalation tiers

### Recommended Setup

✅ **Multi-Tier Channel Strategy**

```mermaid
graph TD
    A[#production-critical - PagerDuty + Slack]
    B[#production-errors - Slack only]
    C[#sandbox-digest - Email daily]
    A --> A1[Critical, user-facing, revenue-impacting]
    B --> B1[All other production Issues]
    C --> C1[Non-production for awareness]
```

✅ **Business Hours vs. 24/7**

* PagerDuty: Only for issues requiring immediate response
* Slack: Business hours awareness
* Email digest: Non-urgent updates

✅ **Team-Specific Routing**

* Integration team owns middleware failures
* Data team owns batch job Issues
* Dev team owns application code Issues

{% hint style="success" %}
**Next Steps:**

1. Set up your first notification rule (all Issues → one Slack channel)
2. Monitor for one week to understand baseline noise
3. Add filtering criteria to reduce false positives
4. Introduce PagerDuty for true emergencies only
5. Review and refine monthly based on action rates

→ Start with [Notification Channels](https://docs.pharos.ai/platform-overview/pharos-platform-architecture/alerting/notification-channels.md) to understand integration options.
{% endhint %}
