> For the complete documentation index, see [llms.txt](https://docs.pharos.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.pharos.ai/platform-overview/pharos-platform-architecture/alerting/log-level-alerts.md).

# Log-level Alerts

Understand individual log occurrence alerts and when to proceed with caution to avoid noise fatigue.

Log-level alerts trigger a notification **every time a single log is created**, regardless of whether it's part of an existing issue or a new one. While powerful for certain scenarios, they require careful configuration to avoid overwhelming teams with notification spam.

## What Are Log-Level Alerts?

Unlike issue-level alerts that fire once when an issue is created, log-level alerts fire **for every individual error occurrence**.

**Example Scenario:** The same flow fails 10 times in an hour:

* **Issue-level alert:** 1 notification (when issue is created)
* **Log-level alert:** 10 notifications (one per failure)

This fundamental difference makes log-level alerts both powerful and potentially problematic depending on your use case.

## When to Use Log-Level Alerts

Log-level alerts have legitimate use cases, but they're limited and specific.

### Valid Use Case 1: Active Development in Sandbox

**Scenario:** Developer working on a new flow or Apex class in a sandbox environment, wants immediate feedback when code fails.

**Configuration:**

```
Rule (Log Create):
  Filter: Organization = "Developer Sandbox - John"
  Action: Post to Slack direct message @john
  
Timeline: Activate during development session, deactivate when done
```

**Why This Works:**

* Immediate feedback loop during active coding
* Isolated to single developer (no team impact)
* Temporary (rule deactivated after dev session)
* Low error volume (controlled testing environment)

### Valid Use Case 2: Testing Specific Scenarios

**Scenario:** Team deploying a critical fix to production, wants to monitor for any occurrences of a specific error pattern for the next hour.

**Configuration:**

```
Rule (Log Create):
  Filter: Error Message contains "PaymentProcessorTimeout"
     AND Organization = "Production"
  Action: Post to Slack #deployment-monitoring
  
Timeline: Activated during deployment window, deactivated after validation
```

**Why This Works:**

* Time-limited (1-2 hour window)
* Highly specific filter (not a catch-all)
* Team is actively monitoring the channel
* Expected low volume (hopefully zero!)

### Valid Use Case 3: Critical Process Monitoring

**Scenario:** Payment processing errors must be investigated individually, even if they're the same root cause.

**Configuration:**

```
Rule (Log Create):
  Filter: Functional Area = "Payment Processing"
     AND Category = "Apex"
     AND Organization = "Production"
  Action: Post to Slack #payments-critical + Create PagerDuty incident
```

**Why This Works:**

* Business-critical process (revenue impact)
* Each failure represents a failed transaction requiring investigation
* Team has capacity to investigate each occurrence
* Trade-off: accept higher notification volume for zero missed payments

{% hint style="success" %}
**Key Principle for Log-Level Alerts**

Only use log-level alerts when **each individual occurrence requires separate investigation or action**. If you're grouping related errors mentally ("oh, it's just that same issue again"), you should be using issue-level alerts instead.
{% endhint %}

## When NOT to Use Log-Level Alerts

In most situations, log-level alerts do more harm than good. Avoid these patterns.

### Anti-Pattern 1: General Production Monitoring

```
❌ Bad Configuration:
Rule (Log Create):
  Filter: Organization = "Production"
  Action: Post to Slack #prod-errors
```

**Problem:**

* High-volume errors (batch job fails 1,000 times) = 1,000 notifications
* Team drowns in alerts
* Critical issues lost in noise
* Channel becomes unusable

**Solution:** Use issue-level alerts instead.

### Anti-Pattern 2: No Filter Criteria

```
❌ Bad Configuration:
Rule (Log Create):
  Filter: (none - catch all)
  Action: Post to Slack
```

**Problem:**

* Every log from every org/environment → notification
* Sandbox test errors spam production channel
* Volume makes channel completely unusable
* Team starts ignoring all notifications (including critical ones)

**Solution:** If you must use log-level alerts, add highly specific filters.

### Anti-Pattern 3: Permanent Log-Level Rules

```
❌ Bad Practice:
Create log-level rule, leave it active indefinitely
```

**Problem:**

* Error patterns change over time
* What starts as 5 notifications/day becomes 500
* No one remembers to deactivate the rule
* Notification fatigue sets in gradually

**Solution:** Log-level rules should be temporary or have built-in review/expiration.

## The Notification Fatigue Problem

Notification fatigue occurs when teams receive so many alerts that they:

1. Stop reading them carefully
2. Miss critical issues buried in noise
3. Mute channels or disable notifications entirely
4. Lose trust in the observability system

**Real-World Example from Training:**

During a customer onboarding, the team initially wanted log-level notifications for all errors. After demonstrating the volume (generating 10 errors resulted in 10 notifications flooding Slack), they immediately understood the problem and switched to issue-level alerts.

The training facilitator explained: *"If we were to do what we did in our first example, where we set up a log rule to send one every time, every time I click the button here, we see a notification. So you take anything away from today's call, this is it right here: the difference between issue and log notification and the mechanics."*

## Comparing Log-Level and Issue-Level Alerts

The trade-offs come down to volume, environment fit, and maintenance — log-level alerts are high-volume, sandbox-oriented, and need active management, while issue-level alerts are low-volume, production-oriented, and largely set-and-forget. See [Issue-level Alerts](https://docs.pharos.ai/platform-overview/pharos-platform-architecture/alerting/issue-level-alerts.md#log-level-vs-issue-level-notifications) for the side-by-side comparison.

## How Log-Level Alerts Work

Log-level notification rules are configured in **Admin → Notifications → Logs**.

**Evaluation Process:**

1. A log is created in Salesforce (error occurs)
2. Pharos evaluates all active log-level notification rules (top to bottom)
3. The first rule with matching filter criteria fires
4. Notification is sent immediately
5. Process completes (no relationship to issue creation)

**Timing:** Log-level notifications fire **extremely quickly**—typically within seconds of the error occurring. This is faster than issue-level notifications because there's no fingerprinting or grouping step required.

## Filter Criteria for Log-Level Rules

To use log-level alerts safely, apply strict filter criteria:

**Recommended Filters:**

* **Organization:** Limit the rule to one org, so sandbox errors never reach a production channel
* **Category:** Only specific error types (e.g., "Apex")
* **Functional Area:** Only business-critical processes (e.g., "Payment")
* **Summary:** Specific error text you're monitoring for, using the "contains" operator
* **User:** Specific user or integration user

**Example Safe Configuration:**

```
Rule (Log Create):
  Filter: 
    Organization = "Production"
    AND Category = "Apex"  
    AND Functional Area contains "Payment"
    AND Summary contains "Timeout"
  Action: Slack #payments-urgent
```

This rule is safe because:

* Highly specific (must match ALL conditions)
* Limited to critical business process
* Expected low volume (payment timeout is rare)
* Team is prepared to investigate each occurrence

## Managing Log-Level Rules

Because log-level rules carry real risk, treat them as temporary and review them regularly.

### Activation/Deactivation Workflow

**Best Practice Pattern:**

```
1. Create log-level rule in deactivated state
2. Activate only when needed (deployment, testing, troubleshooting)
3. Monitor channel actively during activation period
4. Deactivate immediately after window closes
5. Review if rule should become permanent (usually no)
```

### Review Cadence

If you must maintain permanent log-level rules:

* **Weekly review:** Check notification volume
* **Monthly review:** Assess if rule is still valuable
* **Quarterly review:** Consider converting to issue-level or removing entirely

### Documentation

Document why each log-level rule exists:

```
Rule Name: Payment Processor Timeout Monitoring
Purpose: Each timeout represents lost revenue, requires individual investigation
Owner: Payments Team
Active Period: Permanent (review quarterly)
Expected Volume: <10 per day
Last Reviewed: 2026-01-15
```

## Alternatives to Log-Level Alerts

Before creating a log-level alert, consider these alternatives:

### Alternative 1: Issue-Level Create Alert

```
Instead of: Log-level alert for all errors
Use: Issue-level alert (one notification per problem type)
```

### Alternative 2: Issue-Level Update Alert with Thresholds

```
Instead of: Log-level alert to know when errors recur
Use: Issue update alert when log count exceeds threshold

Rule (Issue Update):
  Filter: Log Count > 10 in 1 hour
  Action: Notify team
  
Result: One notification when problem escalates, not 50 individual notifications
```

### Alternative 3: Scheduled Digest

```
Instead of: Log-level real-time alerts for non-urgent errors
Use: Daily email digest of new logs

Rule (Scheduled):
  Frequency: Daily at 8 AM
  Filter: Category = "Apex" AND Created Yesterday
  Action: Email summary to team
```

### Alternative 4: Dashboard Review

```
Instead of: Log-level alerts for awareness
Use: Dashboard showing recent log trends

Process: Team reviews dashboard during daily standup
Benefit: No interruptions, but still maintains visibility
```

## Transitioning from Log-Level to Issue-Level

If you currently have log-level alerts and want to reduce noise:

**Step 1: Measure Current Volume**

* How many notifications per day?
* How many are actionable?
* How many are duplicates of same problem?

**Step 2: Create Parallel Issue-Level Rules**

* Set up issue-level rules with same filters
* Run both in parallel for 1 week

**Step 3: Compare Results**

* Log-level: 500 notifications
* Issue-level: 25 notifications (same problems)

**Step 4: Deactivate Log-Level Rules**

* Keep issue-level rules active
* Monitor for missed critical issues (likely none)

**Step 5: Refine**

* Add issue update rules for escalation scenarios
* Adjust filters based on team feedback

## Real-World Success Story

**Before (Log-Level Alerts):**

```
Setup: All production errors → Slack #prod-errors
Volume: 400-800 notifications per day
Result: Team muted channel, missed critical payment processing issue
```

**After (Issue-Level Alerts):**

```
Setup: 
  - Issue-level create for new problems → #prod-errors
  - Issue-level update for escalations (>100 occurrences) → #escalations
  - Log-level ONLY for payment processing → #payments-critical
  
Volume: 20-40 issue notifications per day + <5 payment logs
Result: Team actively monitors, critical issues addressed within 2 hours
```

## Summary

Log-level alerts have a place in your alerting strategy, but they're the exception, not the rule:

**Use log-level alerts for:**

* Active development in sandbox (temporary)
* Monitoring specific critical transactions individually
* Time-limited testing scenarios

**Don't use log-level alerts for:**

* General production monitoring
* High-volume error types
* Permanent "set and forget" rules
* Any scenario where you expect >20 notifications per day

**Default Recommendation:** Start with issue-level alerts for production. Only add log-level alerts for specific, documented use cases with clear activation/deactivation plans.

{% hint style="warning" %}
**Avoid Notification Fatigue**

If your team is getting 100+ notifications per day, you're likely over-alerting. Review your log-level rules first—they're usually the culprit. Switch to issue-level alerts and use update rules for escalations.
{% endhint %}

For production environments, see [Issue-level Alerts](https://docs.pharos.ai/platform-overview/pharos-platform-architecture/alerting/issue-level-alerts.md) for the recommended notification pattern.
