> For the complete documentation index, see [llms.txt](https://docs.pharos.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.pharos.ai/platform-overview/pharos-platform-architecture/issue-tracker/issue-details/tracing.md).

# Tracing

Understand the tracing process for dynamically fetching and analyzing Salesforce debug logs.

Tracing is Pharos's capability to capture full Salesforce debug logs specifically when an Issue occurs. While standard debug views show stack traces and error messages, tracing gives you complete execution context—every SOQL query, DML operation, variable assignment, and CPU measurement leading up to and following the error.

## Why Tracing Matters

### The Debug Log Challenge

Salesforce debug logs are powerful but difficult to work with:

* **Manual setup required** → Must enable trace flags for specific users
* **Time-limited** → Logs expire after hours
* **Hard to capture** → Need to know exactly when error will occur
* **Storage intensive** → Can't leave trace flags on permanently

### How Tracing Solves This

Pharos Tracing **automates debug log capture** when specific errors occur:

* No manual trace flag setup
* Captures logs automatically when Issue's fingerprint is detected
* Stores logs permanently (within retention policy)
* Provides context logs before and after error

## When to Use Tracing

### Ideal Scenarios

Tracing pays off in a few specific situations where a stack trace alone falls short.

**Intermittent Errors:**

* Happens sporadically, can't reproduce on demand
* Need to capture log during actual failure in production

**Governor Limit Issues:**

* CPU timeouts or heap size exceeded
* Need to see full transaction context (what else ran?)
* Identify cumulative limit consumption

**Complex Transaction Flows:**

* Multi-step process (trigger → workflow → process → flow → apex)
* Need to see full execution sequence
* Understand order of operations

**Variable State Debugging:**

* Need to see variable values at time of failure
* Decision logic not clear from stack trace alone
* Null pointer but unclear which variable was null

### When NOT to Use Tracing

**Don't use tracing if:**

* Error is consistent and easily reproducible → Just debug in sandbox
* Stack trace provides sufficient information → Simple null check fix
* Error happens constantly → Too many logs, performance impact
* Debug view already shows clear root cause → Fix directly

## How Tracing Works

### Trace Schedule

A **Trace Schedule** defines when and how to capture debug logs for an Issue.

**Key components:**

1. **Associated Issue** → Which Issue to trace
2. **Trace Windows** → Time periods when tracing is active
3. **Captured Log Window** → How many surrounding logs to capture
4. **Status** → Scheduled, In Progress, Completed, or Inactive

### Trace Window

A **Trace Window** is a specific time period when tracing is active—think of it like a calendar appointment.

**Example:**

```
Trace Window 1: Monday 9:00 AM - 5:00 PM PST
Trace Window 2: Tuesday 9:00 AM - 5:00 PM PST
Trace Window 3: Wednesday 9:00 AM - 5:00 PM PST
```

**Why multiple windows?**

* Error might happen during specific timeframe (business hours, scheduled job)
* Can't trace 24/7 (performance and storage concerns)
* Target high-probability windows

### Captured Trace Log Window

The **Captured Trace Log Window** setting determines how many logs around the error to capture.

**Default: Setting = 5**

* Captures a couple of logs **before** the error log
* Captures the error log itself
* Captures a couple of logs **after** the error log

**Why capture surrounding logs?**

* See what else was running in org
* Understand sequence leading to error
* Identify if earlier transaction contributed to failure

**Configurable by admin:**

* Check your org's setting in Pharos Admin console
* Default is 5 logs; increase if you need more surrounding context
* Higher number = more context, more storage used

### Trace Lifecycle

A trace moves through a predictable lifecycle from creation through activation, capture, and review:

```
1. Create Trace Schedule → Define windows
2. Activate Trace Schedule → Tracing begins
3. Error occurs during window → Debug logs captured
4. Trace Result created → Links to captured logs
5. Last window ends → Schedule auto-deactivates
6. Review Trace Results → Analyze logs
```

## Creating a Trace Schedule

### From the Issue Record

You create a trace schedule directly from the Issue you want to investigate:

1. Navigate to **Tracing** tab or related list on Issue
2. Click **New Trace Schedule**
3. Add one or more trace windows, each its own record (`pharos__Trace_Request__c`) linked to the schedule:
   * **Start Date/Time** → When that window begins
   * **End Date/Time** → When that window ends
   * Windows can be chained to one another (`Parent__c`, `Next_Trace_Start_Date__c`) for a recurring sequence, rather than a single "recurrence" setting
4. Save
5. Trace Schedule Status → `Scheduled`

### Defining Trace Windows

**Single Window:**

```
Start: Today 2:00 PM
End: Today 5:00 PM
```

→ Trace active for 3 hours today

**Multiple Windows (Daily):**

```
Window 1: Monday 9:00 AM - 5:00 PM
Window 2: Tuesday 9:00 AM - 5:00 PM
Window 3: Wednesday 9:00 AM - 5:00 PM
Window 4: Thursday 9:00 AM - 5:00 PM
Window 5: Friday 9:00 AM - 5:00 PM
```

→ Trace active business hours for the week (five separate `Trace_Request__c` windows under one schedule)

**Specific Time Slots:**

```
Window 1: Monday 9-11 AM
Window 2: Monday 2-4 PM
Window 3: Tuesday 9-11 AM
Window 4: Tuesday 2-4 PM
```

→ Trace only during high-activity periods

### Best Practices for Window Selection

**Consider:**

* When does the error typically occur? (check Occurrence chart)
* Business hours vs. off-hours?
* Scheduled job times (if applicable)
* User reported timeframes

**Tips:**

* Start with narrow windows (2-4 hours)
* Expand if error doesn't occur
* Don't trace 24/7 (performance impact)
* Limit to 1-2 days initially

## Trace Results

When an error matching the Issue's fingerprint occurs during an active trace window, a **Trace Result** is created.

### Viewing Trace Results

1. From Issue record, navigate to **Trace Results** related list
2. Click on a Trace Result
3. View captured logs

**What you'll see:**

* List of captured debug logs
* **Highlighted log** → The one containing the actual error
* Logs before and after (based on Captured Log Window setting)
* Download options for each log

### Analyzing Trace Results

**Step 1: Open the error log**

* Identified by highlight or error indicator
* This log contains the failure

**Step 2: Review execution timeline**

* Look at timestamps
* See sequence of operations
* Identify what triggered the error

**Step 3: Check surrounding logs**

* Did another transaction contribute?
* Governor limits accumulating?
* Related record locks?

**Step 4: Download for deep analysis**

* Click **Download** on any log
* Opens `.log` file
* Analyze in VS Code, Chrome extension, or Salesforce IDE

## Downloading and Analyzing Debug Logs

### Download Process

From Trace Result:

1. Click on a specific log
2. Click **Download Debug Log** button
3. Saves `.log` file to your computer

### Analysis Tools

Several tools can open and analyze a downloaded log, from Salesforce's own consoles to third-party extensions.

**Salesforce Developer Console:**

* Upload log via File → Open
* Use Execution Overview and Timeline views

**VS Code with Salesforce Extension:**

* Right-click log file → Salesforce: Analyse Log
* Interactive timeline and filter tools

**Chrome Extensions:**

* Salesforce Inspector
* ApexLog Analyzer

**Raw Text Editor:**

* Search for specific classes, methods, errors
* Grep for patterns

### What's in a Full Debug Log

**Execution flow:**

```
EXECUTION_STARTED
METHOD_ENTRY: OpportunityTriggerHandler.afterInsert
  SOQL_EXECUTE_BEGIN: SELECT Id, Industry FROM Account WHERE ...
  SOQL_EXECUTE_END: 1 rows retrieved
  VARIABLE_ASSIGNMENT: acc = {Id=001xx...}
  SYSTEM_METHOD_ENTRY: acc.Industry
  EXCEPTION_THROWN: NullPointerException (line 156)
METHOD_EXIT
```

**Governor limits:**

```
CUMULATIVE_LIMIT_USAGE
  SOQL queries: 15 of 100
  DML statements: 3 of 150
  CPU time: 3,245 ms of 10,000 ms
  Heap size: 1.2 MB of 6 MB
```

**Full context:**

* Every method call
* Every query with results
* Every variable value
* Timestamp for each operation
* Limit consumption at each step

## Managing Trace Schedules

### One Active Trace Per Issue

**Rule:** Only one active Trace Schedule allowed per Issue at a time.

**Why?**

* Prevent duplicate log captures
* Control performance impact
* Simplify result management

**To create a second trace:**

1. Deactivate current Trace Schedule
2. Create new Trace Schedule
3. Activate new one

### Automatic Deactivation

Trace Schedules **auto-deactivate** when:

* Last trace window has ended
* All scheduled windows have passed
* Regardless of whether logs were captured

**Example:**

```
Trace Window: Monday 9 AM - 5 PM
Current Time: Monday 5:01 PM
→ Trace Schedule Status = Completed
```

**To trace again:**

* Must create new Trace Schedule
* Cannot reactivate completed schedule

### Manual Deactivation

You can manually deactivate an active trace:

1. Open Trace Schedule record
2. Click **Deactivate** button
3. Status → `Inactive`

**When to manually deactivate:**

* Error captured, don't need more logs
* Trace window too broad, want to refine
* Need to create new schedule with different windows

## Trace Schedule View

Pharos provides a **Trace Schedule Calendar View** showing all active traces across all Issues.

### What It Shows

The calendar view surfaces everything currently being traced across your org:

* All active Trace Schedules
* Trace windows as calendar blocks
* Issue name and priority
* Overlapping traces (if multiple Issues being traced)

### Why It's Useful

**Capacity management:**

* See how many Issues are being traced concurrently
* Avoid too many active traces (performance concern)

**Team coordination:**

* Know which Issues are under investigation
* Avoid duplicate tracing efforts

**Scheduling:**

* Plan trace windows to avoid conflicts
* Stagger traces to distribute load

## Performance and Storage Considerations

### Performance Impact

Active tracing has minimal impact:

* Only captures logs when error occurs
* Doesn't slow down transactions
* Isolated to specific Issue fingerprint

**However:**

* Too many concurrent active traces can add overhead
* Very broad trace windows (24/7 for weeks) not recommended

**Best practices:**

* Limit to 5-10 concurrent active traces org-wide
* Use narrow trace windows (hours, not days)
* Deactivate when logs captured

### Storage Impact

Debug logs are large:

* Typical log: 1-10 MB
* With Captured Log Window at its default of 5: 5-50 MB per trace result
* Can accumulate quickly

**Storage management:**

* Logs respect org's retention policy
* Old trace results may be purged
* Download critical logs for permanent storage

## Tracing Workflow Example

### Scenario: Intermittent CPU Timeout

```
Issue: CPU_TIME_LIMIT_EXCEEDED in AccountTriggerHandler
Frequency: 2-3 times per week
User reports: "Happens mid-morning, can't predict when"
```

**Step 1: Create Trace Schedule**

* Start: Monday 9 AM
* End: Wednesday 12 PM
* Windows: M/T/W 9 AM - 12 PM (high-activity period)
* Captured Log Window: 30 (need extra context)

**Step 2: Wait for Error**

* Trace runs in background
* Tuesday 10:23 AM: Error occurs
* Trace Result created automatically

**Step 3: Review Trace Result**

* Open Trace Result
* See 30 logs captured
* Error log highlighted
* Download error log for analysis

**Step 4: Analyze Debug Log**

* Open in VS Code
* See cumulative CPU usage
* Identify: 3 triggers firing in sequence
* Trigger 3 pushing CPU over limit

**Step 5: Implement Fix**

* Refactor Trigger 3 to reduce CPU
* Deploy to sandbox, test
* Deploy to production
* Deactivate trace

**Step 6: Monitor**

* Mark Issue status → `In Review`
* Watch for recurrence
* No new logs → Mark `Completed`

## Tracing and Regression

If an Issue regresses (marked Completed but reoccurs):

* Old trace schedules remain inactive
* Must create **new trace schedule** if debugging needed
* Can reference old trace results for comparison

## Limitations of Tracing

### What Tracing Can't Do

**Can't capture:**

* Errors outside trace windows
* Logs for different Issues (single Issue per schedule)
* Logs from sandboxes (production only, unless Pharos installed in sandbox)

**Can't predict:**

* When error will occur (you define windows, not error timing)
* Which user will trigger it
* Exact log before/after (depends on org activity)

### Known Challenges

**Storage-intensive:**

* Large trace windows or many results consume storage
* May need to clean up old results

**Window timing:**

* Narrow windows might miss error
* Broad windows capture too much
* Requires iteration to optimize

**Performance edge cases:**

* Extremely high transaction volume orgs may see slight overhead
* Very large Captured Log Window settings (>50) can be slow

{% hint style="success" %}
**Next Steps:**

* [Resolution](https://docs.pharos.ai/platform-overview/pharos-platform-architecture/issue-tracker/issue-lifecycle/resolution.md) – Full resolution workflow including tracing step
* [Debug Views](https://docs.pharos.ai/platform-overview/pharos-platform-architecture/issue-tracker/issue-details/debug-views.md) – Understanding stack traces and error messages
* [Related Logs](https://docs.pharos.ai/platform-overview/pharos-platform-architecture/issue-tracker/issue-details/related-logs.md) – Reviewing individual log occurrences
  {% endhint %}
