> For the complete documentation index, see [llms.txt](https://docs.pharos.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.pharos.ai/platform-overview/pharos-platform-architecture/logs/log-presentations/bulk-api-logs.md).

# Bulk API Logs

Learn about Bulk API log presentations including error summaries, pattern analysis, and batch operation details.

Bulk API logs in Pharos address the unique challenges of understanding failures across thousands of records processed in batch operations. Rather than presenting each failed record individually, Pharos aggregates, patterns, and summarizes bulk failures to surface actionable insights.

{% hint style="success" %}
**From Noise to Patterns:** Without aggregation, 1,000 records failing the same validation rule creates 1,000 individual error messages. Pharos condenses this to a single pattern: "Validation Rule 'X' failed for 1,000 records."
{% endhint %}

## The Bulk Operation Challenge

Bulk API operations—whether data imports from external systems, mass updates from Data Loader, or large-scale record deletions—process records in batches that can range from hundreds to hundreds of thousands of records. When failures occur, they generate one error message per failed record.

**Without Aggregation:**

* 1,000 records fail with the same validation rule error → 1,000 separate messages
* 500 records fail due to duplicate detection → 500 separate messages
* 250 records fail with missing required fields → 250 separate messages
* 100 records fail for various data quality issues → 100 separate messages

**The Key Question:** Not "What happened to record ID 001abc123?" but rather "What failure patterns exist across this batch, and how many records does each pattern affect?"

**Pharos Solution:** Pharos provides a multi-level presentation—high-level summaries for pattern identification, drill-down views for affected record lists, and individual record details for specific failure investigation.

## Bulk API Log Structure

When a Bulk API operation completes, Pharos ingests the results and creates a structured log with several components. This structure provides both breadth (what happened across the entire operation) and depth (what happened to specific records).

| Component              | What's Included                                                                                              | Purpose                                   |
| ---------------------- | ------------------------------------------------------------------------------------------------------------ | ----------------------------------------- |
| **Operation Metadata** | Job ID, operation type (Insert/Update/Upsert/Delete), object type, total/successful/failed counts, timestamp | Complete operation context                |
| **Batch Information**  | Batch IDs, records per batch, batch processing status                                                        | Track multi-batch operations              |
| **Result Summary**     | Aggregated failure patterns, common error types/frequencies, affected field analysis                         | Pattern identification and prioritization |

## Error Summary View

The **Error Summary** is the primary interface for understanding bulk failures. It presents aggregated error patterns rather than individual record failures.

### Error Pattern Grouping

Pharos analyzes all failed records in the operation and groups them by error signature:

| Error Pattern                 | What's Grouped                                                        | Example                                                                                              |
| ----------------------------- | --------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
| **Validation Rule Failures**  | Validation rule name, affected record count, common field values      | "Validation Rule: 'Opportunity\_Amount\_Required' failed for 847 records where Amount was null/zero" |
| **Duplicate Detection**       | Duplicate rule name, duplicate pair count, matching criteria          | "Duplicate Rule: 'Standard\_Account\_Duplicate' found 312 matches based on Name + Billing Street"    |
| **Required Field Violations** | Field name, affected count, object type                               | "Required field 'Industry' missing on 156 Account records"                                           |
| **Data Type Mismatches**      | Field name/expected type, affected count, malformed data example      | "Field 'Annual\_Revenue\_\_c' expects Currency but received 'N/A' in 89 records"                     |
| **Permission/Access Errors**  | Inaccessible field/object, affected count, user context               | "User lacks Edit permission on 'Contract\_Value\_\_c' for 203 Opportunity records"                   |
| **Custom Error Messages**     | Error pattern (normalized), affected count, trigger/validation source | "Trigger 'OrderValidation' rejected 67 records: 'Cannot create order without approved quote'"        |

### Error Distribution Visualization

The summary view includes visual representations of error distribution:

**Error Type Breakdown** — A chart showing the proportion of failures by category (validation rules, duplicates, required fields, etc.). This immediately reveals whether failures are dominated by a single issue or spread across multiple problems.

**Top Errors Ranking** — The most frequent error patterns ranked by affected record count. This prioritization guides remediation efforts—fix the top error and you address the majority of failures.

**Success vs. Failure Ratio** — Visual indication of how many records succeeded versus failed, providing context for the severity of the issue. 10 failures out of 10,000 records is very different from 5,000 failures out of 10,000.

## Detailed Error View

Expanding an error pattern in the summary reveals the **Detailed Error View** with specifics about affected records.

### Affected Record List

For each error pattern, a table lists the specific records that experienced that error:

* Record ID (if the record already existed) or External ID
* Key identifying fields (Name, Email, Account Number, etc.)
* Relevant field values that contributed to the error
* Error message specific to that record (if it varies slightly from the pattern)

This list is paginated for large failure sets and supports filtering and sorting by field values. You can export this list to CSV for further analysis or correction in external tools.

### Sample Records

For error patterns affecting hundreds or thousands of records, examining every failed record is impractical. The Detailed Error View provides **sample records** that exemplify the error pattern:

* 5-10 representative records from the failure set
* Field values highlighting why the error occurred
* Contextual information about the record's state

This sampling allows quick understanding of the failure cause without scrolling through massive lists.

### Field Value Analysis

For errors related to specific fields, the Detailed Error View includes field value analysis:

**Value Distribution** — If 500 records failed validation on the "Industry" field, the analysis shows which industries were present in the failed records and their frequencies. This can reveal data quality patterns like "All failures have Industry = 'Unknown'" or "Failures are evenly distributed across industries, suggesting the validation rule itself may be wrong."

**Empty/Null Field Analysis** — Identification of which required or expected fields were empty across the failed records.

**Data Type Issues** — For type mismatch errors, examples of the problematic values that couldn't be converted to the expected type.

## How Error Summaries Are Constructed

Understanding how Pharos generates error summaries helps interpret them correctly.

### Error Fingerprinting

Each failed record has an error message. Pharos applies fingerprinting logic to normalize these messages:

1. **Dynamic Value Removal** — Error messages often include record-specific values like IDs, dates, or user names. These are normalized to patterns:
   * "Account '001abc123' was not found" becomes "Account '{RECORD\_ID}' was not found"
   * "Value '2026-13-45' is not a valid date" becomes "Value '{VALUE}' is not a valid date"
2. **Pattern Matching** — Normalized error messages are matched to identify identical patterns. All errors matching the same pattern are grouped.
3. **Categorization** — Error patterns are categorized by type (validation, duplicate, required field, etc.) based on Salesforce error codes and message structure.
4. **Aggregation** — Records sharing the same error pattern are counted and aggregated into a single summary entry.

This process transforms "3,000 individual error messages" into "5 error patterns affecting various record counts."

### Field Context Extraction

For each error pattern, Pharos analyzes the failed records to determine:

* Which fields are mentioned in the error message
* Which fields have unusual values (nulls, outliers, invalid formats)
* Which fields are common across all failed records (indicating a systematic issue)

This context is surfaced in the summary to guide investigation. If all failed records share a common field value, that's highlighted as a potential root cause.

## Batch-Level Details

Large bulk operations are split into multiple batches by Salesforce. Pharos tracks batch-level information:

**Batch Success/Failure Status:**

* Which batches completed successfully
* Which batches had partial failures
* Which batches failed entirely (rare, usually due to system issues)

**Per-Batch Statistics:**

* Records processed in each batch
* Failure rate per batch

**Batch Timing:**

* How long each batch took to process
* Whether certain batches experienced significantly longer processing times

Batch-level views help identify whether failures are uniformly distributed or concentrated in specific batches (which might indicate data-specific issues in those record sets).

## Bulk API Operation Types

Different operation types have specialized presentations:

### Insert Operations

For inserts, failures typically relate to:

* Required fields missing
* Validation rules rejecting the new records
* Duplicate rules blocking creation

The summary highlights which aspect of record creation failed most frequently.

### Update Operations

For updates, failures often stem from:

* Records not found (invalid IDs)
* Field-level security (user can't update certain fields)
* Validation rules on modified fields
* Record locks (records being edited by other users)

The summary distinguishes between "record not found" errors (data loading issue) and "validation failed" errors (data quality issue).

### Upsert Operations

Upserts use an external ID field to match existing records or create new ones. Failures include:

* External ID field not found or incorrectly named
* Multiple records matching the same external ID (ambiguous match)
* Validation failures on either the update or insert path

The summary clarifies whether failures occurred during the "match existing record" phase or the "create/update" phase.

### Delete Operations

Delete failures relate to:

* Records not found
* Master-detail relationships preventing deletion
* Triggers rejecting deletion

The summary identifies which records couldn't be deleted and why.

## Integration with Data Quality

Bulk API failure patterns often reveal systemic data quality issues:

**Incomplete Data Migrations** — If a bulk import fails because 500 records lack required fields, this suggests the source system data is incomplete or the field mapping is incorrect.

**Outdated Validation Rules** — If hundreds of records fail a validation rule during a bulk update, the validation rule may need revision to accommodate valid business scenarios.

**Permission Gaps** — If the integration user lacks field permissions for hundreds of records, this indicates a permission set or profile misconfiguration.

Pharos surfaces these patterns in the error summary, enabling proactive data quality improvements rather than record-by-record firefighting.

## Remediation Workflow

The Bulk API presentation supports a structured remediation workflow:

1. **Review Error Summary** — Identify the top failure patterns and their affected record counts.
2. **Prioritize by Impact** — Address the error pattern affecting the most records first.
3. **Investigate Root Cause** — Use the Detailed Error View to understand why records failed (data issue, validation logic, permission problem, etc.).
4. **Correct and Retry:**
   * Export the list of failed records
   * Correct the data in the source file or CSV
   * Re-run the bulk operation for only the failed records
5. **Validate Success** — Confirm that the corrected batch succeeds without errors.

This workflow is far more efficient than attempting to manually review thousands of individual error messages.

## Bulk API Logs and Issue Tracking

Bulk API failures are fingerprinted and grouped into Issues in Pharos. A single Issue might represent:

"Bulk insert to Account object fails validation rule 'Opportunity\_Amount\_Required' for records missing Amount field"

From the Issue detail page, you can see:

* How many bulk operations have encountered this pattern
* When the issue first appeared (perhaps after a validation rule was added)
* Which integration or user is triggering the failures most frequently

This Issue-level tracking identifies recurring bulk operation problems that require systemic fixes rather than one-off data corrections.

## Performance Considerations

Bulk operations can take minutes or hours to process. The log captures performance metrics:

* Total operation duration
* Time per batch
* Records processed per second

Slow bulk operations might indicate:

* Complex validation rules or triggers executing for each record
* Governor limit throttling
* External system callouts in triggers (which should be bulkified)

These performance insights guide optimization efforts.

## Summary

Bulk API log presentations in Pharos transform overwhelming lists of individual record failures into actionable error summaries that reveal patterns, prioritize remediation, and enable efficient bulk data operation troubleshooting.

**Key Features:**

* **Error Pattern Grouping:** Validation rules, duplicates, required fields, data type mismatches, permissions, custom errors
* **Multi-Level Presentation:** High-level summaries → drill-down to affected records → individual record details
* **Visual Analytics:** Error distribution charts, top errors ranking, success vs. failure ratios
* **Corrective Recommendations:** Specific guidance for fixing validation rules, duplicates, required fields, permissions
* **Performance Metrics:** Operation duration, time per batch, records per second
* **Issue Tracking Integration:** Recurring bulk operation problems tracked at pattern level

**The Result:** By aggregating errors by type and providing drill-down capabilities, Pharos makes it straightforward to understand what went wrong across thousands of records and what needs to be corrected. Whether processing daily integration feeds or one-time data migrations, Bulk API presentations ensure failures are analyzed at the pattern level rather than getting lost in record-level noise.
