PagerDuty Alert Severity Not Respecting Configuration

Problem

PagerDuty alerts are being triggered with high urgency or critical severity even when configured to use lower severity levels (e.g., warning, info, or low priority) in LangSmith.

Symptoms

• All alerts appear as "critical" severity regardless of configuration

• Alerts sent with "warning" severity still trigger high-urgency notifications

• PagerDuty integration ignores the severity level dropdown setting

• Alert payload shows "severity": "critical" even when set to low priority

Root Cause

The PagerDuty integration's notification urgency setting is configured as "High-urgency notifications" (static) instead of "Dynamic notifications based on alert severity". When set to high-urgency mode, PagerDuty overrides the alert severity and sends all notifications with high urgency, regardless of the severity level specified in the alert payload.

Solution

Configure the PagerDuty integration to use dynamic notification urgency:

1. Navigate to your PagerDuty integration settings

2. Locate the "Assign and Notify" section

3. Find the dropdown "How should responders be notified?"

4. Change from "High-urgency notifications" to "Dynamic notifications based on alert severity"

5. Review the "Alert Severity to Notification Urgency Mapping" table that appears:

   • Critical → High urgency

   • Error → High urgency

   • Warning → Low urgency

   • Info → Low urgency

   • Not Available → High urgency

6. Click "Save Changes" to apply the configuration

Testing

After updating the configuration, test with a sample alert:

curl --request 'POST' \--url 'https://events.pagerduty.com/v2/enqueue' \--header 'Content-Type: application/json' \--data '{
  "payload": {
      "summary": "Test alert",
      "severity": "warning",
      "source": "Alert source"
  },
  "routing_key": "your_routing_key",
  "event_action": "trigger"
}'

The alert should now respect the "severity": "warning" setting and trigger a low-urgency notification instead of a high-urgency one.