EX1030895: Identifying NDRs During the Recent Microsoft Exchange Online Outage

Identifying NDRs During the Recent Microsoft Exchange Online Outage

Introduction

During the recent Microsoft Exchange Online outage, many users experienced issues with Non-Delivery Reports (NDRs) indicating "554 5.6.0 Corrupt message content." To address this, I created a PowerShell script to identify these specific NDRs across all users in the tenant. In this blog post, I'll walk you through the script and explain how it works.

The Script

Here's the PowerShell script I used:

# Get all users in the tenant
$users = Get-Mailbox -ResultSize Unlimited
$totalUsers = $users.Count
$currentUser = 0

# Loop through each user
foreach ($user in $users) {
    $currentUser++
    $progressPercent = [math]::Round(($currentUser / $totalUsers) * 100)
    Write-Progress -Activity "Processing users" -Status "$progressPercent% Complete" -PercentComplete $progressPercent

    $senderAddress = $user.PrimarySmtpAddress
    Get-MessageTrace -StartDate (Get-Date).AddDays(-2) -EndDate (Get-Date) -Status Failed -PageSize 1000 -SenderAddress $senderAddress | ForEach-Object { 
        $trace = $_
        Get-MessageTraceDetail -MessageTraceId $trace.MessageTraceId -RecipientAddress $trace.RecipientAddress | Where-Object { $_.Detail -like "*554 5.6.0 Corrupt message content*" } | Select-Object @{Name="MessageTraceId";Expression={$trace.MessageTraceId}}, @{Name="MessageId";Expression={$trace.MessageId}}, @{Name="Date";Expression={$trace.Received}}, @{Name="Event";Expression={$_.Event}}, @{Name="Action";Expression={$_.Action}}, @{Name="Detail";Expression={$_.Detail}}, @{Name="Sender";Expression={$trace.SenderAddress}}, @{Name="Recipient";Expression={$trace.RecipientAddress}}, @{Name="Subject";Expression={$trace.Subject}}
    } | Export-Csv -Path "C:\ramki\ndrrepo.csv" -NoTypeInformation -Append
}

How It Works

1. Retrieve All Mailboxes: The script starts by retrieving all mailboxes in the tenant.
2. Loop Through Each User: It then loops through each user, calculating the progress percentage to provide feedback on the script's progress.
3. Get Message Trace Details: For each user, the script retrieves message trace details for messages with a "Failed" status within the last two days.
4. Filter for Specific Errors: It filters the message trace details to find messages with the error "554 5.6.0 Corrupt message content."
5. Export Results: Finally, the script exports the filtered results to a CSV file.

Additional Tips and Considerations

• Execution Time: The script's execution time can vary significantly depending on the number of mailboxes in your Exchange Online tenant. For large organizations with many users, the script may take a considerable amount of time to complete. It's advisable to run the script during off-peak hours to minimize any potential impact on system performance.

• Error Handling: Ensure you have proper error handling in place. While the script is designed to handle common scenarios, unexpected issues may arise. Adding try-catch blocks can help you manage and log errors more effectively.

• Permissions: Make sure you have the necessary permissions to run the script. You need to have appropriate administrative rights to access mailbox and message trace data in Exchange Online.

• Resource Usage: Be mindful of resource usage, especially if you're running the script on a local machine. The script processes a large amount of data, which can be resource-intensive. Monitoring system performance during execution can help you identify any bottlenecks.

• Data Privacy: When exporting data to a CSV file, ensure that sensitive information is handled securely. Consider encrypting the CSV file or storing it in a secure location to protect user data.

• Script Customization: Feel free to customize the script to better suit your needs. You can modify the date range, filter criteria, or output format based on your specific requirements.

Conclusion

This script proved invaluable during the outage, allowing us to quickly identify and address the specific NDRs affecting our users. I hope you find it useful in your own environment. If you have any questions or suggestions, feel free to leave a comment below!