Exchange 2019 Hybrid Lab:
Rebuilding the Edge Transport Server
A Complete Troubleshooting &
Recovery Guide
April 2026 | Exchange 2019 CU15 | Home
Lab Edition
Lab Environment Overview
This guide documents the
complete troubleshooting journey of an Exchange 2019 Hybrid lab environment
that experienced mail flow failures due to Edge Transport server ADAM database
corruption. The lab consists of:
|
Component |
Server Name |
Role |
|
Domain Controller |
CMDC01 |
Active Directory, DNS |
|
Hybrid Mailbox Server |
CMHYBD01 |
Exchange 2019 Mailbox + HCW |
|
Edge Transport Server |
CMEDGE01 |
Exchange 2019 Edge
(192.168.1.25) |
|
Azure AD Connect |
CMHYBD01 |
Identity Sync to EXO |
|
Exchange Online |
cloudmonkeys.xyz |
Microsoft 365 Tenant |
Initial Symptoms
After running the Hybrid
Configuration Wizard (HCW) successfully and confirming connectors were created,
two mail flow problems were observed:
•
Inbound mail (Exchange Online → On-Premises): Mail
reached the Edge server but was not being delivered to the Mailbox server
•
Outbound mail (On-Premises → Exchange Online or
self-send): Mail was stuck in Drafts and never entered the transport queue
Investigation & Diagnosis
Step 1 — Transport Services and Queue Check
All Exchange transport services
on the Mailbox server were confirmed running. The mail queue showed only the
Submission queue with 0 messages, confirming mail was not even being submitted.
Get-Queue
Identity:
cmhybd01\Submission Status: Ready MessageCount: 0
Step 2 — Send Connector Analysis
Two critical issues were found
in the Send Connectors:
|
🔴 Error: Wrong
SmartHost on Outbound to Office 365 SmartHosts:
{mail.cloudmonkeys.xyz} ← Pointing to own domain MX, causing mail loop |
|
🔴 Error: Address
Space Cost Conflict EdgeSync - Inbound:
smtp:*;1 Outbound to Office 365: smtp:*;1 Both connectors have identical cost
— routing ambiguity |
Step 3 — EdgeSync Failure
Running Test-EdgeSynchronization
revealed the core problem:
|
🔴 Error: EdgeSync
LDAP Failure SyncStatus: Failed
FailureDetail: EdgeSync service cannot connect to this subscription because
of error "The LDAP server is unavailable." CredentialRecords:
Number of credentials 0 |
Step 4 — Port Connectivity Testing
All ADAM ports were confirmed
open from the Mailbox server to the Edge server. TCP connections to ports 50389
and 50636 succeeded, ruling out firewall as the cause.
Test-NetConnection
-ComputerName cmedge01.cloudmonkeys.xyz -Port 50389 →
TcpTestSucceeded: True
Test-NetConnection
-ComputerName cmedge01.cloudmonkeys.xyz -Port 50636 →
TcpTestSucceeded: True
Step 5 — ADAM Database Corruption Confirmed
Querying the ADAM database
directly returned an operations error, confirming the Exchange configuration
partition was missing or corrupt:
|
🔴 Error: ADAM
Database Inaccessible [ADSI]"LDAP://localhost:50389/CN=MSExchangeTransportService..."
Error: "An operations error occurred." |
|
💡 Root Cause
Identified The ADAM (AD LDS) database on
the Edge server was corrupt — the Exchange configuration partition was
missing entirely. This meant EdgeSync had no valid credential store, causing
the LDAP unavailable error despite ports being open. This is commonly caused
by VM snapshot/restore operations or improper service shutdowns. |
Errors Encountered During Recovery
The recovery process involved
multiple attempts, each producing specific errors that required individual
resolution. Below is a complete log of every error and its fix.
Error 1 — RecoverServer Mode Fails on Edge
|
🔴 Error: Server not
found in Active Directory E:\Setup.exe
/Mode:RecoverServer Result: "Server is not found in Active Directory.
Server must be present in Active Directory to recover." |
|
✅ Fix: Use Install mode
instead RecoverServer only works
for domain-joined servers. Edge servers are never domain-joined. Correct
command: Setup.exe /Mode:Install /Roles:EdgeTransport |
Error 2 — ADAM Service Fails After Database Deletion
After manually deleting the
corrupt ADAM database files (.dit, .log, .chk), ADAM refused to start:
|
🔴 Error: ADAM
Cannot Reinitialize EventID 7023: The
Microsoft Exchange ADAM service terminated with error: %%3221225487
(0xC000000F = The system cannot find the file specified) |
|
✅ Fix: ADAM cannot
self-reinitialize from scratch Unlike a full AD DS,
ADAM/AD LDS requires schema LDIF files to bootstrap. Solution: Use
adamuninstall.exe to remove the instance and adaminstall.exe to recreate it. |
Error 3 — Exchange Setup Uninstall Blocked by ADAM
|
🔴 Error: Setup
prerequisite check fails Setup.exe
/Mode:Uninstall Error: Service 'ADAM_MSExchange' failed to reach status
'Running' Setup requires ADAM to be running in order to uninstall it —
chicken and egg problem. |
|
✅ Fix: Use
adamuninstall.exe directly C:\Windows\ADAM\adamuninstall.exe
/instancename:MSExchange /unattend This bypasses Exchange setup and removes
the ADAM instance directly. |
Error 4 — AdamInstallAnswer.ini Has Empty Placeholders
|
🔴 Error: ADAM
reinstall does nothing silently C:\Windows\ADAM\adaminstall.exe
/answer:"...\AdamInstallAnswer.ini" /quiet Result: No files
created, service still stopped. Cause: Answer file has blank values for
InstanceName, ports, and paths. |
|
✅ Fix: Populate the answer
file with correct values from registry Registry path:
HKLM\SYSTEM\CurrentControlSet\Services\ADAM_MSExchange\Parameters Required
values: InstanceName=MSExchange LocalLDAPPortToListenOn=50389 LocalSSLPortToListenOn=50636 DataFilesPath=C:\Program
Files\Microsoft\Exchange Server\V15\TransportRoles\Data\Adam
NewApplicationPartitionToCreate=CN=Configuration,CN={GUID} |
Error 5 — Setup Fails: Missing Registry Keys
|
🔴 Error: Registry
key does not exist Error: The registry key
"SOFTWARE\Microsoft\ExchangeServer\v15\Transport" does not exist
under HKEY_LOCAL_MACHINE Error: The registry key
"SOFTWARE\Microsoft\ExchangeServer\v15\Pickup" does not exist |
|
✅ Fix: Create all required
registry keys before running setup reg add
"HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\Transport" /f reg add
"HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\Pickup" /f reg add
"HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\Replay" /f reg add
"HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\QueueDatabase" /f reg
add "HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\EdgeTransport" /f
reg add "HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\Gateway" /f
(and more — see complete list in recovery steps below) |
Error 6 — ASDat.MSI Installation Fails (Error 1603)
|
🔴 Error: Anti-spam
MSI package corrupt Installing product
ASDat.MSI failed. Fatal error during installation. Error code is 1603. Last
error: 'An installation package for the product Microsoft Exchange 2007
Standard Anti-spam Filter Updates cannot be found.' |
|
✅ Fix: Clean MSI
registration and pre-install from ISO 1. Remove broken MSI
registry entries from
HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall 2. Copy fresh MSIs
from ISO: E:\Setup\ServerRoles\TransportRoles\agents\Hygiene\ 3. Pre-install:
msiexec.exe /i ASDat.MSI ALLUSERS=1 /qn 4. Also pre-install: ASEntIRS.MSI and
ASEntSig.MSI |
Error 7 — ASEntIRS.MSI Fails During Setup GUI
|
🔴 Error: Enterprise
Block List MSI fails Installing product
ASEntIRS.MSI failed. Fatal error. Error code 1603. Setup shows dialog: 'The
installed product does not match the installation source' |
|
✅ Fix: Pre-install all
three Hygiene MSIs before running setup @('ASDat.MSI','ASEntIRS.MSI','ASEntSig.MSI')
| ForEach-Object { Start-Process
msiexec.exe -ArgumentList "/i `"$destPath\$_`" ALLUSERS=1
/qn" -Wait } |
Complete Recovery Steps — Clean Rebuild
The following is the definitive
step-by-step procedure for rebuilding an Exchange 2019 Edge Transport server on
an existing Windows installation where the previous Edge was broken.
Phase 1 — Mailbox Server Cleanup
Run these commands on the
Mailbox server (CMHYBD01) before touching the Edge server:
1. Remove
the Edge Subscription (also removes AD object automatically)
Remove-EdgeSubscription
-Identity cmedge01 -Confirm:$false
2. Remove
orphaned EdgeSync Send Connectors
Remove-SendConnector
"EdgeSync - Default-First-Site-Name to Internet" -Confirm:$false
Remove-SendConnector
"EdgeSync - Inbound to Default-First-Site-Name" -Confirm:$false
3. Verify
only legitimate connectors remain
Get-SendConnector | Select
Name, Enabled
Phase 2 — Edge Server Registry Cleanup
Run on the broken Edge server to
clean up all Exchange service registrations:
# Stop all Exchange
services
Get-Service | Where-Object
{$_.Name -like "*MSExchange*" -or $_.Name -like "*ADAM*"} |
Stop-Service -Force -ErrorAction SilentlyContinue
# Remove all Exchange
service registry entries
$services =
@("ADAM_MSExchange","MSExchangeADTopology","MSExchangeAntispamUpdate",
"MSExchangeDiagnostics","MSExchangeFrontEndTransport","MSExchangeHM",
"MSExchangeHMHost","MSExchangeHMRecovery","MSExchangeMitigation",
"MSExchangeServiceHost","MSExchangeTransport","MSExchangeTransportLogSearch",
"MSExchangeEdgeCredential")
foreach ($svc in
$services) { reg delete "HKLM\SYSTEM\CurrentControlSet\Services\$svc"
/f 2>$null }
# Remove Exchange software
registry keys
reg delete
"HKLM\SOFTWARE\Microsoft\ExchangeServer" /f
# Reboot
Restart-Computer -Force
Phase 3 — Pre-install Required Registry Keys
After reboot, create all
registry keys that Exchange setup requires before running the installer:
$keys = @(
"HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\Transport",
"HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\Pickup",
"HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\Replay",
"HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\QueueDatabase",
"HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\EdgeTransport",
"HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\Setup",
"HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\Gateway",
"HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\AdminTools",
"HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\Diagnostics",
"HKLM\SOFTWARE\Microsoft\ExchangeServer\v15\TransportRoles"
)
foreach ($key in $keys) {
reg add $key /f }
Phase 4 — Pre-install Hygiene MSI Packages
Pre-install the anti-spam MSI
packages from the Exchange ISO to avoid setup failures:
# Find the correct ISO
path
Get-ChildItem E:\ -Recurse
-Filter "ASDat.MSI" -ErrorAction SilentlyContinue
# Typically:
E:\Setup\ServerRoles\TransportRoles\agents\Hygiene\
# Remove any broken MSI
registrations
$uninstallKeys =
Get-ChildItem
"HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall" |
Where-Object { (Get-ItemProperty $_.PSPath
-ErrorAction SilentlyContinue).DisplayName -like "*Exchange 2007*" }
$uninstallKeys |
ForEach-Object { Remove-Item $_.PSPath -Force -Recurse -ErrorAction
SilentlyContinue }
# Copy fresh MSIs from ISO
$isoPath =
"E:\Setup\ServerRoles\TransportRoles\agents\Hygiene"
$destPath =
"C:\Program Files\Microsoft\Exchange
Server\V15\TransportRoles\agents\Hygiene"
Copy-Item
"$isoPath\ASDat.MSI" $destPath -Force
Copy-Item
"$isoPath\ASEntIRS.MSI" $destPath -Force
Copy-Item
"$isoPath\ASEntSig.MSI" $destPath -Force
# Pre-install all three
@("ASDat.MSI","ASEntIRS.MSI","ASEntSig.MSI")
| ForEach-Object {
Start-Process msiexec.exe -ArgumentList
"/i `"$destPath\$_`" ALLUSERS=1 /qn" -Wait
}
Phase 5 — Run Exchange Edge Transport Setup
E:\Setup.exe /Mode:Install
/Roles:EdgeTransport /IAcceptExchangeServerLicenseTerms_DiagnosticDataOFF
After setup completes, reboot
when prompted, then verify:
Get-Service
ADAM_MSExchange, MSExchangeTransport | Select Name, Status
# Both should show:
Running
Phase 6 — Create and Import Edge Subscription
On the Edge server, generate the
subscription file:
New-EdgeSubscription
-FileName "C:\EdgeSub_Fresh.xml" -Force
Get-Item
"C:\EdgeSub_Fresh.xml" | Select Name, Length # Must be several KB
# Copy to Mailbox server
Copy-Item
"C:\EdgeSub_Fresh.xml"
"\\192.168.1.13\C$\EdgeSub_Fresh.xml"
On the Mailbox server, import
and synchronize:
New-EdgeSubscription \
-FileData ([byte[]]$(Get-Content -Path
"C:\EdgeSub_Fresh.xml" -Encoding Byte -ReadCount 0)) \
-Site "Default-First-Site-Name" \
-CreateInternetSendConnector $true \
-CreateInboundSendConnector $true \
-Confirm:$false
Start-Sleep -Seconds 60
Start-EdgeSynchronization
-Server CMHYBD01
Start-Sleep -Seconds 30
Test-EdgeSynchronization
-FullCompareMode
|
💡 Success Criteria SyncStatus: Normal
CredentialRecords: Number of credentials 2 (must be non-zero)
RecipientStatus: Synchronized TransportServerStatus: Synchronized |
Phase 7 — Fix Send Connector SmartHost
After EdgeSync is healthy, fix
the Outbound to Office 365 connector:
# Fix SmartHost - must
point to EXO inbound endpoint, NOT your own domain MX
Set-SendConnector
"Outbound to Office 365" -SmartHosts
"cloudmonkeys-xyz.mail.protection.outlook.com"
# Scope the connector to
only your domain (recommended)
Set-SendConnector
"Outbound to Office 365" -AddressSpaces
"smtp:cloudmonkeys.xyz;1"
Key Learnings & Best Practices
|
Lesson |
Detail |
|
Never delete ADAM .dit
manually |
ADAM cannot
self-reinitialize. Always use adamuninstall.exe to remove and adaminstall.exe
with a populated answer file to recreate. |
|
RecoverServer doesn't work
for Edge |
Edge is not domain-joined,
so /Mode:RecoverServer always fails. Use /Mode:Install /Roles:EdgeTransport
instead. |
|
CredentialRecords: 0 =
broken subscription |
If EdgeSync shows 0
credentials, the bootstrap never completed. Re-subscription is the only fix. |
|
LDAP unavailable ≠ firewall
issue |
Ports can be open
(TcpTestSucceeded: True) while ADAM still rejects connections due to auth or
DB corruption. |
|
Pre-install MSIs before
Edge reinstall |
ASDat.MSI, ASEntIRS.MSI,
ASEntSig.MSI must be pre-installed when reinstalling on an existing Exchange
installation. |
|
SmartHost must be EXO
endpoint |
Outbound to Office 365
SmartHost must be tenant.mail.protection.outlook.com, NOT your own domain MX
record. |
|
Address space costs must
not conflict |
Two connectors with
smtp:*;1 create routing ambiguity. Scope EXO connector to your domain only. |
|
Remove-EdgeSubscription
cleans AD automatically |
Running
Remove-EdgeSubscription on the Mailbox server automatically removes
CN=cmedge01 from Active Directory. |
— End of Document —
Comments
Post a Comment