Configure IIS and SMTP Virtual servers for relay with Microsoft 365 - Exchange Online - Issue and Troubleshooting

Hello All

Wishing you all a very happy new year 2024 and hopefully we will be having the great year ahead  interms  of health and knowledge

Today we are going to cover the below topics in this article and let us go through one by one

1. What is IIS SMTP virtual server
2. How to configure the IIS SMTP virtual server in windows server 2016
3. Error Code and Troubleshooting
4. Limitations

What is IIS SMTP virtual server?

The SMTP virtual server is automatically configured with default settings that enable it to accept local client computer connections and to process messages. You can either accept the default settings or change the SMTP virtual server configuration

How to configure the IIS SMTP virtual server in windows server 2016

Set up Exchange Online as an SMTP Relay Using Windows Server 2016

Step 1: Install Internet Information Services (IIS)

In Server Manager, select Add Roles.

On the Before you begin page in the Add Roles Wizard, select Next.

On the Select Installation Type page, select Role-based or Feature-based installation.

On the Select destination server page, choose Select a server from the server pool, and select the server that will be running SMTP services. Select Next.

On the Select Server Roles page, select Web Server (IIS), and then select Next. If a page that requests additional features is displayed, select Add Features and then select Next.

On the Select Role Services page, make sure that Anonymous under Security is selected, and then select Next.

On the Confirm Installation Steps page, select Install.

Step 2: Install SMTP Feature 

Open Server Manager and select Add Roles and Features.

Select Server Selection and make sure that the server that will be running the SMTP server is selected and then select Features.

On the Select Features screen, choose SMTP Server. You may be prompted to install additional components. If that’s the case, select Add Required Features and select Next.

Select Install. After the installation is finished.

Set the SMTP service to automatic, By default its Manual after installation

Step 3: Set up SMTP

Open Server Manager, select Tools, and then select Internet Information Services (IIS) 6.0.

Expand the current server, right-click the SMTP Virtual Server, and then select Properties.

On the General tab, select Advanced > Add.

In the IP Address box, specify the address of the server that’s hosting the SMTP server.

In the Port box, enter 25 and select OK.

On the Access tab, do the following:

Select Authentication and make sure that Anonymous Access is selected.

Select Connection > Only the List Below, and then specify the IP addresses of the devices that will be connecting to the SMTP server, such as printers.

Select Relay > Only the List Below, and then specify the IP address of the devices relaying through this SMTP server

On the Delivery tab, select Outbound Security, and then do the following:

Select Anonymous Authentication.

Select TLS Encryption.

Select Outbound Connections, and in the TCP Port box, enter 25 and select OK.

Select Advanced and specify domainname-com.mail.protection.outlook.com as the Smart Host.

Note : Replace cloudmonkeys-xyz.mail.protection.outlook.com with your own tenants mail exchanger address. 

You can find this value by looking for the MX record of your domain in the Setup section in the M365 admin portal.

How do you know this worked?

To test SMTP relay services, use the following steps.

Create a text file using Notepad or another text editor. The file should contain the following code. 

Replace the source and destination email addresses with the addresses you will use to relay SMTP

FROM: <source email address> TO: <destination email address> SUBJECT: Test email This is a test email sent from my SMTP server

Save the text file as Email.txt.

Copy the Email.txt file into the following folder: C:\InetPub\MailRoot\Pickup.

After a short time, the file should automatically be moved to the C:\InetPub\MailRoot\Queue folder.

 When the SMTP server delivers the mail, the file is automatically deleted from the local folder.

 Note : If the SMTP server can’t deliver the message, a non-delivery report (NDR) is created in the C:\InetPub\MailRoot\BadMail folder. You can use this NDR to diagnose delivery issues.

NON Delivery Report 

During my setup i have encounter the non delivery report in BADMail folder

Error Code

 Unable to deliver this message because the follow error was encountered: "Error is processing file in pickup directory.".

 The specific error code was 0xC00402CE.

Checked the errors in the google and found the email address was in wrong format. i made it in correct as below without any quotes or brackets 

Finally email got delivered

Troubleshooting

for any kind of troubleshooting , you may enable the below logging for root cause 

Limitations

 In this article is part of IIS 6.0, the support for which has ended with the support of Windows Server 2003

 To relay emails to Exchange OnPrem and/or Office 365, use one of the supported versions of Exchange Server, as its  no longer support doing so with IIS SMTP.