Tuesday, April 2, 2013

IIS ARR Suddenly Stop Working (HTTPS Website)

Received a call from customer today complaining that their web application suddenly stop working. Luckily I was on the way to the data center to settle other issue.

The web application consist of a web server running IIS (Internet Information Services) and a proxy server running IIS ARR (Application Request Routing). The web application which is accessible via HTTPS port 443 has been working fine all along. Nobody was accessing the server recently. No system patch, no Windows update, totally nothing. Nobody touch the system. But... it suddenly refused to work.

Preliminary investigation:

1. Try accessing the website locally on the web server, everything seems fine

2. Try accessing the website from external network, all traffic must pass through IIS ARR Proxy, the website in inaccessible

At first, I was thinking this must be IIS ARR Proxy Server issue. But it can't be, nothing changed. After messing around with the ARR configuration for some time without any result, server certificate issue come to my mind. I remember during the initial set up that I was facing some issue with the ARR due to the IIS HTTPS certificate.

This turn out to be the actual cause of the problem. After some checks, I found out that the Root Certification Authority Cert for the certificate that I use for my IIS HTTPS is indeed expire today. Replacing the certificate immediately solve the problem.

IIS ARR Reverse Proxy will not work for HTTPS website if there is a problem with the server certificate.

If you are:

- Running IIS ARR Reverse Proxy

- With HTTPS

- Website suddenly stop working or become inaccessible

Root Certification Authority Cert expiry may be the cause of the problem.

Solutions to the problem:

1. Click Start | Run
2. Type mmc and press Enter
3. Click File | Add and Remove Snap In
4. Select Certificate
5. Choose Computer Account and Next
6. Click Finish
7. Go to Trusted Root Certification Authority
8. Check the expiry date of the Root Certification Authority Cert that your IIS Cert is using
9. Replace the certificate if it has indeed expired

Note: I noticed that if your IIS Cert has expired but the Root Certification Cert is still valid, the website can still be accessible. The browser will display some warning though.

In my case, below are the exact steps that I did to solve my problem:

1. Go to the server running IIS ARR
2. Run IIS Manager
3. Create a certificate request, save the request file as certreq.txt
4. Google and download OpenSSL
5. Install OpenSSL
6. Run command prompt as administrator
7. Run: set OPENSSL_CONF=c:\[PATH TO YOUR OPENSSL DIRECTORY]\bin\openssl.cfg
8. Run: openssl genrsa -des3 -out cakey.pem 2048
9. Run: openssl req -new -key cakey.pem -x509 -days 1825 -extensions v3_ca -out ca.crt
10. Run: openssl x509 -req -days 365 -in certreq.txt -CA ca.crt -CAkey cakey.pem -CAcreateserial -out iis.cer
11. Close the command prompt
12. Now I have ca.cer and iis.cer
13. Run mmc, add snap-in and install ca.cer under your Trusted Root Certification Authority then delete the expired ca cert
14. Back to IIS Manager and complete the certificate request using iis.cer
15. Go to bindings and select the new cert as the HTTPS cert
16. Export back the certificate as iis.pfx
17. Go to the web server
19. Run mmc, add snap-in and  install ca.cer under your Trusted Root Certification Authority then delete the expired ca cert
20. Go to IIS Manager and import the IIS certificate using iis.pfx
21. Go to bindings and select the new cert as the HTTPS cert
22. Restart IIS and everything is working