I am sure we all would have encountered the “500 Internal Server Error” while trying to access many different Applications. It is an Error which can be caused due to many different reasons. I am going to discuss one such scenario here in this Post. I am sure this will be helpful in resolving similar issues.
So, we started our Troubleshooting by collecting our Favourite UAG trace on the UAG server while trying to reproduce the issue. You can get more information on how to collect and Analyse UAG trace from the Article below:
While Analysing the UAG trace we could see the below Error there for our Request:
As we can see above it looks that the UAG was having issues in comprehending the Response Headers coming from the Internal Web Server.
So, to check it further we took a Simultaneous Network Trace on the UAG and the Internal Web Server to see what kind of Response are we getting from there. As the Traffic between the UAG and Web Server was on Port 80, it could have been possible for us to look into the Network Captures. If it would have been SSL, it would not have been possible for us to look at it as it will all be Encrypted.
Then we looked at the Network Trace and specifically look for the Response to our Request and here is what we saw in the HTTP Headers:
As we can see a above in the HTTP Headers in the Response coming from the Web Server, we don't see the CRLF (Carriage-Return/Line-Feed) Pair at the End of the Headers. The CRLF Pair actually denotes the End/Termination of the Headers according to the HTTP RFC 2822, which means that the Server has sent All the Headers completely.
Please refer to the Section 2.1 in the RFC below to have a better understanding on CRLF Or Headers Termination:
As our Forefront UAG is completely RFC Compliant, the above behaviour shows that the Web Server is not Sending the Complete Headers and hence it throws the Error as shown in the Above Snippet from the UAG trace and as a result of that the Client gets the “500 Internal Server Error”.
Now, let me show you an Example of a Good Header Termination:
So, as we can see in the above Snippet, the HTTP Headers are getting Terminated with a Pair of CRLF. Also pay Attention to the HEX Details of CRLF, it is A pair of OD OA.
As we can clearly see here that the Web Server was not Terminating the Headers Correctly and Hence we had this issue.
We Fixed the Code of the Web Server to send the CRLF and Terminate the Headers properly while sending the Responses, as per the RFC. And after that we could Access the Application through UAG as well.
SUPPORT ESCALATION ENGINEER, FOREFRONT EDGE SECURITY, MICROSOFT