Step One: Turn off custom errors in your ASP.Net app. Usually this means editing web.config and settings <customErrors mod="Off" />. E.g.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
    <system.web>
         <customErrors mode="Off" />
    </system.web>
</configuration>

Step Two: Configure TAS to tell you more about the error. 

To do this you need to edit the file TasWorker.exe.config. If it exists under %Program Files%\Microsoft Speech Server\TAS then add this line:

      <add key="LogWebserverErrors" value="true" />

If the file doesn't exist then create it and make it look like this:

<configuration>

   <appSettings>

      <add key="LogWebserverErrors" value="true" />

   </appSettings>

</configuration>

Now restart TAS. Now when it tries to download a web page and gets a 500 error, it will include the body of the HTML page in a Trace statement (these go to the ETL file). Read it as it is or cut and paste it into and html file for readability.