Why doesn't my XAP load on a cross-domain page? Probably because you're not setting the mime-type of the XAP to "application/x-silverlight-app".
We were also worried that sites allowing users to upload arbitrary content would now need to check for files that look like XAP. We really wanted to make sure that web developers and webmasters don't need to read all about Silverlight just to make sure it can't attack their site, even if they never plan to use it. By demanding to see the mime-type, Silverlight knows that the web server knows what a Silverlight application is.
There's more info here.