Overview
You will use the Works with Tool to:
Works with Tool is:
Server and Client components can be tested with the Works with Tool on 64-bit versions of:
The Works with Tool can now be launched in 9 different languages:
The current Works with Tool is installed with the Software Certification Toolkit from the Works with Windows Server landing page.
The test cases performed are outlined, in English, in the Works with Windows Server 2008 R2 Test Framework.
Prepare Test Environment
Prep clean virtual machines running on Hyper-V:
Prerequisite setup prior to testing:
Install and configure any application prerequisites, prior to testing. For example:
Configure each computer used for testing to allow for network discovery and sharing Public Folders.
Install the Works with Tool
Test Your Application
Ready to Begin Testing
Follow the simple guides of the Wizard throughout testing. Example: Name your Server or Client component, click the appropriate component type radio button, click Next, etc.
The Works with Tool will instruct you when to:
Note any yellow highlighted textboxes throughout wizard with important information regarding requirements, optional tests, and recommendations.
The Works with Tool has many uses and may show configuration status as a failure on the Prerequisite screen. Depending on level of testing being performed, the tool will allow you to continue test and ignore error warnings.
Focus of the following is the Works with Windows Server 2008 R2 Logo Program, and how to interpret your results. These are the same steps you will use as a developer during milestone testing or an IT Pro testing applications for compatibility on the platform.
The Works with Tool workflow is as follows:
When submitting test results for Windows Server 2008 R2 Logo, all component testing results must use the same language combination, the Prerequisite screen must show all status as green, and final report shows Status achieved of Works with Windows Server 2008 R2 as green
Start a New Test
Launch the Works with Tool:
Assure the Prerequisite screen status show as passing or are enabled for Windows Server 2008 R2 or Windows 7:
Note: If Windows Error Reporting is listed as Disabled, do not click Next. In Windows Taskbar, Start/Search, type services.msc and hit Enter. Click through any UAC to launch Services Manager. In Services list, Name column, locate Windows Error Reporting Service, and double click. Change Startup type: to Automatic or Manual. Click OK to close, then close Services Manager. On Prerequisite screen, click the F5 link to refresh.
Assure an Antivirus program is installed and running.
The Works with Tool performs all testing in the background, including monitoring the above prerequisites.
On Installer Information screen select either an MSI package, or setup executable. Alternately, if the application under test does not include an installer, you may select the appropriate radio button and include requested information.
At any time during the following screens, you may click View Report link to see current testing status and logs.
Important: To qualify for Works with Windows Server 2008 R2 Logo, the Works with Tool must be used.
The Works with Tool will inform when to install, test, and uninstall the application. Rebooting may be necessary. The Works with Tool will allow you to resume a test after reboot. Select Resume an existing test radio button, Select the proper test, and you will return to the last page prior to rebooting.
Between steps the tool is monitoring system state changes and some screens may take up to 15 minutes depending on speed, capacity, and resources of the system.
If your application contains drivers, you may be prompted to reboot to set up special driver testing during boot up and during primary functionality testing.
Client components: If the application contains client components, assure the Server components are operational, then follow the above steps on another machine to test Client components using Works with Tool. If client components were designed to run on a Server, then another Server can be used, otherwise run client components on Windows 7 64-bit. The client components will be used to exercise the Server components. Note that testing on a virtual machine is optional for clients running on Windows 7.
When testing is nearly complete, you will be presented with a series of quick, but important questions about the testing.
When testing is complete, a quick report will launch presenting the results of the individual component tests, and testing status achieved.
The Works with Report is a very detailed, comprehensive report that can provide a rich look into how your application interacts with the platform. This is what makes the Works with Tool valuable during milestone testing an application under development, or when testing in-house or 3rd party applications for compatibility on the platform you are adopting.
You may now package all results into a Submission package which includes the test results from the Server components and any Client components from another machine. This can be created from the Finish screen or by launching the Works with Tool and selecting the Create Submission Package option.
Create a Submission Package
Launch the Works with Tool, and select the same language used during all component testing.
Select the Create a Submission Package option:
Waiver Requests
In some failing test cases, there may be a limited waiver available. The Create a Submission Package workflow contains option to apply for waivers:
When all tests results in consolidated Submission Package are passing, or when granted waivers are returned by Microsoft, you may Prepare a Submission Package to send to Microsoft.
Prepare a Submission Package to send to Microsoft
This step will prepare a passing Submission Package to upload to the Microsoft Platform Ready website. The Submission Package will have all passing test results, or approved waivers from Microsoft.
Launch the Works with Tool and select the Prepare a submission package to send to Microsoft option:
You are now finished with the Works with Tool and can proceed to Upload the Prepared Submission Package to the Microsoft Platform Ready website.
When submission package has been validated by the Microsoft Platform Ready website, you will be granted the use of the Works with Windows Server 2008 R2 Logo artwork. See the Microsoft Platform Ready web site for benefits associated with the Logo Program.
Since the Works with program is one of self testing, you will be interpreting the test logs generated.
Help can be had on the dedicated MSDN Forum for Windows Server 2008 Application Compatibility and Certification issues. There you will find Windows experts, as well as a community of developers and Independent Software Vendors helping each other with the new operating system and Logo program requirements.
Below are the most common issues encountered when attempting to interpret your own logs. Most are common to many of the test case logs, so they do not bear repeating for each log. The Works with Tool has built in intelligence to automatically remove noise from the logs and allowed failures.
Interpreting log failures:
3rd party files
Definition: Any file not part of your build process, and not owned by your Product group or Company. If these files do not have signatures or valid file properties, these files will fail the optional test cases, but will NOT block achieving Works with Logo. Do not sign files that do not belong to your Product group or Company. Do not assign file properties to files that do not belong to your Product Group or Company. Document, no waiver required. Documentation must include all 3rd party file names and owners which appear as failing the test cases. Documentation must be publicly facing for the use and convenience of your Customers: ReadMe, FAQ, Product manual or webpage Publicly facing documentation links may be included in the waiver details.
Definition: Any file not part of your build process, and not owned by your Product group or Company.
If these files do not have signatures or valid file properties, these files will fail the optional test cases, but will NOT block achieving Works with Logo.
Do not sign files that do not belong to your Product group or Company.
Do not assign file properties to files that do not belong to your Product Group or Company.
Document, no waiver required.
Documentation must include all 3rd party file names and owners which appear as failing the test cases.
Documentation must be publicly facing for the use and convenience of your Customers: ReadMe, FAQ, Product manual or webpage
Publicly facing documentation links may be included in the waiver details.
Temporary files
Files in all temp folders that are expected to be deleted by Windows can be safely ignored. Temp folders: Temp IIS, ASP, ASP.NET folders (transient files, or created on the fly, and expected to be deleted) Temp .NET folders Temp Java folders Temp Windows Installer folders (example: x:\Windows\Installer, many icon files appear here as 16 bit executables.) Any temp files not automatically ignored by the Works with Tool may fail the test case, so must be documented in waiver details as temporary files on which the application does not have a dependency. Limited waiver available for failing test cases based on temporary files.
Files in all temp folders that are expected to be deleted by Windows can be safely ignored.
Temp folders:
Any temp files not automatically ignored by the Works with Tool may fail the test case, so must be documented in waiver details as temporary files on which the application does not have a dependency. Limited waiver available for failing test cases based on temporary files.
Interop files
Interop files do not contain file properties, by design. This known by Microsoft, no documentation required, and ignored by the Works with Tool. If built and owned by Product group or Company, Interop files should be signed as a best practice, but will not fail the Works with test.
Interop files do not contain file properties, by design.
This known by Microsoft, no documentation required, and ignored by the Works with Tool.
If built and owned by Product group or Company, Interop files should be signed as a best practice, but will not fail the Works with test.
Files using known binary extensions
Some (text, log) files used by application using known binary extensions must document these extensions. (Example, .sys, .bat, .com, etc.) For instance, a text log file extension of .com may be improperly labeled a 16 bit binary. Provide technical justification for use of known binary extensions for text based logs in waiver details.
Some (text, log) files used by application using known binary extensions must document these extensions. (Example, .sys, .bat, .com, etc.)
For instance, a text log file extension of .com may be improperly labeled a 16 bit binary.
Provide technical justification for use of known binary extensions for text based logs in waiver details.
Internal Consistency Evaluators (ICE) Errors or Warnings in MSI packages
ALL ICE Errors appearing on logs must be fixed. See ICE Error reference MSDN page for guidance on fixing. ICE Warnings do not require a fix. All Warnings should be investigated, and as a best practice, should be fixed. These are the ICE errors being validated in Works with Tool: (ICE) 1-2, 4-7, 9-15, 17-24, 27-29, 31, 33-36, 38, 40-42, 44-56, 59, 61-63, 65, 67-71, 74-78, 81-84, 86-87, 89-94, 96-105
ALL ICE Errors appearing on logs must be fixed.
See ICE Error reference MSDN page for guidance on fixing.
ICE Warnings do not require a fix. All Warnings should be investigated, and as a best practice, should be fixed.
These are the ICE errors being validated in Works with Tool:
ICE 27 Error - Regarding MsiConfigureServices for MSI 5.0
Issue: ISV receives ICE Error 27 when validating their MSI package:
ICE27 ERROR Unknown action: 'MsiConfigureServices' of InstallExecuteSequence table. Not a standard action and not found in CustomAction or Dialog tables
ISVs may safely ignore this ICE 27 Error.
Background: MsiConfigureServices is a new “standard action” starting with MSI 5.0. Adding it to the InstallExecuteSequence table with the condition (VersionMsi >=”5.00”) will cause it to execute only on versions of MSI that support it.
As MsiConfigureServices is a standard action, it is allowable in the InstallExecuteSequence Table.
Known issue by MSI Team: No KB article at this point.
Package Identity
Windows Installer packages must comply with ICE validation as well as identify installer package and properly prepare for upgrade.
To Pass this test case the Property Table and the Upgrade Table must contain:
Property Table: Manufacturer ProductCode ProductLanguage ProductName ProductVersion (major and minor) Upgrade Table: UpgradeCode VersionMin (Major.minor.build) VersionMax (Major.minor.build) The UpgradeCode in the Upgrade table must be identical to the UpgradeCode in the Property table. VersionMin and VersionMax: BOTH cannot be null.
Property Table:
Upgrade Table:
The UpgradeCode in the Upgrade table must be identical to the UpgradeCode in the Property table.
VersionMin and VersionMax: BOTH cannot be null.
Custom Actions
Custom table starting with MSI : MsiSFCBypass or MsiDriverPackage These can be safely ignored. No Waiver required. These are allowed MSI Prefixed Standard Tables.
Custom table starting with MSI : MsiSFCBypass or MsiDriverPackage
Custom Columns
Custom columns must not be added to standard tables. MsiPatchOldAssemblyName This is a common Logo error based on older InstallShield versions. Here is the optional workaround which is accepted by the Logo Program. From InstallShield support: In the MsiPatchOldAssemblyName table, change the Assembly_ column to read 'Assembly' In the _Validation table, locate the record for the Assembly_ column for the MsiPatchOldAssemblyName Table, and remove the trailing underscore Save the project.
Custom columns must not be added to standard tables.
This is a common Logo error based on older InstallShield versions. Here is the optional workaround which is accepted by the Logo Program.
From InstallShield support:
16 bit files found
Some installer programs create an icon file with an .exe extension. These may be flagged as failures, and appear as 16 bit files to Work with Tool. If these are found in the C:\Windows\Installer directory or similar User redirected folder, these are considered to be installed into a temporary directory and will be automatically ignored by the Works with Tool. Be sure you do not own any files you list as 3rd party.
Some installer programs create an icon file with an .exe extension. These may be flagged as failures, and appear as 16 bit files to Work with Tool.
If these are found in the C:\Windows\Installer directory or similar User redirected folder, these are considered to be installed into a temporary directory and will be automatically ignored by the Works with Tool.
Be sure you do not own any files you list as 3rd party.
No Installer, No Windows executables
The Windows Server Logo Program will evaluate on a case by case basis any application which has no installer, and installs no Windows executables.
The Logo Program requires a Waiver be filed for failure in the Works with 2008 R2 Report labeled: “Were any binaries installed for this Component?”
The ISV must document: All major components installed Their interaction with specific hosted application(s) or Windows feature(s) Any other prerequisite software installed and versions Submit these details in waiver to wslogofb@microsoft.com
The ISV must document:
With no other failures in the Works with logs, there should not be any issues in achieving the Works with Logo.
The Logo Program will only review waivers containing above details and respond with Microsoft approval. Incomplete details will results in delayed waiver and logo process.
Waivers
There will be very limited Waivers granted.
Examples: Logo Tool errors, Windows limitation, or technical documentation exists why no other workaround existed for exemption (very limited).
When Waiver is necessary, technical details must be sent to Microsoft for pre-approval, using the Works with Tool waiver workflow, prior to sending submission package to Microsoft Platform Ready website.
Waivers are not required when only documentation is sufficient to detail test case issues as mentioned above. If the test case is marked as a failure, then use the waiver process to provide details and publicly facing documentation links.
Hope the above is helpful,
-PaulS