Check If the Applications Install and Run Correctly in an Isolation Environment
1. Create an isolation environment for each application.
2. Install each application into separate isolation environments.
3. Test the applications from within the isolation environments.
4. Run both applications from valid user accounts and verify that there are no conflicts.
5. Determine if the applications exhibit problems when accessed by multiple users.
If the applications install and run correctly, then they were successfully isolated. If the problem is not resolved, investigate further using the guidelines below:
Advanced Workflow
If you are still experiencing application compatibility issues, follow the troubleshooting workflow described below.
Check for System Services
Applications that install new services will not work correctly in an isolation environment. To determine if an application did attempt to install a service, follow steps below:
Open the Windows Event Viewer and review the Applications section for CtxSbxAppMsg messages related to Services.
If the application installed a service or attempted to delete a service, then do not isolate this application.
If the application did not attempt to install or delete a service, proceed to the next step in the workflow.
Check for Named Objects Issues
Named objects are used for inter-process communications. These named objects need to be isolated, but compatibility issues can occur when some named objects need to be shared between isolated and non-isolated processes. To check if named objects are causing the issue, follow steps below:
1. In the Management Console, right-click the isolation environment into which you installed the application and select Properties.
2. In the Properties window, select Rules.
3. In the Rules pane, click Add to create a new rule. Create an Ignore All rule to switch off Named Objects isolation. Typically, such a rule is used to identify any communications with system resources outside the isolation environment. An example is the communication between Setup.exe and the MSI service using named pipes.
4. Test the isolated application with the new rule.
5. If the application works correctly, then attempt to narrow down the set of named objects that you need to ignore. You can use CDF logging to identify named objects which are causing problems.
6. Using your CDF viewer, run a log of the system object code. Specifically, you need to log GUID dab267b2-296f-4042-9271-549f4d920855 MF_Hook_CtxSbx with an access mask of 0x2000, and a level of 3. This provides you with a list of object names that were modified by application isolation. You can then further narrow down this list to determine which object is causing the problem.
7. Run Process Explorer to determine which objects are being used by the isolation environment process.
8. Now that you have a list of Named Objects, add specific ignore rules for the isolation environment.
Note: You can use an asterisk (*) as a wildcard character only at the end of an ignore named object rule. For example, the rule “ignore object*” ignores all named objects with a name starting with ‘object’.
9. Pay particular attention to pipes and section objects as these are often used for communications between applications inside the isolation environment and systems services outside the isolation environment.
If ignoring named objects does not resolve the application issues, proceed to the next section.
Check for File System Issues
Most of the file system is isolated by default; however you may need to add additional rules that allow an isolated application to access files updated by non-isolated processes.
You may need to add additional rules for the file system, if isolated applications exhibit any of the following issues:
• Isolated applications have stale data. For example, applications have shared template files.
• Isolated applications crash or hang.
• Isolated applications report errors. Typically these are “DLL not found” errors.
To diagnose file system related problems, do the following:
1. Install the FileMon tool (shareware utility available from http://www.sysinternals.com/) to help diagnose faulty file system behavior. FileMon monitors and displays file system activity on a system.
2. Capture a FileMon trace of the application execution when it is run within an isolation environment, as well as when it is run directly on the system.
Experienced trouble-shooters may find the Advanced Output option in FileMon useful.
3. Compare the two traces to help identify deviations in file system activity by the isolated application process. Important points to note are:
• An isolation environment always fails FAST_IO_QUERY_OPEN requests for isolated applications.
• Sometimes Filemon path strings for isolation environment issued requests will have non-printable character labels prefixed to them; these appear as “???...” These labels are used internally by isolation environments.
4. Identify any files or folders that need to be ignored using rules. You can start by enforcing Ignore rules for applications associated with an isolation environment; when the problem is identified, you can refine the rule set to a narrower scope. For example, if an isolated application needs a particular version of a system DLL rather than the version available in the isolation environment, you can create an Ignore rule which specifically ignores the version of the DLL inside the isolation environment.
5. Identify issues related to environment variables such as %PATH%, %WINDIR%. To view settings for these variables as visible to an isolated application, launch a command prompt within an isolation environment by typing the following command:
AIERUN Isolation_Environment_Name cmd.exe
If the value of the PATH variable is incorrect, you can create a batch file that correctly sets the path variable and then launches the isolated application. You can publish the batch file instead of the isolated application executable. This batch file can then be published instead of the original application executable.
6. It may be possible to identify issues related to the file system by viewing the file system as visible to an isolated application. To view the file system within an isolation environment, launch a file explorer associated with the isolation environment as follows:
AIERUN Isolation_Environment_Name c:\program files\Internet Explorer\Iexplore –e
This allows you to view the file system as visible to an isolated application. If a particular required directory is not visible because it was previously deleted in the isolation environment, run Windows Explorer outside the isolation environment and clear the User Profile Root to remove the appropriate deleted record.
7. By default the %USERPROFILE% directory is ignored. However, in some cases, a version of an isolated application may write configuration information to this common area. This could affect the functioning of other isolated versions of that application. In this case, remove this ignore rule and determine if it corrects the problem. Subsequently, attempt to narrow down the solution with more specific isolate rules while restoring the original ignore rule.
For example, if you are running multiple versions of Microsoft Outlook, create an Isolate rule for %USERPROFILE%\Application Data\Microsoft.
Important: Clear contents of the User Profile Root and the Installation Root and reinstall/rerun the application every time you modify a rule. This removes stale data in these locations.
If the compatibility issue you are experiencing is not related to the file system, proceed to the next section.
Check the Registry
Applications store configuration information in the system registry. Most of the registry is isolated by default, however you may need to add additional rules that allow an isolated application to access configuration data updated by non-isolated processes.
You may need to add additional rules for the registry, if isolated applications exhibit any of the following issues:
• Isolated applications have stale data. For example, ICA printers are not found.
• Isolated applications crash or hang.
• Isolated applications report errors.
To diagnose registry related problems, do the following:
1. Install the Regmon tool (shareware utility available from http://www.sysinternals.com/) to help diagnose registry configuration errors.
Regmon is a Registry monitoring utility that will show you which applications are accessing your registry, which keys they are accessing, and the registry data that they are reading and writing.
2. Capture a Regmon trace of the application execution when it is run within an isolation environment, as well as when it is run directly on the operating system.
3. Review the trace to determine which keys are accessed during installation or execution.
4. Determine if certain registry keys require Ignore rules.
Check the Regmon log to see if registry keys that were accessed by the isolated application, were also accessed by non-isolated services or helper applications. If these non-isolated components write to keys that were isolated during installation or execution, then you may get stale data in the isolation environment, requiring ignore rules to be added.
In such a situation, adding an ignore rule enables the isolated application to access data written by non isolated applications or services successfully.
An example of such behavior is the HKCU\Printers key. This key is updated with information about client printers. If this key is not ignored, then isolated applications are unable to get updated information about which printers are available for a given session.
Install and/or run the application again to verify that the ignore rule works as expected.
5. Determine if certain registry keys require Isolate rules. Check the Regmon log to see if registry keys which are ignored are shared between users of an application or between multiple applications. Compatibility problems can occur if you have ignored rules that are too wide in scope.
For example, if you have an ignore rule for the registry key, HKCU\Software\Microsoft\Windows NT\CurrentVersion, every key under this key is ignored. Microsoft Outlook user information is stored under this key in HKCU\Software\Microsoft\Windows NT\CurrentVersion\Windows Messaging Subsystem.
To resolve any issues which may arise with this profile information, you can add an isolate per user rule for HKCU\Software\Microsoft\Windows NT\CurrentVersion\Windows Messaging Subsystem.
Important: Clear contents of the User Profile Root and the Installation Root and reinstall/rerun the application every time you modify a rule. This gets rid of stale data in these locations.
