Document scope: The following document is intended to guide a Citrix customer utilizing On-Premise Citrix Virtual Apps / Desktops through collecting data to determine root cause behind one or more of these common issues:
1. Application/Desktop Enumeration Failure
2. Application/Desktop Launch Failure
3. VDA Registration Failure
Note: This is not a troubleshooting guide. Instead, this is a guide that describes how to collect a broad, ideal spectrum of data to assist in determining root cause behind 3 common, high impact problems in customer environments.
Disclaimer: Although the data sets described here are intended to be comprehensive, the specific nature of the issue or environment may require multiple or more targeted data captures depending on the particulars of the situation.
Instructions
Definitions: 1. Application/Desktop Enumeration Failure – This issue is defined as failure to enumerate published Applications or Desktops after login to the Access Layer (typically Citrix Storefront, Workspace or Workspace App) by a user. It has been confirmed that the user has the appropriate security and configuration to where they should be seeing their Apps and Desktops but are not on either a constant or intermittent basis.
2. Application/Desktop Launch Failure – This issue is defined as a failure to launch some or all published Applications or Desktops available to the user. Typical errors are “Cannot Start App”, “Cannot Start Desktop” or “Unknown Client Error 1110”. It has been confirmed that the resource layer VDAs providing the Apps or Desktops are online and registered.
3. VDA Registration Failure – This issue is defined as a VDA failing to register with any Controller (DDC) in the Site. It has been confirmed that the VDA has been properly configured with a
method to find the DDCs and that at least one of the configured DDCs is online and functional.
Complex environments: In deployments with multiple load balanced or redundant components such as Storefront Servers, CVAD Controllers or Citrix Gateways, consider reducing these components to one element each for the purposes of testing/data capture. This will simplify data collection, data analysis as well as potentially help rule out a single faulty element.
Data space requirements: The primary data capture method is multiple sequential with no ring buffer. As such closely monitor disk space on all systems being traced to ensure drive space is not filled during data capture. The location for CDF Tracing log files is indicated in CDFControl under Tools > Options > Log File path. If circular/ring buffer tracing is desired see the addendum below on Circular Tracing.
Issue 1 – Application / Desktop Enumeration FailureContext Required: Test Username:
Time of Login where enumeration failed:
Specific examples of Apps or Desktops that should have enumerated but did not:
FQDN/IP of Storefront Server(s):
FQDN/IP of Controller / DDC:
FQDN/IP/vServer name of Citrix Gateway (if in use):
Ideal Data Capture: Storefront Server(s)
o Application
o System
o Citrix Delivery Services (under Applications and Services)
CVAD Controller/DDC(s):
These data sets should be captured on DDCs that are specified in Storefront for the site that has the enumeration problem. There is no need to capture data sets for an enumeration problem on DDCs that are not being used for enumeration.
•
CDF Trace – All Modules, multiple sequential logs, MaxLog 100MB, no Ring Buffer
•
Wireshark Trace
• Event Viewer Logs – Export as CSV and as EVTX (2 sets)
o Application
o System
Citrix ADC Gateway(s) – if in use:
Issue 2 – Application / Desktop Launch Failure
Context Required:
Test Username:
Time of attempt where launch failed:
Display name of the App or Desktop that should have launched but did not:
FQDN/IP of Storefront Server(s):
FQDN/IP of Controller / DDC:
FQDN/IP/vServer name of Citrix Gateway (if in use):
FQDN/IP of VDA:
FQDN/IP of Endpoint:
Ideal Data Capture:
Storefront Server(s)
o Application
o System
o Citrix Delivery Services (under Applications and Services)
CVAD Controller/DDC(s) - All:
•
CDF Trace – All Modules, multiple sequential logs, MaxLog 100MB, no Ring Buffer
•
Wireshark Trace• Event Viewer Logs – Export as CSV and as EVTX (2 sets)
o Application
o System
Citrix ADC Gateway(s) – if in use:
VDA:
•
CDF Trace – All Modules, multiple sequential logs, MaxLog 100MB, no Ring Buffer
•
Wireshark Trace
• Event Viewer Logs – Export as CSV and as EVTX (2 sets)
o Application
o System
Endpoint:
Issue 3 – VDA Registration Failure
Context Required:
Time of attempt where Registration failed (Restart Citrix Desktop Service on the VDA):
FQDN/IP of Controller(s) / DDC(s):
FQDN/IP of VDA:
Describe the method to find the DDCs:
Ideal Data Capture:
CVAD Controller/DDC(s) - All:
•
CDF Trace – All Modules, multiple sequential logs, MaxLog 100MB, no Ring Buffer
•
Wireshark Trace
• Event Viewer Logs – Export as CSV and as EVTX (2 sets)
o Application
o System
VDA:
o Application
o System
Addendum: Circular Tracing
Circular (or Ring Buffer) tracing is a method whereby trace logs are held in a buffer file or set of files that are a fixed size. When this buffer file fills up, the oldest data is "overwritten" in order for the file to be kept to a predefined size. The obvious advantage of this method is the conservation of disk space as compared to the multiple sequential file method. Circular tracing is also useful when the issue is not readily reproducible and is more intermittent.
However this method is not the one primarily recommended in this guide (we instead recommend multiple sequential without a ring buffer) due the following factors:
- If a Circular trace is not stopped in a timely manner, trace messages appropriate to the failure will likely be overwritten, resulting in the entire data collection being useless and needing to be repeated - something this guide is actively trying to avoid.
- Gauging the appropriate fixed size of the Circular trace file can be tricky. The rate at which the Circular trace file is filled will vary largely depending on the number of users and utilization at the time of collection. A size that may give you 30 minutes of logs in one scenario may only retain 2 minutes in another, resulting in the data collection being potentially useless and needing to be repeated.
Ideally the 3 issues described in this guide are reproducible and in that case multiple sequential logs should have a minimal impact on disk space utilization for the duration needed to reproduce and capture. However in the event the issue is not readily reproducible Circular Tracing may be used instead with an understanding of the two limitations above.
To mitigate these two limitations do the following:
- Do a sample capture of data using Circular methods from the same machines during the same typical time period of the issue you anticipate capturing. This sample capture should continue until the log file(s) reach their maximum configured number and size.
- Have the data analyzed to determine the length of time captured in the trace file(s). This length of time is the amount of time you have from the moment the issue occurs to when someone must terminate the trace. If this is not done in time the trace file will be useless. Increase the size of the Circular file or number of files in the Ring Buffer if more time is needed.
- Have a plan in place to ensure that the moment the issue occurs someone is alerted ASAP to stop the traces.
To configure Circular Tracing in Wireshark see: https://www.wireshark.org/docs/wsug_html_chunked/ChCapCaptureFiles.html - The term used here is "Ring Buffer" instead of Circular.
To configure Circular Tracing in CDF Control (GUI) go to Tools -> Options and select either Circular Log or Multiple Sequential Logs and set a Max Log Size. If you select Multiple Sequential logs check the box to "Use a ring buffer with x files" (defaults to 10).
To configure Circular Tracing in CDF Control (Command Line) utilize the -mode 2 or -mode 8 options and -maxlog for the maximum log file size.