• In Large Environments “Check requirements and configuration task" for MPXAXD Agent fails with Error: "OData server returned: 500 Internal Server Error”

• We also see that MPXAXD agent service spikes memory usage and takes up all the RAM.

 

• From the Support Information Logs and debug logs saved at %ProgramData%\Citrix\SCOM MP Agent\logs we found that the below ODATA query was timing out:

2017-07-24 14:56:07,232 WARN [connection-discovery] ComTrade.ResourceMonitoring.Discovery.EventDiscoveryTask`1 - Critical tasklet DiscoverConnections has failed, aborting the whole workflow: ODataException: OData server returned: 500 Internal Server Error, RequestUrl='http://ddc.xxx.com/Citrix/Monitor/OData/v2/Data/Connections?$expand=Session/Failure&$filter=((Id gt 32774762) or ((DisconnectDate gt datetime'2017-07-12T15:43:39.2370000') or ((DisconnectDate eq null) and (Session/EndDate gt datetime'2017-07-12T15:43:39.2370000')))) and (((LogOnEndDate ne null) and (Protocol ne null) and (Protocol ne 'Console') and (Protocol ne 'RDP')) or (Session/FailureDate ne null))&$top=2000', HttpStatusCode=InternalServerError, RequestId='' ---> System.Net.WebException: The remote server returned an error: (500) Internal Server Error. 
at System.Net.HttpWebRequest.GetResponse() 
at Citrix.OData.ThinClient.RequestHandler.GetContent(Uri requestUrl, TimeSpan timeout, ICredentials credentials) 
--- End of inner exception stack trace --- 
at Citrix.OData.ThinClient.RequestHandler.GetContent(Uri requestUrl, TimeSpan timeout, ICredentials credentials) 
at Citrix.OData.ThinClient.ODataClient.GetContent(String url) 
at Citrix.OData.ThinClient.ODataClient.GetFeed(String url) 
 

We disabled the "Process Processor Time" and "Process Private Memory" SCOM Monitors for Citrix MPXAXD Agent.

---

Problem Cause

• From the Process dumps of  MPXAXD agent collected after enabling User Mode Stack Tracing, we found that
  memory spikes seems to be related to two SCOM monitors that restart the MPXAXD Agent service in case of high memory / CPU consumption. 

 

• We identified this issue with other customers and also in our large test environments with high amount of monitoring data, we now recommend the thresholds for CPU and memory monitors for MPXAXD agent to be increased/disabled .In these cases the monitors cause unnecessary restarts of MPXAXD agent, which results in memory spikes, since the Citrix MPXAXD Agent needs to load everything already discovered and monitored for the last 24 hours back in the memory to avoid any potential data loss.

 

• This has been documented in new SCOM XA/XD Management Pack release 3.14.

           https://docs.citrix.com/en-us/scom-management-packs/xenapp-xendesktop/3-14/performance-overview/config.html
​

In Large Environments “Check requirements and configuration task" for MPXAXD Agent fails with Error: "OData server returned: 500 Internal Server Error” We also see that MPXAXD agent service spikes memory usage and takes up all the RAM.