This article discusses how to diagnose and fix issues related to Secure Mail push notifications on iOS devices. 

Background

Push notifications for Secure Mail allow users to receive updates when the app refreshes, and notifications about email and calendar activity through the Apple Push Notification service (APNs) when the app is running in the background or is closed. For example, users can see updates of unread mail counts when the badge on the Secure Mail app icon updates. Push notifications also facilitate the timely delivery of lock screen updates for new messages.

For a deep dive into push notifications on Secure Mail for iOS, see the article Push Notifications for Secure Mail for iOS in the Citrix product documentation.

Issues that users may report

These are some common issues that users report that involve APNs:

• The badge update reflects local unread messages only, and not unread messages from the server. 
• The badge does not update.
• The badge is slow to update.
• The badge updates only when users bring Secure Mail to the foreground. 
• Lock screen notifications do not appear in a timely fashion when the device is not in low power mode.

Troubleshooting Steps

To resolve these issues, you can follow these troubleshooting steps so that you and Citrix technical support, if they are involved, can determine the root cause of issues quickly and make a fix. When any of the solutions is effective, push notifications start working on the device again.

Obtain the Secure Mail logs from the device

The Secure Mail logs are always the first piece of information to obtain when dealing with an APNS issue. Ideally, you should ask users to send the logs immediately when an issue occurs. This way, they'll capture the logs corresponding to the error. 

Ensure that Background App Refresh is enabled

In order to generate notifications on the device lock screen, Secure Mail requires that Background App Refresh be enabled. This can be enabled in your iOS settings under Settings à General à Background App Refresh.
 
Make sure that Background App Refresh is switched on both globally as well as for Secure Mail.
 

Verify if the push notifications are enabled for Secure Mail in the app policies file

The first thing to check is that APNs is enabled by policy. To do so, open the app support bundle and find the file Ctxlog_AppPolicies.xml for Secure Mail.

Inside this file, the following XML entities are related to APNs. If any of the required policies for your environment are not set, ensure that you set them to the required value in the XenMobile console.

Entity Name	Description	Required value in order for APNS to function
PushNotificationsEWSHostName	The host name of the Exchange Web Services (EWS) endpoint.	This value does not need to be set. If this value is not set, the value of the ExchangeServer entity is used. If you use autodiscovery to provision the account, and this value is not set, the EWS endpoint is resolved during autodiscovery. An empty value here does not indicate a problem or concern.
PushNotificationsEnabled	Flag indicating whether APNs is enabled	Must be set to true.
PushNotificationsListenerServiceHostName	URL of the listener service endpoint that is used to generate the push notifications.	Must be set to a valid endpoint URL. The valid endpoints are documented at https://docs.citrix.com/en-us/xenmobile-apps/10/secure-mail/push-notifications.html
PushNotificationsTenantId	The tenant ID that's used when communicating with the listener service endpoint.	This is only needed when using an enterprise deployment of Secure Mail. For app store Secure Mail deployments, you do not need this value because Secure Mail requests the required tenant ID from the Citrix cloud listener service.

Look for errors in the Secure Mail push notification events file

If you don't see anomalies in the app policies, the next step is to determine whether there are errors during the subscription to push notifications. Several high level events are involved when subscribing to push notifications:

1. Obtaining the Secure Mail tenant ID. This ID is obtained from a policy when you use enterprise distribution for Secure Mail. When you use the app store version of Secure Mail, the app obtains the ID from the Citrix cloud listener service.
2. Registering the device with the Citrix cloud listener service.
3. Subscribing to EWS push notifications with Exchange Server. Note that Secure Mail relies on EWS and not ActiveSync for its push notification functionality.
4. Updating the Citrix cloud listener service with details of the EWS push notifications. 

Network or authentication failures during any of these steps may cause push notifications to malfunction. 

To troubleshoot these and other push notification-related events, the Secure Mail 10.5.5 release includes a file called CtxLog_APNS_Events.csv. 

This file contains information about the following events:

1. Subscription to push notifications.
2. Validation of the push notification subscription with the Citrix cloud listener service.
3. Retrieval of the tenant ID from the Citrix cloud listener service.

The following figure shows an example of this file: 

The entries contain the following information:

1. The date and time of the operation.
2. The operation that was carried out.
3. The completion status of that operation.
4. The tenant ID. 
5. The listener service token.
6. The tenant ID + listener service token combination.

The last three entries allow Citrix to easily find the cloud listener service logs for the device. Those strings line up with entries in the listener service logs, allowing technical support to more quickly find information relevant to any failures.

Look for push notification events in the background application refresh log file

The log file bundle from Secure Mail includes a file called CTXLog_backgroundFetchSchedule.csv. This file logs background application refresh events, as well as events that indicate a push notification has been delivered to the device.

By using this file, you can get a ballpark figure of when Secure Mail received the last push notification-related wake-up. Note that push notifications are not always delivered to Secure Mail. Examples of when they are not delivered include the following:

1. The device is in low power mode.
2. There was a change in the unread count of the inbox caused by events other than new mail arriving. For example, a message was marked as read.

When looking at this file, you'll see the following as an example.

BackgroundFetch	01-Dec-2016 12:47:19:310 (+0000)	01-Dec-2016 12:47:34:826 (+0000)	15	UIBackgroundFetchResultNewData
Remote Notification	01-Dec-2016 12:47:41:727 (+0000)	01-Dec-2016 12:47:44:488 (+0000)	2	UIBackgroundFetchResultNewData
Remote Notification	01-Dec-2016 12:48:01:026 (+0000)	01-Dec-2016 12:48:03:345 (+0000)	2	UIBackgroundFetchResultNewData
Remote Notification	01-Dec-2016 12:51:31:254 (+0000)	01-Dec-2016 12:51:33:473 (+0000)	2	UIBackgroundFetchResultNewData
Remote Notification	01-Dec-2016 12:55:41:419 (+0000)	01-Dec-2016 12:55:44:738 (+0000)	3	UIBackgroundFetchResultNewData
BackgroundFetch	01-Dec-2016 12:55:41:483 (+0000)	01-Dec-2016 12:56:09:489 (+0000)	28	UIBackgroundFetchResultNewData
Remote Notification	01-Dec-2016 13:00:51:084 (+0000)	01-Dec-2016 13:00:53:511 (+0000)	2	UIBackgroundFetchResultNewData
Remote Notification	01-Dec-2016 13:02:56:311 (+0000)	01-Dec-2016 13:02:59:193 (+0000)	2	UIBackgroundFetchResultNewData
BackgroundFetch	01-Dec-2016 13:02:56:373 (+0000)	01-Dec-2016 13:03:24:330 (+0000)	27	UIBackgroundFetchResultNewData
Remote Notification	01-Dec-2016 13:06:16:308 (+0000)	01-Dec-2016 13:06:18:586 (+0000)	2	UIBackgroundFetchResultNewData
Remote Notification	01-Dec-2016 13:09:31:460 (+0000)	01-Dec-2016 13:09:33:480 (+0000)	2	UIBackgroundFetchResultNewData

The columns are as follows:

1. The type of wake-up (background app refresh or push notification).
2. The time that the wake-up occurred.
3. The time that the wake-up processing completed.
4. The number of seconds it took to process the wake-up.

Note that the Remote Notification text shows that push notifications are being received. You can use this file to give an idea as to when and if the device is receiving push notifications.

Check the Secure Mail logs for push notification-related errors

You can use the Secure Mail logs to filter out APNs-related log messages. To do so, you filter on Secure Mail.APNS in Excel.

1. Select "Filter".

   

2. Select the "Module" column and choose to display only Secure Mail.APNS. 

   

   Only then the logs related to the push notification aspects of Secure Mail appear. 

These logs show the following:

• Remote notifications that are received by the app.
• Logs related to the app subscribing to APNs notifications.
• Logs related to the app performing validity checks for the current subscription.

From this information, you can look for any errors or warnings that may help narrow down the issue.

Common errors that you can see in the Secure Mail APNs logs

Errors that may be seen in the Secure Mail logs include the following.

The push notification entitlement has not been added to Secure Mail (enterprise builds only)

This occurs on enterprise distribution builds in which the customer has not enabled the Push notification entitlement on their App ID. You can use the following indication in the logs to ascertained this situation.

Failed to register for remote notifications with error
error.domain=NSCocoaErrorDomain, error.code=3000, error.userInfo={

NSLocalizedDescription = no valid 'aps-environment' entitlement string found
for application";

In this case, look at your provisioning profile and ensure that that the Push Notifications entitlement is enabled as shown in this figure. 

Subscription to APNs notifications has failed

The subscription to APNs notifications may fail completely. You can find out by looking for ERROR category messages related to the subscription to push notifications.

Common failure reasons include.

• Network failures communicating to the endpoint. Check that network access policy is configured appropriately for your environment.
• Inability to get a device APNs token. This is a rare situation that is caused by network failures talking to the Apple APNs server, or a problem associated with the Apple ID of the user. A reset of the device may be required in these rare cases.
• HTTP 500 responses from the push service. Check that the tenant ID is correct.

Some points to note:

• If split tunneling is enabled, network errors in the logs may point to the issue.
• If full tunneling is enabled, check that NetScaler is configured in a way that allows the client to connect to the listener service endpoint.

For HTTP 500 errors, ensure that the tenant ID policy has been correctly set.

The validity check indicates that the device subscription is no longer valid

Secure Mail periodically checks to see that its subscription to notifications is valid. Secure Mail checks by making a network request to the Citrix cloud listener service. The status may indicate that the subscription is no longer valid. At this point, Secure Mail resubscribes to push notifications.
A subscription will be considered to have become invalid when Exchange Server sends no status updates for a subscription to the Citrix cloud listener service for a period of approximately a half an hour.

You should check on the following.

1. Can all of the Exchange Servers communicate externally to the Citrix listener service endpoint? For a list of the Citrix listener service endpoints, see the article Push Notifications for Secure Mail for iOS in the Citrix product documentation.Ensure that the appropriate firewall rules are configured.
2. Note any errors in the Exchange Server event logs that show communication problems with the listener endpoint. The event log information you need is described later in this blog post. 

Authentication failures are seen when subscribing to EWS

If the logs reveal authentication failures when registering for push notifications, it's likely that EWS is configured to use a different form of authentication than the way ActiveSync is configured. Ensure that if you are using certificate-based authentication for Secure Mail that EWS is also configured to use certificate-based authentication.

If you are using HTTP Basic authentication for Secure Mail, ensure that NTLM authentication is being used for EWS.

For more information on this topic, please refer to:

Configuring certificate-based authentication with EWS for Secure Mail push notifications

HTTP 413 errors when subscribing to EWS subscriptions

When certificate based authentication is used for EWS authentication, the client may fail to subscribe with the logs indicating HTTP 413 errors. In this case, ensure that the uploadReadAheadSize is configured to an appropriate value on IIS. The uploadReadAheadSize attribute is discussed at https://www.iis.net/configreference/system.webserver/serverruntime. Set this value to be at least the size of the maximum attachment size that is configured for ActiveSync.

Look for connectivity failures in the EWS event logs 

If you suspect a problem with the EWS subscription because the subscription has become invalid, you can look at the EWS event logs.

You can open the event logs on a Windows-based computer and then view and search them in the event viewer application. You can use the application to search for the instances of the listener service endpoint URL that you are using. For example, you could search for us-east-1.mailboxlistener.xm.citrix.com.

The event logs that you need are the Application Logs from the client access server that hosts the user's mailbox. These logs looks like the following figure. They show the connection failure to the listener service. The event source in this example is MSExchange Web Services.

If you see multiple errors in the EWS events logs related to the Citrix cloud listener service endpoint URL, ensure that your firewall rules allow your Exchange Server to communicate outwards to the Citrix cloud listener service.

Conclusion

As we discuss in this article, you can follow many steps on your own to troubleshoot push notification issues with Secure Mail for iOS. By using both client side and server logs, you can narrow down the root cause of any push notification issues quickly.

This article discusses how to diagnose and fix issues related to Secure Mail push notifications on iOS devices.