Cert-Based Authentication for Device isn't working after NetScaler Upgrade, even when the client certificates are valid.
Please validate the traces and there you will be able to see that the ocsp response for the request is failing: use the filter as "ocsp" prootocol

This is a known and fixed issue and is caused when the ocsp server doesn't send a valid NONCE Extension in response.
There are two fixes for the same:
1: Enable the ocsp nonse extension response at the ocsp server end
2: or disable the nonce extension forwarding to the backend.
To disable the Nonce extention at the NetScaler end run below command:
ssl ocspResponder ocsp-responder-server -url "http://ocsp.server:80/ocsp" -trustResponder -useNonce NO -insertClientCert YES
 

---

Problem Cause

Fixed issue # 658120, 684909 The Certificate Revocation Lists (CRL) checks and Online Certificate Status Protocol (OCSP) validation are not done on a NetScaler appliance through an SSL renegotiation as part of certificate based authentication.

If you have recently upgraded the NetScaler build to 12.0 53.13+ from 11.1 53.11 or earlier, certificate validation may fail when ocsp responder is bound.