There are two WebLogic applications, application A and application B. Both A and B are load balanced on the NetScaler.
The agEncoder is hosted on application B. When you click agEncoder from application A, before redirecting to application B for agEncoder, the client browser sends the application B link to application A for validation. Application A sends a HTTP GET request to application B for connectivity verification (application B is load balanced on NetScaler). The HTTP GET request is sent to NetScaler LB VIP.
The NetScaler LB VIP for application B rejects the GET request with certificate error because of which the validation fails.
To resolve this issue install SHA1 certificate on NetScaler LB VIP for application B.
The WebLogic server uses Java for HTTP GET request. When application A establishes SSL session with NetScaler LB VIP for application B, the application A WebLogic server throws error "Level: Fatal, Description: Bad Certificate" (the certificate installed on NetScaler LB VIP for application B).
The WebLogic server did not support SHA2 certificate.