Securing HTTP Callout Using SSL With Server Authentication

Securing HTTP Callout Using SSL With Server Authentication

book

Article ID: CTX310762

calendar_today

Updated On:

Description

The HTTP callout feature allows Citrix ADC appliances to use external web services during policy evaluation. The Citrix ADC sends an HTTP GET or POST request to the web service and receives an HTTP response, which can be used in configured policies and actions. An HTTP callout can be configured to use SSL/TLS (Secure Sockets Layer/Transport Layer Security) for the connection to the web service, to provide privacy and integrity. Optionally, the SSL connection can be configured to use server authentication, which verifies that the web service is correct.

In this article, we will discuss how to configure an HTTP callout to use SSL and how to enable server authentication for the callout. We will demonstrate these using a sample REST web service, ipgeolocation.io, which returns JSON-formatted data about IP addresses. (Citrix makes no claims as to the accuracy of this web service. It is only used here to demonstrate the concepts involved.) The web service’s time zone API is:

https://api.ipgeolocation.io/timezone?apiKey=API_KEY&ip=IPADDR


This and other APIs from this web service are discussed at https://ipgeolocation.io/documentation.html .

Like many web services, this API requires registration before use, and during registration the service assigns a 16-byte hexadecimal API key that identifies the user. For security reasons the API key used for these examples will not be shown. The API is free for limited use (up to 1,000 requests a day and 30,000 requests a month) and the web service provides higher request rates for a fee. Figure 1 shows an example of this API.

Figure 1. The ipgeolocation.io time zone API

We will configure an HTTP callout for the ipgeolocation.io API to get time zone information for the request client’s IP address and then a rewrite action and policy to insert an HTTP header with the GMT offset for the client’s time zone.

 
 

Instructions

Configuring the SSL Service

First, we need to configure an SSL service for the web site. In the CLI, this could be done after the HTTP callout is configured, but the configuration GUI requires the SSL Service to exist when the HTTP callout is set up.
  1. Configure a server for the web site (ipgeolocation.io). In the Citrix ADC configuration GUI, navigate to the Servers page (Traffic Management -> Load Balancing -> Servers) and click Add. Fill in the Create Server form
    • Name: ipgeolocation_server
    • Select Domain Name. (This is necessary to make the SSL server authentication work. Note that this requires that a DNS Name Service be configured.
    • FDQN: ipgeolocation.io
    • Click Create.

CLI command: 
add server ipgeolocation_server ipgeolocation.io
  1. Configure the SSL service for https://ipgeolocation.io. Navigate to the Services page (Traffic Management -> Load Balancing -> Services) and click Add. Fill in the Load Balancing Service form:
    • Name: ipgeolocation_service
    • Select Existing Server.
    • Server: ipgeolocation_server
    • Protocol: SSL
    • Port: 443
    • Click OK.
    • Click Done in the Load Balancing Service page for the newly created service.

CLI command: 
add service ipgeolocation_service ipgeolocation_server SSL 443

Note that at this point, we are not setting SSL server authentication. We will do that later once we have verified that the HTTP callout works without server authentication.
  1. Configure a dummy LB vserver to be used by the HTTP callout. Navigate to the Virtual Servers page (Traffic Management -> Load Balancing -> Virtual Servers) and click Add. Fill in the Load Balancing Virtual Server form:
    • Name: ipgeolocation_vserver
    • Protocol: HTTP
    • IP Address Type: Non Addressable
    • Click OK.
    • Click Continue. This brings up the Load Balancing Virtual Server page needed in step 4.

CLI command: 
add lb vserver ipgeolocation_vserver HTTP
  1. In the Load Balancing Virtual Server page, under the Services and Service Groups heading, click on “No Load Balancing Virtual Server Service Binding”. In the Load Balancing Virtual Server Service Binding page, under Select Service, click on the “>” arrow. In the Service Binding page:
    • Check the ipgeolocation_service and click Select.
    • Click Bind.
    • Click Done in the Load Balancing Virtual Server page. The ipgeolocation_service should now be in the UP state.

CLI command: 
bind lb vserver ipgeolocation_vserver ipgeolocation_service

Configuring the HTTP Callout

The callout will send an HTTP GET request to the URL: 
https://api.ipgeolocation.io/timezone?apiKey=API_KEY&ip=CLIENT-IPADDR

The HTTP response contains a JSON document as shown in Figure 1.

In the configuration GUI, navigate to the HTTP Callouts page (AppExpert -> HTTP Callouts) and click Add. Fill out the Create HTTP Callout form:
  • Name: ipgeolocation_callout
  • Select Virtual Server for Server to receive callout request
  • Virtual Server: ipgeolocation_vserver
  • Request Type: Attribute-Based (default)
  • Method: GET (default)
  • Host Expression: "api.ipgeolocation.io"
  • URL Stem Expression: "/timezone?apiKey=<API_KEY>&ip=" + CLIENT.IP.SRC (Insert the actual apiKey value into the expression.)
  • Body Expression: (empty, default)
  • Headers: (none, default)
  • Parameters: (none, default)
  • Scheme: https
  • Return type: TEXT
  • Expression to extract data from the response: HTTP.RES.BODY(1000) [Note 1]
  • Cache Expiration Time (in sec): 3600 [Note 2]
  • Click Create

CLI command: 
add policy httpCallout ipgeolocation_callout 
  -vServer ipgeolocation_vserver -hostExpr '"api.ipgeolocation.io"'
  -urlStemExpr '"/timezone?apiKey=<API_KEY>&ip=" + CLIENT.IP.SRC'
  -returnType TEXT -scheme http -resultExpr HTTP.RES.BODY(1000)
  -cacheForSecs 3600

Notes:
  1. The size of the body needs to be large enough to include the entire JSON document returned by the web service. It may be tempting to try to use HTTP.RES.BODY(HTTP.RES.CONTENT_LENGTH), but the response does not include a Content-Length header, so that does not work.
 
  1. Citrix ADC can optionally cache responses received for HTTP callouts, and if the callout request URL (Host expression + URL Stem expression) matches a cached response, an actual request is not set to the web server and the cached response is used instead. As is usual with caching, there is a trade-off between memory used for cached results vs. performance improvement by not sending a request and waiting for a reply. The ipgeolocation HTTP callout is a good candidate for caching, since the information for an IP address will not change frequently, the size of the cached data is small, and the API usage is rate-limited.
 

Configuring the Rewrite Action and Policy

  1. Configure a rewrite action to execute the HTTP callout, extract the timezone_offset field from the response using the XPATH_JSON() function, and insert an HTTP header into the request. In the configuration GUI, navigate to the Rewrite Actions page (AppExpert -> Rewrite -> Actions) and click Add. Fill out the Create Rewrite Action form.
    • Name: get_gmt_offset_action
    • Type: INSERT_HTTP_HEADER
    • Header Name: GMT-offset
    • Expression: SYS.HTTP_CALLOUT(ipgeolocation_callout).XPATH_JSON(xp%/timezone_offset%)
    • Click Create

CLI command: 
add rewrite action get_gmt_offset_action
  insert_http_header GMT-offset      
  "SYS.HTTP_CALLOUT(ipgeolocation_callout).XPATH_JSON(  
    xp%/timezone_offset%)"
  1. Configure a rewrite policy to execute the action. Navigate to the Rewrite Policies page (AppExpert -> Rewrite -> Policies) and click Add. Fill out the Create Rewrite Policy form.
    • Name: get_gmt_offset_policy
    • Action: get_gmt_offset_action
    • Log Action: (empty, default)
    • Undefined-Result Action: NOREWRITE (If an error occurs, the header will not be inserted.)
    • Expression: true (Execute the action for all requests.)
    • Click Create

CLI command: 
add rewrite policy get_gmt_offset_policy true get_gmt_offset_action 
  NOREWRITE
  1. Bind the policy to the relevant LB vserver(s). Note that the policy cannot be bound globally, because the HTTP callout on the dummy LB vserver would trigger the policy and action, leading to a potentially infinite recursion. The HTTP callout code will detect this condition and return an Undef error. In the configuration GUI, navigate to the Rewrite Policies page (AppExpert -> Rewrite -> Policies) and in the Rewrite Policies page click on Policy Manager. In the Bind Point page,
    • Bind Point: Load Balancing Virtual Server
    • Protocol: HTTP
    • Connection Type: Request
    • Virtual Server: <name of the relevant LB vserver>
    • Click Continue.
    • On the next page
      • Select Policy: get_gmt_offset_policy
      • Priority: 100 (default)
      • Goto Expession: END (default)
      • Invoke LabelType: None (default)
      • Click Bind
      • Click Done

CLI command: 
bind lb vserver <ipgelocation_vserver> -policyName get_gmt_offset_policy
  -priority 100 -type REQUEST

At this point we can test the callout and rewrite action using a printenv CGI-bin script (example: https://en.wikipedia.org/wiki/Common_Gateway_Interface#Example ) that returns the headers passed to the back-end server. We verify that the GMT-offset header is set as expected for the client IP address.
 

Obtaining and Installing the Root CA Certificate for Server Authentication

After testing the HTTP callout over the SSL service, we can enable server authentication. To do this, we need to establish trust in the web site used for the callout. A quick overview how SSL establishes trust:
  1. A web site generates two keys, one public and one private, that can be used for asymmetric encryption – data encrypted using the public key can only be decrypted by the private key and vice versa.
  2. The web site sends a certificate request to an organization called a Certificate Authority (CA). The request contains the web site’s DNS name and its public key, among other information. The CA verifies that the web site name is authentic and issues it an X.509 certificate with the name, the web site’s public key, the CA’s name, a digital signature generated using the CA’s own private key, and other information.
  3. When an SSL client connects to the web site, the web site sends the client its X.509 certificate. The client verifies the CA’s digital signature on the X.509 certificate. To do this, the web site needs the CA’s public key, which is typically provided in another X.509 certificate
  4. After the client verifies the web site’s certificate is valid, it uses the public key in the certificate to encrypt random data to send back to the web server. Only the web server with its private key can decrypt the data, which it uses to establish a key for bulk symmetric encryption between the client and server.

Often, the CA that certified the web site is an intermediate CA, which is certified by another CA. This forms a hierarchy, called a certificate chain or path, which ends in a root CA with a self-signed certificate. The web site sends the certificate chain, minus the root CA, to the client, and the client verifies each certificate in the chain using the certificate above it. It then uses an installed root CA certificate to complete verifying the certificate chain. It is the installation of the root CA certificate that ultimately establishes trust in the web site.

Web browsers come with a pre-installed collection of certificates for popular root CAs, so the browser user does not need to explicitly establish trust (and in fact is unaware of the process if the certificate chain is valid). But Citrix ADC does not have pre-installed root CA certificates, so you have to obtain the necessary certificate and install it using the configuration GUI.

To obtain the root CA for a web site, we will access the site using the Chrome browser on Windows 10. Other browsers have similar facilities.
  1. Go to the web address using the browser.
  2. Click the lock next to the web site address in the browser (see Figure 1). This displays a Connection is secure panel. 
  3. Click Certificate (Valid). This displays a Certificate panel for the web site’s certificate.
  4. Click Certificate Path tab. This displays the Certificate Path, as shown in Figure 2 for the ipgeolocation.io certificate.
  5. The Certificate Path tab shows that the web site is certified by an intermediate CA named “CloudFlare Inc ECC CA-2”, which in turn is certified by a root CA named “Baltimore CyberTrust Root”. Double-click the root CA name to display the Certificate panel for the root CA.
  6. Click the Details tab for the root CA certificate panel.
  7. Click the Copy to File… button. This initiates the Certificate Export Wizard. Click Next.
  8. Select the Base-64 encoded X.509 format and click Next.
  9. In the File to Export panel, choose a local file name for the certificate. Click Next.
  10. Click Finish and OK on the Export was successful notification.
Figure 2. ipgeolocation.io Certificate Path

Now we have the root CA certificate file on my local machine. The next steps are to install it on the Citrix ADC in an SSL certKey configuration object.

In the configuration GUI, navigate to the CA Certificates page (Traffic Management -> SSL -> Certificates -> CA Certificates). Click Install. Fill out the Install CA Certificate form:
  • Certificate-Key Pair Name: DigiCertBaltimoreRoot
  • Choose the local certificate file: DigiCert Baltimore Root.cer
  • Click Install. This copies the file to the Citrix ADC and creates the DigiCertBaltimoreRoot certKey object.

CLI command:  
add ssl certKey DigiCertBaltimoreRoot
  -cert "DigiCert Baltimore Root.cer"
Note that if you do not use the GUI to install the certificate, you have to manually copy the certificate file to the /nsconfig/ssl directory on the Citrix ADC.

For more details, please refer to the Install, link, and update certificates within Citrix ADC documentation.
 

Enabling Server Authentication

  1. Finally, we can enable server authentication for the web site’s SSL service. We first have to specify that the root CA we had previously installed is to be used for the SSL service. In the configuration GUI, navigate to the Services (Traffic Management -> Load Balancing -> Services) and double-click ipgeolocation_service. This brings up the Load Balancing Service page for the service.
    • Click “+ Certificate” under Advanced Settings to show the Certificate header (at the bottom of the page).
    • Click “No CA Certificates” to bring up the CA Certificate Binding page.
    • Select the DigiCertBaltimoreRoot certificate.
    • CRL and OCSP Check: (empty, default; for now. We will come back to this later.)
    • Click Bind.

CLI command: 
bind ssl service ipgeolocation_service
 -certkeyName DigiCertBaltimoreRoot -CA
  1. In the Load Balancing Service page for the service, click the pencil on the “SSL Parameters” heading to edit the SSL parameters.
    • Check Enable Server Authentication.
    • Common Name: (empty; disables common name checking, to be done in the next section)
    • Click OK.

CLI command: 
set ssl service ipgeolocation_service -serverAuth ENABLED

After enabling server authentication, we can retest and verify that the GMT-offset header is still set correctly.

 

Checking the Web Site Name

Now we can enable checking that the web server DNS name matches the name in its certificate, important because it ensures that the web site is actually the one we intend to use for the HTTP callout. The SSL Service parameter to do this is labelled Common Name, referring to the CN part of the certificate Subject field. However, the Subject CN of the web site’s certificate, sni.cloudflaressl.com, does not match the web site DNS name api.iogeolocation.io. There is another certificate field, Subject Alternative Name, that also needs to be checked. Figure 3 shows this field in our certificate. It has an entry: 
DNS Name=*.ipgeolocation.name
that does match the web site address. Citrix ADC will check the Subject Alternative Name if the Subject CN does not match.
 
Figure 3. ipgeolocation.io Certificate Details

In the Load Balancing Service page for the service, click the pencil on the “SSL Parameters” heading to edit the SSL parameters:
  • Common Name: api.ipgeolocation.io
  • Click OK.

CLI command: 
set ssl service ipgeolocation_service -commonName api.ipgeolocation.io
After setting the common Name, we can retest and verify that configuration still works as expected. We can also try setting commonName to a value that does not match the ipgeolocation.io web site and verify that the HTTP callout fails as expected.

 

Checking for Revoked Certificates Using CRLs

X.509 certificates can be revoked by their issuing certificate authorities for a number of reasons, including security breaches of a web site or even the CA itself. There are a couple of ways that Citrix ADC can check if a certificate has been revoked. The first is using certificate revocation lists (CRLs) that are periodically published by CAs. To obtain the CRL for a CA, we will again access the site using the Chrome browser on Windows 10.
  1. Click the lock and Certificate.
  2. Click the Details tab and scroll down to get the CRL Distribution Points field. Click that field to get details (Figure 4). The CRL can be obtained from http://crl3.digicert.com/CloudFlareIncECCCA2 .
 
Figure 4. CRL Distribution Point for the ipgeolocation.io Certificate
  1. Enter a CRL distribution URL in the browser. This downloads the CRL to the local Downloads folder.
  2. To use this CRL on Citrix ADC, we also need to import and install the CloudFlareIncECCCA2 certificate, following the steps described in Obtaining and Installing the Root CA Certificate.

To install and use the CRL on the Citrix ADC:
  1. In the configuration GUI, navigate to the CRL page (Traffic Management -> SSL -> CRL) and click Add. Fill out the Create CRL form:
    • CRL Name: CloudFlareIncECCCA2
    • CRL File: Chose File
      • Upload \Downloads\CloudFlareIncECCCA2
      • Select CloudFlareIncECCCA2 and click Open
    • Inform: Select DER
    • CA Certificate: CloudFlareIncECCCA2
    • Check CRL Auto Refresh
      • Method: HTTP (default)
      • Scope: One (default)
      • Port: 80
      • URL: http://crl3.digicert.com/CloudFlareIncECCCA2.crl
      • Interval: Weekly
      • Days: 1 (default)
    • Click Create

CLI command: 
add ssl crl CloudFlareIncECCCA2    
  /var/netscaler/ssl/CloudFlareIncECCCA2.crl -inform DER
  -refresh ENABLED -CAcert CloudFlareIncECCCA2 -method HTTP
  -url http://crl3.digicert.com/CloudFlareIncECCCA2.crl
  -port 80 -interval WEEKLY -day 1
  1. We can follow these same steps to download the Omniroot2025 CRL for the DigiCertBaltimoreRoot CA, which is needed to determine if the CloudFlare intermediate CA certificate has been revoked. 

CLI command: 
add ssl crl Omniroot2025 /var/netscaler/ssl/Omniroot2025.crl
  -inform DER -refresh ENABLED -CAcert DigiCertBaltimoreRoot
  -method HTTP -url \"http://crl3.digicert.com/Omniroot2025.crl
  -port 80 -interval WEEKLY -day 1
  1. To enable the CRL checking for the SSL service: Navigate to the Services page (Traffic Management -> Load Balancing -> Services) and click ipgeolocation_service. Click on CA Certificates. If the CA certificate is already bound, unbind it. Under Select CA Certificate, click on the “>” arrow. In the CA Certificates Binding page:
    • Select CA Certificate: DigiCertBaltimoreRoot
    • CRL and OCSP Check: CRL Mandatory
    • Click Bind.

CLI commands: 
unbind ssl service ipgeolocation_service
  -certkeyName DigiCertBaltimoreRoot -CA 
bind ssl service ipgeolocation_service
  -certkeyName DigiCertBaltimoreRoot -CA -crlCheck Mandatory
We can once again retest with the CRL checking enabled to verify the GMT-offset header is still correct.
 

Checking for Revoked Certificates Using OCSP

The second technique for checking certificate validity is OCSP (Online Certificate Status Protocol). If the CA provides the OCSP service, Citrix ADC can be configured to send OCSP requests to the CA to check if a web site certificate is valid. The Details panel for the web service certificate has a field named Authority Information Access that provides a URL for the CA’s OCSP service (Figure 5). For the ipgeolocation.io certificate, this URL is http://ocsp.digicert.com.
 
Figure 5. OCSP URL for the ipgeolocation.io Certificate
  1. In the configuration GUI, navigate to the OCSP Responder page (Traffic Management -> SSL -> OCSP Responder) and click Add. Fill out the Create OCSP Responder form:
    • Name: DigiCertOCSP
    • URL: http://ocsp.digicert.com
    • Check Cache
    • Time-Out: 1440 minutes (24 hours)
    • Batching Depth: 1 (default)
    • Batching Delay: 1 (default)
    • Produced at Time Skew: 300 (default)
    • Certificate: DigiCertBaltimoreRoot
    • HTTP Method: POST (default)
    • Click Create

CLI command: 
add ssl ocspResponder DigiCertOCSP -url http://ocsp.digicert.com
 -cache ENABLED -cacheTimeout 1440 -batchingDelay 1
 -responderCert DigiCertBaltimoreRoot
  1. To enable the OCSP checking for the SSL service: Navigate to the Services page (Traffic Management -> Load Balancing -> Services) and click ipgeolocation_service. Click on CA Certificates. If the CA certificate is already bound, unbind it. Fill it the CA Certificate Binding For
    • Select CA Certificate: DigiCertBaltimoreRoot
    • CRL and OCSP Check: OCSP Mandatory
    • Click Bind.

CLI commands: 
unbind ssl service ipgeolocation_service
  -certkeyName DigiCertBaltimoreRoot -CA 
bind ssl service ipgeolocation_service
  -certkeyName DigiCertBaltimoreRoot -CA -ocspCheck Mandatory

We can now retest with the OCSP checking enabled.
 

Issue/Introduction

How to configure an HTTP callout to use SSL and how to enable server authentication for the callout, demonstrated using a sample REST web service.

Additional Information

Troubleshooting Server Authentication Problems

If the SSL server authentication for the HTTP callout fails, the callout will have a non-zero UNDEFINED HITS field in the HTTP Callouts page of the configuration GUI. The CLI command to check this is 
> show policy httpCallout ipgeolocation_callout
        . . .
        Undef Hits: 1
Unfortunately, there is no other information reported on why the server authentication failed. Here are some possible reasons:

Bad commonName: The commonName set for the SSL service does not match the CN part of the Subject or a DNS name in the Subject Alternate Name of the web site’s X.509 certificate, which is supposed to be the DNS name for the web site.

You can view the certificate Subject and the Subject Alternate Name fields in the Details tab of the Certificate panel of a browser connected to the web site. You can determine if this is the problem by disabling the commonName check in the configuration GUI by deleting the Common Name value in the SSL Parameters of the Load Balancing Service page for the SSL service. The CLI command is 
unset ssl service ipgeolocation_service -commonName
With the check disabled, you can test the callout as before, and if it works that indicates the common name was the problem. However, it is not recommended to leave the common name check disabled because you are no longer assured the web site is correct.

Bad or missing root CA certificate: The correct root CA was not bound to the SSL service. Review the instructions again in Obtaining and Installing the Root CA Certificate and Enabling Server Authentication.

Expired server certificate: All X.509 certificates have Valid From and Valid To fields that specify the period during which the server is valid. All certificates will eventually expire and need to be replaced with newer certificates. If the web site has an expired certificate, this should be reported by a web browser when you access the web site. There is nothing that you can do on the Citrix ADC to make the certificate valid. You will have to request that the web site owner update the certificate.  

Expired root CA certificate: All root CA certificates will also eventually expire. Citrix ADC will notify you that a root CA certificate is about to expire, and you should obtain and install a new certificate using the steps described earlier.

Revoked certificate: A web site certificate can be revoked by its issuing CA if that web site has been compromised and its private key disclosed. If CRL or OCSP checking is enabled, this would be detected on Citrix ADC. Certificate revocation is a serious issue and the web site should no longer be trusted until it installs a valid certificate. 

Incorrect Citrix ADC Date or Time: If the date or time setting on the Citrix ADC is incorrect, a valid certificate or CRL may be interpreted as not yet valid or expired. Check that the date and time is correct and fix them if they are not. (Presumably other things on the Citrix ADC would not work if the date or time were incorrect.)
Powered by Wolken Software