NetScaler AppExpert Policies for Apache mod_rewrite Samples

NetScaler AppExpert Policies for Apache mod_rewrite Samples

book

Article ID: CTX120854

calendar_today

Updated On:

Description

Contents

Canonical URLs

Description

On some web servers there are more than one URL for a resource. Usually there are canonical URLs (which should be actually used and distributed) and those which are shortcuts, internal URLs, and so on. Independent of which URL is supplied with the request to the user, he should finally only see the canonical URL.

Solution

Example: Converting URL /~user to /u/user.

Apache_Rewrite:

rewriterule ^/~([^/]+)/?(.*) /u/$1/$2[R]

AppExpert:

add responder action act1 redirect '"/u/"+ HTTP.REQ.URL.HTTP_URL_SAFE.AFTER_STR ("/~")'
add responder policy pol1 'HTTP.REQ.URL.STARTSWITH("/~") && HTTP.REQ.URL.LENGTH.GT(2)' act1
bind responder global pol1 100

Canonical Hostnames

Description

The goal of this rule is to force the use of a particular hostname, in preference to other hostnames, which might be used to reach the same site. For example, if you wish to force the use of www.example.com instead of example.com, you might use a variant of the following rules.

Solution

Example: Changing example.com to www.example.com

Apache_Rewrite:

# For sites running on a port other than 80
RewriteCond %{HTTP_HOST} !^www.example.com
RewriteCond %{HTTP_HOST} !^$
RewriteCond %{SERVER_PORT} !^80$
RewriteRule ^/(.*) http://www.example.com:%{SERVER_PORT}/$1 [L,R]
# And for a site running on port 80
RewriteCond %{HTTP_HOST} !^www.example.com
RewriteCond %{HTTP_HOST} !^$
RewriteRule ^/(.*) http://www.example.com/$1 [L,R]

AppExpert:

Case 1: for sites running on a port other than 80

add responder action act1 redirect '"http://www.example.com:"+CLIENT.TCP.DSTPORT+HTTP.REQ.URL.HTTP_URL_SAFE'
add responder policy pol1 '!HTTP.REQ.HOSTNAME.CONTAINS("www.example.com")&&!HTTP.REQ.HOSTNAME.EQ("")&&!HTTP.REQ.HOSTNAME.PORT.EQ(80)&&HTTP.REQ.HOSTNAME.CONTAINS("example.com")' act1
bind responder global pol1 100 END

Case 2: for sites running on port 80

add responder action act1 redirect '"http://www.example.com” +HTTP.REQ.URL.HTTP_URL_SAFE'
add responder policy pol1 '!HTTP.REQ.HOSTNAME.CONTAINS("www.example.com")&&!HTTP.REQ.HOSTNAME.EQ("")&&HTTP.REQ.HOSTNAME.PORT.EQ(80)&&HTTP.REQ.HOSTNAME.CONTAINS("example.com")' act1
bind responder global pol1 100 END

Moved Document Root

Description

Usually, the Document Root of the web server directly relates to the URL “/”. But in some scenario, the document root could shift to some other directory, also. The following rules can be used to implement this.

Solution

Example: Rewrite the url / to /e/www

Apache_Rewrite:

RewriteEngine on
RewriteRule ^/$ /e/www/ [R]

AppExpert:

Solution 1: Converting ApacheRewrite example as it is to AppExpert
add responder action act1 redirect '"/e/www/"' -bypassSafetyCheck yes
add responder policy pol1 'HTTP.REQ.URL.EQ("/")' act1
bind responder global pol1 100

Solution 2: Generic Case

A request might have file path, query, and so on. You can replace the root directory with the new location (path, query, and so on) and rest is present in the redirected request. For example, a request like: /abc/file.html will be redirected to /e/www/abc/file.html

add responder action act1 redirect '"/e/www"+ HTTP.REQ.URL.HTTP_URL_SAFE'
add responder policy pol1 '!HTTP.REQ.URL.STARTSWITH("/e/www/")' act1
bind responder global pol1 100 END

Move Homedirs to Different Web Server

Description

In this scenario you want to redirect request for homedirs on a web server to another web server. Usually, such a need arises when establishing a newer web server to replace the old one over time. You must redirect all the requests for a particular homedir to another web server.

Solution

Example: Let the hostname for new Web server be newserver.

Apache_Rewrite:

RewriteRule ^/(.+) http://newserver/$1 [R,L]

AppExpert - First Method:

Add responder action act1 redirect '"http://newserver"+ HTTP.REQ.URL.HTTP_URL_SAFE'
Add responder policy pol1 'HTTP.REQ.URL.REGEX_MATCH(re#^/(.+)#)' act1
Bind responder global pol1 100 END

AppExpert -  Second Method:

Add responder action act1 redirect '"http://newserver"+ HTTP.REQ.URL.HTTP_URL_SAFE'
Add responder policy pol1 'HTTP.REQ.URL.LENGTH.GT(1)' act1
Bind responder global pol1 100 END

Structured Homedirs

Description

Some sites with thousands of users usually use a structured homedir layout, that is, each homedir is in a sub-directory which begins, for instance, with the first character of the username. So, /~foo/anypath is /home/f/foo/.www/anypath while /~bar/anypath is /home/b/bar/.www/anypath. The following rules could be used to implement this.

Solution

Apache_Rewrite:

RewriteRule ^/~(([a-z])[a-z0-9]+)(.*) /home/$2/$1/.www$3

AppExpert:

Add rewrite action act1 replace 'HTTP.REQ.URL' '"/home/"+ HTTP.REQ.URL.HTTP_URL_SAFE.AFTER_STR ("~").PREFIX(1)+"/"+ HTTP.REQ.URL.HTTP_URL_SAFE.AFTER_STR ("~").BEFORE_STR("/")+"/.www"+ HTTP.REQ.URL.HTTP_URL_SAFE.SKIP (\'/\',1)'
Add rewrite policy pol1 'HTTP.REQ.URL.PATH.STARTSWITH("/~")' act1
Bind rewrite global pol1 100

Redirect Failing URLs to Another Web Server

Description

In this scenario, the current URL is not valid and must be redirected to another web server. The following steps could be taken.

Solution

You must verify whether the requested filename exists on the server or not. Therefore, if it fails, you must redirect it to another web server (say, webServerB.com). In the case of AppExpert, HTTPCallout is used to verify the presence of the file on server by running a script file_check.cgi on the server. The returned value from HTTPCallout is used to validate the policy.

The Script file_check.cgi takes the URL as the argument and checks for its presence on the server and returns True or False.

Apache_Rewrite:

RewriteCond /your/docroot/%{REQUEST_FILENAME} !-f
RewriteRule ^(.+) http://webserverB.com/$1 [R]

AppExpert - First Method:

add HTTPCallout Call
set policy httpCallout Call -IPAddress 10.102.59.101 -port 80 -hostExpr '"10.102.59.101"' -returnType BOOL -ResultExpr 'HTTP.RES.BODY(100).CONTAINS("True")' -urlStemExpr '"/cgi-bin/file_check.cgi"' -parameters query=http.req.url.path -headers Name("ddd")
add responder action act1 redirect '"http://webserverB.com"+ HTTP.REQ.URL.HTTP_URL_SAFE '
add responder policy pol1 '!HTTP.REQ.HEADER("Name").EXISTS && !SYS.HTTP_CALLOUT(call)' act1
bind responder global pol1 100

AppExpert - Second Method:

Use respondwith in responder action act1, with the help of respondwith, add custom headers, or send some data in body, and so on, like in this example some text in body is added.
add HTTPCallout Call
set policy httpCallout Call -IPAddress 10.102.59.101 -port 80 -hostExpr '"10.102.59.101"' -returnType BOOL -ResultExpr 'HTTP.RES.BODY(100).CONTAINS("True")' -urlStemExpr '"/cgi-bin/file_check.cgi"' -parameters query=http.req.url.path -headers Name("ddd")
add responder action act1 respondwith '"HTTP/1.1 302 Moved Temporarily\r\nLocation: http://webserverB.com"+HTTP.REQ.URL.HTTP_URL_SAFE +"\r\n\r\nHTTPCallout Used"'
add responder policy pol1 '!HTTP.REQ.HEADER("Name").EXISTS && !SYS.HTTP_CALLOUT(call)' act1
bind responder global pol1 100

Time-Dependent Rewriting

Description

Rewriting the URL based on the time.

Solution

Example: Changing the request foo.html to foo.day.html or foo.night.html according to the time.

Apache_Rewrite:

RewriteCond %{TIME_HOUR}%{TIME_MIN} >0700
RewriteCond %{TIME_HOUR}%{TIME_MIN} <1900
RewriteRule ^foo\.html$ foo.day.html [L]
RewriteRule ^foo\.html$ foo.night.html

AppExpert:

Add rewrite action act1 insert_before 'HTTP.REQ.URL.PATH.SUFFIX(\'.\',0)' '"day."'
Add rewrite action act2 insert_before 'HTTP.REQ.URL.PATH.SUFFIX(\'.\',0)' '"night."'
add rewrite policy pol1 'SYS.TIME.WITHIN(LOCAL 07h 00m,LOCAL 18h 59m)' act1
add rewrite policy pol2 'true' act2
bind rewrite global pol1 101
bind rewrite global pol2 102

From Old to New (internally)

Description

Assume you have recently renamed the page foo.html to bar.html and now want to provide the old URL for backward compatibility. Actually, you want users of the old URL to not even recognize that the pages were renamed.

Solution

Rewrite the old URL to the new one internally using the following rule, let the base directory be /~quux/.

Apache_Rewrite:

RewriteEngine on
RewriteBase /~quux/
RewriteRule ^foo\.html$ bar.html

AppExpert - Solution 1: Converting the ApacheRewrite example, as it is, to APPExpert

add rewrite action act1 replace 'HTTP.REQ.URL.AFTER_STR("/~quux").SUBSTR("foo.html")' '"bar.html"'
add rewrite policy pol1 'HTTP.REQ.URL.ENDSWITH("/~quux/foo.html")' act1
bind rewrite global pol1 100

AppExpert - Solution 2 Generic Case: The above case was not generic because it was expecting foo.html to be directly under directory /~quux/. A more generic case is to convert a file foo.html to bar.html without considering any base directory. Also, a request URL could come with some query.

Add rewrite action act1 replace 'HTTP.REQ.URL.PATH.SUFFIX(\'/\',0)' '"bar.html"'
Add rewrite policy pol1 'HTTP.REQ.URL.PATH.CONTAINS("foo.html")' act1
Bind rewrite global pol1 100

From Old to New (externally)

Description

Assume that you have recently renamed the page foo.html to bar.html and now want to provide the old URL for backward compatibility. But this time, you want the users of the old URL to get directed to the new one, that is, their browsers Location field should change, too.

Solution

The following rules can force an HTTP redirect to the new URL, which leads to a change of the browsers and thus the user view:

Apache_Rewrite:

RewriteEngine on
RewriteBase /~quux/
RewriteRule ^foo\.html$ bar.html [R]

AppExpert - Solution 1: Converting the ApacheRewrite example, as it is, to APPExpert

add responder action act1 redirect 'HTTP.REQ.URL.HTTP_URL_SAFE.BEFORE_STR("foo.html")+"bar.html"'
add responder policy pol1 'HTTP.REQ.URL.ENDSWITH("/~quux/foo.html")' act1
bind responder global pol1 100

AppExpert - Solution 2 Generic Case: The above case was not generic because it was expecting foo.html to be directly under the /~quux/ directory. A more generic case is to convert a file foo.html to bar.html without considering any base directory. Also, a request URL could come with some query.

add responder action act1 redirect 'HTTP.REQ.URL.PATH.HTTP_URL_SAFE.BEFORE_STR("foo.html")+"bar.html"+HTTP.REQ.URL.HTTP_URL_SAFE.AFTER_STR("foo.html")'
add responder policy pol1 'HTTP.REQ.URL.PATH.CONTAINS("foo.html")' act1
bind responder global pol1 100

Browser Dependent Content

Description

At least for important top-level pages, it is sometimes necessary to provide the optimum of browser-dependent content, that is, you have to provide a maximum version for the latest Netscape variants, a minimum version for the Lynx browsers, and an average feature version for all others.

Solution

Act on the HTTP header "User-Agent". The following configuration does the following: If the HTTP header "User-Agent" begins with "Mozilla/3", then the page foo.html rewrites to foo.NS.html and the rewriting stops. If the browser is "Lynx" or "Mozilla", of version 1 or 2, the URL becomes foo.20.html. All other browsers receive page foo.32.html. This is done by the following rule set:

Apache_Rewrite:

RewriteCond %{HTTP_USER_AGENT} ^Mozilla/3.*
RewriteRule ^foo\.html$ foo.NS.html [L]
RewriteCond %{HTTP_USER_AGENT} ^Lynx/.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/[12].*
RewriteRule ^foo\.html$ foo.20.html [L]
RewriteRule ^foo\.html$ foo.32.html [L]

AppExpert:

Add patset pat1
Bind patset pat1 Mozilla/1
Bind Patset pat1 Mozilla/2
Bind patset pat1 Lynx
Bind Patset pat1 Mozilla/3
add rewrite action act1 insert_before 'HTTP.REQ.URL.SUFFIX' '"NS."'
add rewrite action act2 insert_before 'HTTP.REQ.URL.SUFFIX' '"20."'
add rewrite action act3 insert_before 'HTTP.REQ.URL.SUFFIX' '"32."'
add rewrite policy pol1 'HTTP.REQ.HEADER("User-Agent").STARTSWITH_INDEX("pat1").EQ(4)' act1
add rewrite policy pol2 'HTTP.REQ.HEADER("User-Agent").STARTSWITH_INDEX("pat1").BETWEEN(1,3)' act2
add rewrite policy pol3 '!HTTP.REQ.HEADER("User-Agent").STARTSWITH_ANY("pat1")' act3
bind rewrite global pol1 101 END
bind rewrite global pol2 102 END
bind rewrite global pol3 103 END

Blocking of Robots

Description

You can block a really annoying robot from retrieving pages of a specific web area and ease-up the traffic at some directories.

Solution

This could be done by using a rule set which forbids the URLs of the web area /~quux/foo/arc/. This could also be accomplished by matching the User-Agent HTTP header information. Let the IP addresses to block be 10.45.67.8 and 10.45.67.9.

Apache_Rewrite:

RewriteCond %{HTTP_USER_AGENT} ^NameOfBadRobot.*
RewriteCond %{REMOTE_ADDR} ^10\.45\.67\.[8-9]$
RewriteRule ^/~quux/foo/arc/.+ - [F]

AppExpert:

add responder action act1 respondwith '"HTTP/1.1 403 Forbidden\r\n\r\n"'
add responder policy pol1 'HTTP.REQ.HEADER("User_Agent").STARTSWITH("NameOfBadRobot")&& (CLIENT.IP.SRC.EQ(10.45.67.8) || CLIENT.IP.SRC.EQ(10.45.67.9)) && HTTP.REQ.URL.STARTSWITH("/~quux/foo/arc")' act1
bind responder global pol1 100

Blocked Inline-Images

Description

Assume you have under http://www.example.de/~quux/ some pages with inline GIF graphics. These graphics are nice, so others directly incorporate them using hyperlinks to their pages. You do not like this practice because it adds useless traffic to your server.

Solution

You can restrict the cases where the browser sends an HTTP Referer header.

Apache_Rewrite:

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://www.example.de/~quux/.*$
RewriteRule .*\.gif$        -                [F]

AppExpert:

Add patset pat1
Bind patset pat1 .gif
Bind patset pat1 .jpeg
add responder action act1 respondwith '"HTTP/1.1 403 Forbidden\r\n\r\n"'
add responder policy pol1 '!HTTP.REQ.HEADER("Referer").EQ("") && !HTTP.REQ.HEADER("Referer").STARTSWITH("http://www.example.de/~quux/")&&HTTP.REQ.URL.ENDSWITH_ANY("pat1")' act1
bind responder global pol1 100

Creating Extensionless Links

Description

Sometimes you might want to support extensionless links, either to hide extension from end users or to make the URL easy to remember.

Solution

This behavior is achieved by adding the extension on the fly using the following rewrite rules.

Case 1: Add a .php extension to all requests

Apache_Rewrite:

RewriteRule ^/?([a-z]+)$ $1.php [L]

AppExpert:

Add rewrite action act1 insert_after 'HTTP.REQ.URL' '".php"'
add rewrite policy pol1 'HTTP.REQ.URL.PATH.REGEX_MATCH(re#^/([a-z]+)$#)' act1
Bind rewrite global pol1 100

Case 2: If you have a mixture of both .html and .php files , then you can use the following rules:

Apache_Rewrite:

RewriteCond %{REQUEST_FILENAME}.php -f
RewriteRule ^/?([a-zA-Z0-9]+)$ $1.php [L]
RewriteCond %{REQUEST_FILENAME}.html –f
RewriteRule ^/?([a-zA-Z0-9]+)$ $1.html [L]

AppExpert: Here HTTPCallout is used. Script file_check.cgi, hosted on 10.102.59.101, is used to check whether a provided argument is a valid file name or not.

add HTTPCallout Call_html
add HTTPCallout Call_php
set policy httpCallout Call_html -IPAddress 10.102.59.101 -port 80 -hostExpr '"10.102.59.101"' -returnType BOOL -ResultExpr 'HTTP.RES.BODY(100).CONTAINS("True")' -urlStemExpr '"/cgi-bin/file_check.cgi"' -parameters query=http.req.url+".html"
set policy httpCallout Call_php -IPAddress 10.102.59.101 -port 80 -hostExpr '"10.102.59.101"' -returnType BOOL -ResultExpr 'HTTP.RES.BODY(100).CONTAINS("True")' -urlStemExpr '"/cgi-bin/file_check.cgi"' -parameters query=http.req.url+".php"
Add patset pat1
Bind patset pat1 .html
Bind patset pat1 .php
Bind patset pat1 .asp
Bind patset pat1 .cgi
Add rewrite action act1 insert_after 'HTTP.REQ.URL.PATH' '".html"'
Add rewrite action act2 insert_after "HTTP.REQ.URL.PATH" '".php"'
Add rewrite policy pol1 '!HTTP.REQ.URL.CONTAINS_ANY("pat1") && SYS.HTTP_CALLOUT(Call_html)' act1
Add rewrite policy pol2 '!HTTP.REQ.URL.CONTAINS_ANY("pat1") && SYS.HTTP_CALLOUT(Call_php)' act2
Bind rewrite global pol1 100 END
Bind rewrite global pol2 101 END

Redirecting a Working URI to a New Format

Description

In an example, say that you have a set of working URLs that look like this: /index.php?id=nnnn. However, you would really like to change them to /nnnn and make sure search engines update their indexes to the new URI format. First, you must redirect the old URIs to the new ones so that search engines update their indexes, but you still must rewrite the new URI back to the old one so that the index.php script runs.

Solution

Place a marker code into the query string that visitors cannot see. Redirect from the old link to the new format only if the marker is not present in the query string. Then, rewrite the new format link back to the old format and add a marker to the query string.

Apache_Rewrite:

RewriteCond %{QUERY_STRING} !marker
RewriteCond %{QUERY_STRING} id=([-a-zA-Z0-9_+]+)
RewriteRule ^/?index\.php$ %1? [R,L]
RewriteRule ^/?([-a-zA-Z0-9_+]+)$ index.php?marker&id=$1 [L]

AppExpert:

Add responder action act_redirect redirect 'HTTP.REQ.URL.PATH.HTTP_URL_SAFE.BEFORE_STR("index.php")+HTTP.REQ.URL.QUERY.VALUE("id").HTTP_URL_SAFE'
Add responder policy pol_redirect '!HTTP.REQ.URL.QUERY.CONTAINS("marker")&& HTTP.REQ.URL.QUERY.VALUE("id").REGEX_MATCH(re/[-a-zA-Z0-9_+]+/) && HTTP.REQ.URL.PATH.CONTAINS("index.php")' act_redirect
Bind responder global pol_redirect 100 END
Add rewrite action act1 replace 'HTTP.REQ.URL.PATH.SUFFIX(\'/\',0)' '"index.phpmarker&id="+ HTTP.REQ.URL.HTTP_URL_SAFE.SUFFIX (\'/\',0)'
Add rewrite policy pol1 '!HTTP.REQ.URL.QUERY.CONTAINS("marker")' act1
Bind rewrite global pol1 100 END

Ensuring that a Secure Server is Used on Selected Pages

Description

In scenarios where you want to ensure that for some selected pages only secure server are used, use the following rules.

Solution

Apache_Rewrite:

RewriteCond %{SERVER_PORT} !^443$
RewriteRule ^/?(page1|page2|page3|page4|page5)$  https://www.example.com/%1 [R,L]

AppExpert - First Method:

Add responder action res_redirect redirect '"https://www.example.com"+HTTP.REQ.URL' -bypassSafetyCheck yes
Add responder policy pol_redirect '!CLIENT.TCP.DSTPORT.EQ(443)&&HTTP.REQ.URL.REGEX_MATCH(re/page[1-5]/)' res_redirect
Bind responder global pol_redirect 100 END

AppExpert - Second Method:

Add patset pat1
Bind patset pat1 page1
Bind patset pat1 page2
Bind patset pat1 page3
Bind patset pat1 page4
Bind patset pat1 page5
Add responder action res_redirect redirect '"https://www.example.com"+HTTP.REQ.URL.HTTP_URL_SAFE'
Add responder policy pol_redirect '!CLIENT.TCP.DSTPORT.EQ(443)&&HTTP.REQ.URL.CONTAINS_ANY("pat1")' res_redirect
Bind responder global pol_redirect 100 END

Objective

Create Regex expression to allow only certain traffic to be served 

Instructions

HTTP.REQ.URL.REGEX_MATCH(re~/<requirement>\/((?:(?:_all|[A-Za-z0-9\*,\+\-]*)\/)?(?:[A-Za-z0-9\*,\+\-]*\/)?_search\/?(?:\?.*)?)$~)

Issue/Introduction

This article describes NetScaler AppExpert policies for Apache mod_rewrite samples.
Powered by Wolken Software