Guest

Cisco CSS 11000 Series Content Services Switches

How ArrowPoint Cookies Function on the CSS 11000 Series In 4.x Code

Document ID: 15038

Updated: May 03, 2004

   Print


Contents


Introduction

In the world of E-commerce, many applications must have a client stuck to a certain server so that the application does not break. The purpose of this document is to provide a detailed explanation on the ArrowPoint cookie functionality.

Older Features

Before the Content Service Switch (CSS) had the added functionality of injecting cookies into the client packets, customers had a choice of the advanced-balanced methods listed below.

  • Sticky source IP

  • Sticky source IP/destination port

  • Cookies

  • Cookies-URL

  • Secure Socket Layer (SSL)

Some customers shy away from using the sticky source IP or sticky source IP/destination port in a content rule because they can run into the America Online (AOL) mega-proxy problem. Cisco recommends that the content rule be configured with a sticky-mask of 255.255.255.240 to avoid this situation.

Setting the servers to dish out cookies is the prefered method. This allows the content rule to take advantage of the advanced-balance methods of cookies and cookies-URL. (Cookies-URL will first look in the Hyper-Text Transfer Protocol (HTTP) GET of the packet for the cookie. If it does not find it there, it will look for it in the URL after a question mark.) Some customers are unaware of how to do this or do not have access to the servers to configure this.

SSL is used explicitly for encrypted traffic. It uses a different port than HTTP, and should be considered a different protocol. It is only mentioned in this document because it is a form of sticky.

New Features

The CSS 11000 series router has an added functionality in 4.01 code called ArrowPoint cookie injection. Instead of the server setting the cookie so that the client can stick to the server, the CSS will inject the cookie into the flow. The CSS will dish out and keep track of the stickiness without the need of the server.

Configuration

You must have a CSS that uses 4.01 software or higher. The CSS must have a content rule configured as a Layer 5 (L5) switch rule. The advanced balance method will need to be configured as ArrowPoint cookies.

Note: Trying to stick a client to a server while using SSL can not be done via server-side cookies or the ArrowPoint cookie injection method. The reason is because SSL will encrypt the body of the packet, and the CSS does not have the functionality to decrypt SSL. An advanced-balance method of SSL must be used.

The Command Line Interface (CLI) command to configure a L5 switch to use ArrowPoint cookies is as follows:

CSS800(coning-owner-Boston[Nomar])#advanced-balance arrowpoint-cookies

Sample Command Output:

Service A
     IP address 4.4.4.4
     String redsox
     Active

Owner Boston
     Content Nomar
      VIP address x.x.x.x
       Protocol TCP
       Port 80
       URL "/*"
       Advanced-balance arrowpoint-cookies
       Active

How It Works

When a L5 rule content rule is configured, all connections to that rule will be spoofed. This means that the CSS will wait until the client sends the HTTP GET before sending it back to the correct server. The Transfer Control Protocol (TCP) connection should look as follows:

Client                          CSS                     Server
SYN ----------- ->
                              <- -------------SYN ACK
ACK ---------- ->
HTTP GET -- ->                        

!-- At this point, the CSS can make a L5 load balancing decision.                                                                      

                            SYN---------------- ->
                                                           <- ----------SYN ACK
				            ACK --------------- ->
                            HTTP GET ------- ->

When a content rule is setup for ArrowPoint cookies, the CSS will wait for the HTTP GET before injecting the cookie. When the user first visits the site, the HTTP GET will not contain a cookie. The CSS always checks for the existence of the ArrowPoint cookie when it receives a client's request. If this cookie does not exist, the CSS performs server load balancing and generates a ArrowPoint cookie configured for the choosen server. Since this is the users first time visiting the site, the CSS will respond by sending the client back a HTTP refresh page that contains the cookie. The HTTP metatags in the response will prompt the client's browser to resend the HTTP GET with the cookie set by the CSS.

A TCP connection to a L5 rule configured with ArrowPoint cookies will look as follows:

Client                        CSS                       Server
SYN ----------- ->
                                    <- -------------SYN ACK
ACK ---------- ->
HTTP GET -- ->                         
                                     <- --------------HTTP Refresh packet with ArrowPoint Cookie                        
HTTP GET with ARPT cookie----- ->
!-- At this point, the CSS can make a L5 load balancing decision. 
                                   SYN---------------- ->
                                                              <- ----------SYN ACK
				                   ACK --------------- ->
                                   HTTP GET ------- ->

Below is a sniffer trace showing where the CSS sends the cookie, and what appears in the HTTP portion of the packet. The client address is 161.44.232.48. The Versatile Interface Processor (VIP) address is 172.27.66.72. This trace is taken between the client and the CSS:

Frame   Source Address    Dest. Address          Summary
1641   [161.44.232.48]   [172.27.66.72]   D=80 S=1847 SYNSEQ=25895616 
1642   [172.27.66.72]    [161.44.232.48]  D=1847 S=80 SYN ACK=25895617 
1643   [161.44.232.48]   [172.27.66.72]   D=80 S=1847 ACK=186008007
1644   [161.44.232.48]   [172.27.66.72]   HTTP: C Port=0 GET / HTTP/1.1
1645   [172.27.66.72]    [161.44.232.48]  HTTP: R Port=1847 HTML Data
1646   [161.44.232.48]  [172.27.66.72]    HTTP: C Port=0 GET / HTTP/1.1

The packet 1644 HTTP portion is as follows:

HTTP: ----- Hypertext Transfer Protocol -----
      HTTP: 
      HTTP: Line  1:  GET / HTTP/1.1
      HTTP: Line  2:  Accept: application/msword, image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, 
	             application/vnd.ms-excel, application/vnd.ms-powerpoint, */*
      HTTP: Line  3:  Accept-Language: en-us
      HTTP: Line  4:  Accept-Encoding: gzip, deflate
      HTTP: Line  5:  User-Agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows NT)
      HTTP: Line  6:  Host: 172.27.66.72
      HTTP: Line  7:  Connection: Keep-Alive
      HTTP: Line  8:  
HTTP: 

As you can see, the packet does not contain a cookie. The response packet sent back from the CSS, packet 1645, HTTP portion contains the following:

HTTP: ----- Hypertext Transfer Protocol -----
      HTTP: 
      HTTP: Line  1:  HTTP/1.1 200 OK
      HTTP: Line  2:  Set-Cookie: ARPT=IIOOQOSredsoxCKUQW; path=/
      HTTP: Line  3:  
      HTTP: Line  4:
HTTP: 

The client's browser will receive this response packet. The browser will re-send the HTTP GET that it set in packet 1644, except this packet will contain the cookie that the CSS has injected. When the CSS finds the cookie in the request, it unscrambles the cookie and validates it. If the cookie format is valid, the CSS forwards the client's request to the server indicated by the cookie or service identifier. Otherwise, the CSS treats this request as an initial request. Looking closely at the HTTP response packet that the CSS sends back to the client you will see the following:

Set-Cookie: ARPT=ITOUQOredsoxCKUQW

  • ITOUQO is the box identifier. Each box will have it's own.

  • redsox is the service identifier. This is the value you configure under the string option in the service. If no string is configured, the CSS will use the service IP as the string identifier.

  • CKUQW is the timeout value.

The HTML that is below the set-cookie parameter are HTTP metatags that tell the client's browser to refresh or go after the content in one second. This HTTP GET will contain the cookie.

ArrowPoint Cookie Expiration Date

The trace taken above shows that the CSS did not send the cookie injection packet with an expiration date. That means that the cookie will only last for the life of the browser. Once the browser is closed on the client, the cookie will be removed. When the client reopens the browser, the next time they go to the site, the whole process of injecting the cookie must happen again.

For customers that want to manually set an expiration date, this can be configured on the content rule, as shown below.

Arrowpoint-cookie expiration dd:hh:mm:ss

  • dd= number of days

  • hh= number of hours

  • mm= number of minutes

  • ss= number of seconds

All of these variables have valid numbers from 00 to 99.

Known Problems with Expiration Date

The problem with the ArrowPoint cookie expiration date is that the CSS does not know the real time because it does not support Network Time Protocol (NTP). NTP is scheduled for the 5.0 code release. In the 4.x code release, the CSS only knows the time that it has been configured for. Knowing that this time could be completely off, the expiration date set with an ArrowPoint cookie could be completely invalid.

To get around this, the code was written to set two expiration dates in the cookie injection packet. The first expiration date can be found in the HTTP header section. This value will be set with a date that is one year from the configured date on the CSS. This value will always be set to one year from the switch date, no matter what the value configured in the content rule. The second expiration date is the configured value in the ArrowPoint cookie content rule. This value will appear in the timeout value within the HTTP metatags set-cookie portion.

If we changed the previous content rule to have an expiration date, we would see the following:

Owner Boston
     Content Nomar
      VIP address x.x.x.x
       Protocol TCP
       Port 80
       URL "/*"
       Advanced-balance arrowpoint-cookies
       ArrowPoint cookies expiration 05:00:00:00    
       !-- The expiration date is set to five days.                                                                         
       Active

The HTTP portion of a cookie injected packet done by the CSS with an expiration date is as follows:

HTTP: ----- Hypertext Transfer Protocol -----
      HTTP: 
      HTTP: Line  1:  HTTP/1.1 200 OK
      HTTP: Line  2:  Set-Cookie: ARPT=IIOOQOSredsoxT0x0000000e_0xb4007cb3COKQY; 
	              expires=Fri, 15-FEB-2002 16:15:47 GMT; path=/
      HTTP: Line  3:  
      HTTP: Line  4:  

As you can see, the timeout value contains a much longer string than the trace that did not contain an expiration date in the content rule. The CSS hashes the five days expiration date value for security reasons.

The reason for expiring the cookie one-year from the date configured on the CSS, even though the ArrowPoint cookie was configured for five days, is to allow for error in the configured time on the CSS. The side effects of expiring the cookie in such a manner are:

    1. The browser caches the cookie for one-year, even though the content rule was configured for days, hours, or minutes.

    2. The client will stay stuck on the server it was first stuck to forever, unless the server goes down, the unit is upgraded to 5.0 code, or the user does not visit the site for one year. The reason the client is always stuck to the same server is because the injected cookie packet has an expiration date of one-year, even though you configured the content rule for less time.

Consider the scenario where we have configured the content rule to have an expiration date for five days. If a user comes back two days after first visiting the site, the HTTP GET will come into the CSS and the code will read the set-cookie portion in the packet. The expiration time will have not expired, so the CSS sticks it to the same server.

This process all seems acceptable, but consider the event where the client does not come back to the site for six days. In this case, you would assume that the cookie would not appear in the next HTTP GET. This would prompt the CSS to treat this as an initial request, load balancing the client and sticking it to a new server or the same server they were stuck to before. Unfortunately, this is not what happens, because the CSS inserts the expiration time into the HTTP header.

After six days, the client will set up a new TCP connection and a new HTTP GET. The HTTP GET packet will contain the cookie because the cookie has been stored on the browser for one-year, via the HTTP expiration date. The CSS will process the client's request. The expiration date in the ArrowPoint cookie timeout value will have expired. The CSS will send a new cookie injected packet back to the client. The response packet will stick the client to the same server and set the timeout value to five days. The packet will also contain the expiration date of one year from the configured date on the CSS in the HTTP headers section.

Each proceeding request to this site will contain a cookie as long as the user comes back within a year. The CSS will process this request and always stick the client to the same server and increase the time the cookie is cached in the browser. This will continue indefinitely unless the three reasons discussed above are met.

Note: This scenario will not happen if you do not set an ArrowPoint cookie expiration date. When no expiration date is configured on the content rule, the packet will not conatin an HTTP header with an expiration date set to one-year of the configured date on the CSS. The cookie will always be removed from the client's browser cache when it is closed down.

ArrowPoint Cookie Path

Using the ArrowPoint cookie path command, we can set the ArrowPoint cookie path to a configured path. Otherwise, the CSS sets the default path attribute to the cookie to "/". The syntax for this content mode command is as follows:

ArrowPoint-cookie path path name

Enter the path name where you want to send the cookie. Enter a quoted text string with a maximum of 99 characters. The default path of the cookie is "/".

For example:

CSS800(config-owner-Boston[Nomar})#ArrowPoint-cookie path "/cgi-bin/"

ArrowPoint Cookie Text Message and Cookie URL

When configuring a content rule with the advanced-balance ArrowPoint cookies option, the customer is developing their site based upon cookies. Some users of the Internet may disable the use of cookies in their browser. This produces a conflict between the browser and content rule. The result is that the user will not be able to gain access from the content rule configured with ArrowPoint cookies.

When a user disables cookies on the browser, the browser is configured not to cache cookies. The first time the user tries to get access to the site via the ArrowPoint cookie content rule, the CSS will inspect the packet. The cookie will not exist, so the CSS will perform server load balancing and generate an ArrowPoint cookie. This response packet will make its way back to the client, who will simply reject the cookie. The HTTP metatags inform the browser to access the site again in one second. The client will send another HTTP GET without a cookie. This packet will be inspected by the CSS, and once again an ArrowPoint cookie injection packet will be sent back. Internet Explorer will continue this endless loop until the user hits stop. Netscape's browser will issue a Page Not Found error after a few failed tries.

This issue should be resolved in 5.0 code by modifying how the ArrowPoint cookie is injected. Currently, there are two workarounds:

  • Since the customer's site is developed based on cookies, the site should inform users that only browsers with cookies enabled should access this part of the site. The site should provide a different means to access the content for users who disable cookies.

  • In version 4.01, build 10s and above, there were two features added to help inform and direct customers that have disabled cookies. These two features are the ArrowPoint cookie text message and the ArrowPoint cookie URL.

    Use the ArrowPoint cookie text message command to enable the display of the default text message, and to specify a custom text message to inform users to configure cookies.

    Use the ArrowPoint cookie URL to direct users to a URL for more information about enabling cookies in their browser. Both commands are entered in content rule. Below is a sample configuration with the syntax of these two commands:

    Owner Boston
         Content Nomar
          VIP address x.x.x.x
           Protocol TCP
           Port 80
           URL "/*"
           Advanced-balance arrowpoint-cookies
           Arrowpoint-cookie text "Please follow the link below for additional information."  
           Arrowpoint-cookie url "www.redsox.com/instructions/cookies-not-enabled.html"
           Active
    

Below is the page you would receive if your browser had cookies disabled:

Remember the following:

  • The string value for both the ArrowPoint cookie text and URL can be a maximum of 99 characters.

  • The default message when ArrowPoint cookie text message is configured is as follows:

    In order to effectively use our Website, your browser must have cookies configured. When the ArrowPoint cookie URL is configured, the CSS will automatically insert "http://" in front of the URL that you specify.

ArrowPoint Cookie Failover

When a client comes in with a valid cookie request but the sticky server is not available, the CSS uses the sticky-serverdown-failover configuration to handle the request.

By default, the sticky-serverdown-failover is configured as balance. The sticky-serverdown-failover balance method will treat the client's request as an initial request without the ArrowPoint cookie. It uses the load-balancing algorithm to choose a server, and then redirects the request with a generated ArrowPoint cookie.

The other option is a failover type of redirect. In this case, the CSS redirects the request to the specified URL.

The command sticky-no-cookie-found-action should not be configured in an ArrowPoint cookie content rule. Not only will this command not work, it produces many irregularities in the CSS.

Other Known Issues

HTTP POST

If the first packet sent in an application must be an HTTP POST, the application will not work with an ArrowPoint cookie content rule. An HTTP POST is used in the HTTP world to write information to a server. The HTTP spec states that when an HTTP POST is refreshed via metatags or is redirected via a 302 redirect, the sending station should resend the HTTP POST packet as an HTTP GET. The result is that the HTTP POST never gets sent to the server. The HTTP GET that is sent to the server will not be able to provide login information, and the application will fail to log in the client. This issue should be resolved in a later version of code by modifying the way the cookie is injected in the flow. Currently, the workaround is to use server side cookies or sticky source IP on the CSS.


Related Information


Updated: May 03, 2004
Document ID: 15038