Why would setting certain categories to "Warn" or "Block" corrupt page layouts on some websites?

Question:

Why would setting certain categories to "Warn/Block" corrupt page layouts on some pages?

Environment:

• Cisco Web Security Appliance (WSA) any AsyncOS version
• Utilizing "Warn" or "Block" feature under URL categories

Symptoms: Some webpages shows up with incorrect layout when certain categories are configured to 'Block' or 'Warn'

When opening a web-page, a browser would typically make multiple HTTP requests through the WSA proxy. Each request would be independent and would be processed & categorized separately by WSA.

For example:

• Say you visit the wesbite http://www.example.com/index.html. Let's assume that this is categorized as "Computers and Internet"
• The "Index.html" page has a references to an image hosted on 'www.advertisements.com', which is categorized as "Advertisement".

Now, let's say we have the access policies on WSA (GUI > Web Security Manager > Access Policies), configured to "Block" the 'Advertisements' category and "Monitor" the 'Computers and Internet' category

• Based on the above Access Policies configuration, access to www.example.com is permitted, but access to www.advertisements.com is blocked.

1. When a user visits http://www.example.com/index.html on the browser, it makes a request to fetch index.html from www.example.com.
2. Next, looking at the downloaded html file, browser would make a request to fetch an image hosted on "www.advertisements.com".
3. When WSA receives this request, it blocks the transaction and returns an "End User Notification (EUN)" indicating that the requested HTTP request was blocked.
4. Browser receives a reply/blocked page from proxy, but it would not be able to render the requested "image" because EUN is in HTML. Instead, browser (for example, Internet Explorer) would show a 'red X' where the image should be displayed.

From the above example above, we can see that an "image" has been blocked. But not all objects are always visible. Example of such objects are java script files, style sheet files (css) etc. Java Script (JS), Style Sheet (CSS) would be executed in the background and browser will not notify the user when the request is blocked. When these objects are blocked, browser may not be able to render the page correctly and show you a page with incorrect layout.

If you come across a website or webpage which doesn't render correctly, please examine your accesslogs to determine which domain or website is being 'Blocked' or 'Warned' by WSA.

For more information on 'grepping' or examining the access logs, please visit the link below.
http://tinyurl.com/2l6qkw

Please refer to the attached excerpt from User Guide which provides detailed explanation on reading the access log output.

Once we find the domains being blocked (like www.advertisements.com in above example), we can take either of the below steps to correct the page layout

1. Configure the category associated with domain - Advertisements in the above example - to 'Monitor' instead of 'Block' or 'Warn'
   
   • You can do this in access policies under GUI --> Web Security Manager --> Access Policies > URL categories column
   • Note: This configuration change would un-block acess to all websites in the concerned category. Hence you should only follow this step if the 'blocked' category is impacting layouts on many websites
2. Configure a custom URL category (Under GUI --> Web Security Manager --> Custom URL categories) with the concerned domains (like advertisemnts.com, .advertisements.com) and configure to the custom URL category to 'Monitor' in access policies
   
   • This configuration will only permit the sites listed in the custom URL category and WSA will still continue to block other sites in the concerned category