Configure pfSense Community Load Balancer for ECE

Introduction

This document describes the steps to setup and configure pfSense Community Edition as a Load Balancer for Enterprise Chat and Email (ECE).

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

• ECE 12.x
• pfSense Community Edition

Components Used

The information in this document is based on these software versions:

• ECE 12.6(1)
• pfSense Community Edition 2.7.2

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Install pfSense

Solution Overview

pfSense Community Edition is a multi-function product that provides a Firewall, Load Balancer, Security Scanner, and many other services in a single server. pfSense is built on Free BSD and has minimal hardware requirements. The Load Balancer is an implementation of HAProxy and an easy to use GUI is provided to configure the product. 

You can use this load balancer with both ECE and Contact Center Management Portal (CCMP). This document gives the steps to configure pfSense for ECE. 

Preparation

Step 1. Download pfSense Software

Use the pfSense website to download the iso installer image.

Step 2. Configure VM

Configure a VM with the minimum requirements:

• 64-bit amd64 (x86-64) compatible CPU

• 1GB or more RAM

• 8 GB or larger disk drive (SSD, HDD, etc)

• One or more compatible network interface cards

• Bootable USB drive or high capacity optical drive (DVD or BD) for initial installation

For a lab install, only one network interface (NIC) is required. There are several ways of running the appliance, but the easiest is with a single NIC, also called one-arm mode. In one-arm mode, there is a single interface that communicates to the network. While this is an easy way and adequate for a lab, it is not the most secure way.

A more secure way of configuring the appliance is to have at least two NICs. One NIC is the WAN interface and communicates directly with the public internet. The second NIC is the LAN interface, and communicates with the internal corporate network. You can also add additional interfaces to communicate with various parts of the network that have different security and firewall rules. For example, you can have one NIC connect to the public internet, one connect to the DMZ network where all the externally accessible web servers are, and a third NIC connect to the corporate network. This allows you to have internal and external users securely access the same set of web servers that are kept in a DMZ. Ensure that understand the security implications of any design before implementation. Consult with a security engineer to ensure best practices are followed for your specific implementation.

Installation

Step 1. Mount the ISO to the VM

Step 2. Power on the VM and follow the prompts to install.

Refer to this document for step-by-step instructions.

Network Setup

You must assign IP addresses to the appliance to continue configuration. 

Note: This document shows an appliance configured in one-arm mode.

Step 1. Configure VLANs

If you require VLAN support, answer y to the first question. Otherwise, answer n.

Step 2. Assign WAN Interface

The WAN interface is the non-secure side of the appliance in two-arm mode and the only interface in one-arm mode. Enter the interface name when prompted.

Step 3. Assign the LAN Interface

The LAN interface is the secure side of the appliance in two-arm mode. If required, enter the interface name when prompted.

Step 4. Assign any other Interfaces

Configure any other interfaces you require for your specific install. These are optional and not common.

Step 5. Assign IP Address to management interface

If your network supports DHCP, then the assigned IP address is shown in the console screen.

pfSense Console

If there is no address assigned, or if you wish to assign a specific address perform these steps.

1. Choose option 2 from the console menu.
2. Answer n to disable DHCP.
3. Enter the IPv4 address for the WAN interface.
4. Enter the netmask in bit counts. (24 = 255.255.255.0, 16 = 255.255.0.0, 8 = 255.0.0.0)
5. Enter the gateway address for the WAN interface.
6. If you would like this gateway to be the default gateway for the appliance, answer y to the gateway prompt, otherwise answer n.
7. Configure the NIC for IPv6 if desired. 
8. Disable DHCP Server on the interface.
9. Answer y to enable HTTP on the webConfigurator protocol. This is used in the next steps.

You then receive confirmation that the settings have been updated.

pfSense Confirmation

Complete Initial Setup

Step 1. Open a web browser and navigate to: http://<ip_address_of_appliance>

Note: You must use HTTP and not HTTPS initially.

pfSense Admin Login

Step 2. Login with the default login of admin / pfSense

Step 3. Complete the initial setup

Click next through the first two screens.

pfSense Setup Wizard - 1

Provide the host name, domain name, and DNS server information.

pfSense Setup Wizard - 2

Validate the IP Address information. If you initially chose DHCP, you can change this now. 

Provide the NTP Time server hostname and select the correct Timezone in the drop-down.

pfSense Setup Wizard - 3

Continue through the setup wizard until the end. The interface GUI restarts and you are redirected to the new URL once complete.

Configure Basic Admin Settings

Step 1. Login to the admin interface

Step 2. Select Advanced from the System drop-down menu

pfSense GUI - Admin Dropdown

Step 3. Update webConfigurator settings

pfSense GUI - Admin Configuration

1. Select the HTTPS (SSL/TLS) protocol.
2. Leave the SSL/TLS Certificate to the self-signed certificate at this time.
3. Change the TCP Port to a port other than 443 to better secure the interface and prevent issues with port overlap.
4. Select the WebGUI redirect option to disable the admin interface on port 80.
5. Select the Browser HTTP_REFERER enforcement option.
6. Enable Secure Shell by selecting the Enable Secure Shell option.

Note: Ensure that you select the Save button before you proceed. You are then redirected to the new https link.

Step 4. Configure Proxy Server if needed

If required, configure the proxy information on the Miscellaneous tab. To complete the setup and configuration, the appliance must have internet access.

pfSense GUI - Proxy Configuration

Note: Ensure that you select the Save button after making changes.

Add Required Packages

Step 1. Select System > Package Manager

Step 2. Select Available Packages

Note: It can take a few minutes to load all of the packages that are available. If this times out, verify that the DNS servers are configured correctly. Often, a reboot of the appliance fixes the internet connectivity.

pfSense GUI - Package List

Step 3. Find and Install required packages

1. haproxy 
2. Open-VM-Tools

Note: Do not select the haproxy-devel package.

Configure Certificates

pfSense can create self-signed certificate or it can integrate with a public CA, an internal CA, or can act as a CA and issue CA-signed certificates. This guide shows the steps to integrate with an internal CA.

Before you begin this section, ensure that you have these items available.

1. Root certificate for CA saved as either a PEM or Base-64 encoded format.
2. All intermediate (sometimes called issuing) certificates for CA saved as either a PEM or Base-64 encoded format.

Step 1. Select Certificates from the System drop-down menu

pfSense GUI - Certificates Dropdown

Step 2. Import the CA Root Certificate

pfSense GUI - CA Certificates List

Select the Add button.

pfSense GUI - CA Import

As shown in the image:

1. Provide a unique, descriptive name

2. Select Import an existing Certificate Authority from the Method drop-down.

3. Ensure that the Trust Store and Randomize Serial check-boxes are selected.

4. Paste the entire certificate into the Certificate data text box. Ensure that you include from the -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- lines.

5. Select Save. 

6. Verify that the Certificate is imported as shown in the image.

pfSense GUI - CA List

Step 3. Import the CA Intermediate Certificate

pfSense GUI - CA Intermediate Import

Repeat the steps to import the root CA certificate to import the intermediate CA certificate.

pfSense GUI - CA Links

Review the Certificate Authorities to ensure that the Intermediate is correctly chained to the root certificate as shown in the image.

Step 4. Create and Export a CSR for the load-balanced web-site

This describes the steps to create a CSR, export the CSR, then import the signed certificate. If you already have an existing certificate in a PFX format, you can import this certificate. Consult the pfSense documentation for these steps.

1. Select the Certificates menu, then select the Add/Sign button.

pfSense GUI - Certificates List

2. Complete the Certificate Signing Request form.

pfSense GUI - CSR Creation

• Method: Select Create a Certificate Signing Request from the drop-down
• Descriptive Name: Provide a name for the certificate
• Key type and Digest Algorithm: Review to ensure they match your requirements
• Common Name: Provide the fully qualified domain name web-site
• Provide the remaining certificate information as required for your environment

pfSense GUI - CSR Advanced

• Certificate Type: Select Server Certificate in the drop-down.
• Alternative Names: Provide any Subject Alternative Names (SAN) required for your implementation. 

Note: The common name is automatically added to the SAN field. You only need to add additional names required.

Select Save once all fields are correct.

3. Export the CSR to a file.

pfSense GUI - CSR Export

Select the Export button to save the CSR, then sign this with your CA. Once you have the signed certificate, save this as a PEM or Base-64 file to complete the process.

4. Import the signed certificate.

pfSense GUI - Certificate Import

Select the Pencil icon to import the signed certificate.

5. Paste the certificate data in the form.

pfSense GUI - Certificate Import

Select Update to save the certificate.

6. Review the certificate data to ensure it is correct.

pfSense GUI - Certificate List

7. Repeat this process if you wish to host multiple sites on this pfSense.

Add Virtual IPs

At least one IP is required to host websites on the pfSense. In pfSense this is done with Virtual IPs (VIPs).

Step 1. Select Virtual IPs from the Firewall drop-down

pfSense GUI - VIP Dropdown

Step 2. Select the Add button

pfSense GUI - VIP Landing Page

Step 3. Provide Address information

pfSense GUI - VIP Configuration

Use the information to add a VIP.

• Type: Select IP Alias
• Interface: Select the interface for this IP Address to be broadcast
• Address(es): Enter the IP Address
• Address Mask: For IP addresses used for load-balancing, the mask must be a /32
• Description: Provide a short text to make it easier to understand the configuration later

Select Save to commit the change.

Repeat this for each IP Address required for your configuration.

Step 4. Apply Configuration

pfSense GUI - VIP List

Select the, Apply Changes button after all VIPs have been added.

Configure Firewall

pfSense has a built-in firewall. The default rule-set is very limited. Before the appliance is put into production, ensure that you build a comprehensive firewall policy. 

Step 1. Select Rules from the Firewall drop-down

pfSense GUI - Firewall Rules Dropdown

Step 2. Select one of the Add buttons

pfSense GUI - Firewall Rules List

Note that one button adds the new rule above the selected line while the other adds the rule below the selected rule. Either button can be used for the first rule.

Step 3. Create firewall rule to allow traffic to port 443 for the IP address

pfSense GUI - Firewall Pass Rule Configuration

Use the information to create the rule.

• Action: Select Pass
• Interface: Choose the Interface the rule applies to
• Address Family and Protocol: Select as appropriate
• Source: Leave selected as Any
• Destination: Select Address or Alias from the Destination drop-down, then enter the IP address the rule applies to
• Destination Port Range: Select, HTTPS (443) in both the From and To drop-down
• Log: Select the check-box to log any packets which match this rule for accounting
• Description: Provide text to refer to the rule later

Select Save.

Step 4. Create a firewall rule to drop all other traffic to the pfSense

Select the Add button to insert the rule below the newly created rule.

pfSense GUI - Firewall Drop Rule Configuration

• Action: Select Block
• Interface: Choose the Interface the rule applies to
• Address Family and Protocol: Select as appropriate
• Source: Leave selected as Any
• Destination: Leave selected as Any
• Log: Select the check-box to log any packets which match this rule for accounting
• Description: Provide text to refer to the rule later

Select Save.

Step 5. Review the rules and ensure that the block rule is at the bottom

pfSense GUI - Firewall Rules List

If required, drag the rules to sort them.

Select, Apply Changes once the firewall rules are in the order required for your environment.

Configure HAProxy

HAProxy Concepts

HAProxy Concepts

HAProxy is implemented with a Frontend/Backend model. 

The Frontend defines the side of the proxy that customers communicate with. 

The Frontend consists of an IP and Port combination, certificate binding, and can implement some header manipulation.

The Backend defines the side of the proxy that communicates with the physical web servers.

The Backend defines the actual servers and ports, the loadbalancing method for initial assignment, health checks, and persistence.

A Frontend knows what backend to communicate with by either a dedicated backend or by using ACLs.

ACLs can create different rules so that a given frontend can communicate with different backends depending on various things.

Initial HAProxy Settings

Step 1. Select HAProxy from the Services drop-down

pfSense GUI - HAProxy Dropdown

Step 2. Configure basic settings

pfSense GUI - HAProxy Main Settings

Select the Enable HAProxy check-box.

Enter a value for Maximum Connections. See the chart in this section for details on the memory required. 

Enter a value for the Internal stats port. This port is used to show HAProxy statistics on the appliance but is not exposed outside of the appliance.

Enter a value for the Internal stats refresh rate.

Review the remaining configuration and update as required for your environment.

Select Save.

pfSense GUI - HAProxy Apply Changes

Note: Configuration changes are not made active until you select the, Apply Changes button. You can make multiple configuration changes and apply them all at one time. Configuration does not need to be applied to be used in another section. 

Configure HAProxy Backend

Start with the backend. The reason for this is that the frontend must reference a backend. Ensure that you have selected the Backend menu.

pfSense GUI - HAProxy Add Backend

Select the Add Button.

pfSense GUI - HAProxy Backend Start

Provide a name for the backend.

Select the down arrow to add the first server to the Server list

Backend - Server list

Provide a name to reference the server. This does not need to match the actual server name. This is the name that is shown on the stats page.

Provide the address for the server. This can be configured as either an IP Address for FQDN.

Provide the port to connect to. This must be port 443 for ECE.

Select the Encrypt(SSL) checkbox.

Provide a value in the Cookie field. This is the content of the session stickiness cookie and must be unique inside the backend.

After the first server has been configured, select the down arrow to configure any other web servers in the environment.

HAProxy Backend - Loadbalancing

Configure the Loadbalancing options. 

For ECE servers, this must be set to Least Connections.

HAProxy Backend - Health checking

Access control lists are not used in this configuration.

Timeout / retry settings can be left at their default configuration.

Configure the Heath checking section.

1. Health check method: HTTP
2. Check frequency: Leave blank to use the default of every 1 second.
3. Log checks: Select this option to write any health changes to the logs.
4. Http check method: Select GET from the list.
5. Url used by http check requests.: For an ECE server enter, /system/web/view/platform/common/login/root.jsp?partitionId=1
6. HTTP check version: Enter, HTTP/1.1\r\n\Host:\ {fqdn_of_server}

Ensure that you include a space after the final backslash but before the FQDN of the server. 

HAProxy Backend - Cookie Persistence

Leave the Agent checks unselected.

Configure Cookie persistence:

1. Cookie Enabled: Select to enable cookie based persistence.
2. Cookie Name: Provide a name for the cookie.
3. Cookie Mode: Select Insert from the drop-down box.
4. Leave the remaining options unset.

HAProxy Backend - HSTS

The remaining sections of the backend configuration form can be left at their default settings.

If you wish to configure HSTS, configure a timeout value in this section. ECE inserts an HSTS cookie as well so this configuration is redundant.

Select, Save.

Configure HAProxy Frontend

Change to the Frontend menu.

pfSense GUI - HAProxy Add Frontend

Select the, Add button

HAProxy - Frontend Header

Provide a name for the Front end.

Provide a description to help identify the frontend later.

In the External address table:

1. Listen address: Select the VIP you created for this website.
2. Port: Enter 443.
3. SSL Offloading: Select this option so that a the session cookie can be inserted.

Leave the Max connections empty.

Ensure the Type is selected as http / https(offloading).

HAProxy Backend - Default backend selection

The easiest configuration is to choose a Default Backend from the drop-down. This can be selected when the VIP hosts a single website. 

HAProxy Backend - ACL Advanced

As shown in the image, ACLs can be used to redirect a single frontend to multiple backends based on conditions. 

You can see that the ACL checks to see if the host in the request starts with a name and port number. or simply the name. Based on this a specific backend is used. 

This is not common with ECE.

HAProxy Frontend - Certificate binding

In the SSL Offloading section, select the certificate created for use with this site. This certificate must be a server certificate.

Select the option, Add ACL for certificate Subject Alternative Names.

You can leave the remaining options at their default values.

Select, Save at the end of this form.

HAProxy - Apply Configuration

Select, Apply Changes to commit the Frontend and Backend changes to the running configuration.

Congratulations, you have completed the setup and configuration of pfSense.