Contents

Introduction

Prerequisites

Requirements

Components Used

Background

Configure CSR1000V exmem allocation

Template: Data Plane Heavy

Summary Steps

Verify

Template: Data Plane Normal

Summary Steps

Troubleshoot

Introduction

This document describes how to increase Data Plane (DP) Quantum Flow Processor (QFP) External Memory Manger (exmem) memory on Cisco Cloud Services Router 1000V (CSR1000V).

Prerequisites

Requirements

Cisco recommends that you have knowledge of these topics:

- CSR1000V routers.

- Cisco IOS®-XE.

- ESXi vCenter.

Components Used

The information in this document is based on these software and hardware versions:

- CSR1000V router deployed in VMware ESXi, 6.7.0

- Cisco IOS®-XE 16.12.6

The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.

Background

It is important to understand how Cisco IOS®-XE is built. With Cisco IOS®-XE, Cisco has moved to a Linux kernel and all the subsystems have been broken down into processes. All the subsystems that were inside Cisco IOS® before, now run as software processes within the Linux Operating System (OS). Cisco IOS® itself runs as a daemon (Cisco IOSd) within the Linux OS. Cisco IOS®-XE retains not only the same look and feel of the classic Cisco IOS®, but also its operation, support, and management.

Memory assigned to Virtual Machine (VM) is allocated to Cisco IOS® daemon (Cisco IOSd) and to Data Plane memory.

By default, memory allocation on Cisco IOSd is 2.5 GB and for Data Plane is 1.5 GB, the amount of memory available to Cisco IOSd is controlled by memory add-on licenses and certain feature licenses (broadband feature license) and it does not change when more memory is allocated to VM.

In Data Plane memory, exmem memory size on CSR1000V by default is 256MB (268435456 Bytes), which can very small amount for certain specific features configurations, like QoS Polices, NAT sessions, IPSec Security Associations, and so on. and when exmem high utilization is observed -

CSR_ESXi#show platform hardware qfp active infrastructure exmem statistics

QFP exmem statistics

Type: Name: DRAM, QFP: 0

Total: 268435456

InUse: 263066746

Free: 5368710

Lowest free water mark: 206885888

Type: Name: IRAM, QFP: 0

Total: 2097152

InUse: 218112

Free: 1879040

Lowest free water mark: 1879040

Type: Name: SRAM, QFP: 0

Total: 0

InUse: 0

Free: 0

Lowest free water mark: 0

Jul 5 08:016:32.486: %QFPOOR-2-LOWRSRC_PERCENT_CRIT: R0/0: cpp_ha: QFP 0 DRAM(EXMEM) at 99 percent, exceeds critical level 95

Exmem is the dynamic memory that allows the actual packet handling, used for features to scale and manages static and dynamic memory used by Cisco Packet Processor (CPP).

Note: Refer to Cisco CSR 1000v Memory Allocation for detailed Cisco IOSd memory add-on information.

Configure CSR1000V exmem allocation

This section describes how to configure Data Plane templates to increase QFP exmem size.

Template options available are -

control-plane-extra-heavy

control-plane-heavy

data-plane-heavy

data-plane-normal

service-plane-heavy

service-plane-medium

Note: This document does not cover control plane templates and service plane templates, for more details about these templates please refer to - Cisco CSR 1000v and Cisco ISRv Software Configuration Guide

Template: Data Plane Heavy

The next table shows the QFP exmem memory allocation for the Data Plane Heavy template.

VM Memory

QFP exmem Size

4GB

256MB (default value)

8GB

512MB

16GB

1024MB

Router#configure terminal

Enter configuration commands, one per line.  End with CNTL/Z.

Router(config)#platform resource data-plane-heavy

Please reboot to activate this template

Router(config)#end

Router#write memory

Building configuration...

[OK]

CSR_ESXi#reload

Summary Steps

1. enable

2. configure terminal

3. platform resource data-plane-heavy

4. end

5. write memory

6. reload

Verify

1. 8GB assigned to VM .

cisco CSR1000V (VXE) processor (revision VXE) with 2295972K/3075K bytes of memory.

Processor board ID 949XJJNMSHM

3 Gigabit Ethernet interfaces

32768K bytes of non-volatile configuration memory.

8105924K bytes of physical memory.

7774207K bytes of virtual hard disk at bootflash:.

0K bytes of WebUI ODM Files at webui:.

2. No changes observed in terms of memory allocated to Cisco IOSd and Data Plane after template change.

Router#show platform software vmemory info

Memory Upgrade Limits:

Total System Memory:            7915 MB

Memory From Upgrade Licenses:   N/A(Smart License Enabled)

Memory From Feature Licenses:   N/A(Smart License Enabled)

Memory Available For Upgrade:

Available System Memory:        3819 MB

Available Upgrade Licensed Memory:      N/A(Smart License Enabled)

Available Feature Licensed Memory:      N/A(Smart License Enabled)

Current Memory Allocation:

Cisco IOSD:   2560 MB (default) + 0 MB upgrade

Data Plane:     1536 MB (default) + 0 MB upgrade

3. QFP exmem total available is now 512MB.

Router#show platform hardware qfp active infrastructure exmem statistics

QFP exmem statistics

Type: Name: DRAM, QFP: 0

Total: 536870912

InUse: 61156352

Free: 475714560

Lowest free water mark: 475321344

Type: Name: IRAM, QFP: 0

Total: 2097152

InUse: 218112

Free: 1879040

Lowest free water mark: 1879040

Type: Name: SRAM, QFP: 0

Total: 0

InUse: 0

Free: 0

Lowest free water mark: 0

Router#show platform software cpu alloc

CPU alloc information:

Control plane cpu alloc: 0

Data plane cpu alloc: 1-7

Service plane cpu alloc: 0

Template used: CLI-data_plane_heavy

Template: Data Plane Normal

Router#configure terminal

Enter configuration commands, one per line.  End with CNTL/Z.

Router(config)#platform resource data-plane-normal

Please reboot to activate this template

Router(config)#end

Router#write memory

Building configuration...

[OK]

CSR_ESXi#reload

Summary Steps

1. enable

2. configure terminal

3. platform resource data-plane-normal

4. end

5. write memory

6. reload

Data Plane Normal, uses 256MB in QFP exmem regardless of amount of VM Memory assigned.

cisco CSR1000V (VXE) processor (revision VXE) with 2295972K/3075K bytes of memory.

Processor board ID 949XJJNMSHM

3 Gigabit Ethernet interfaces

32768K bytes of non-volatile configuration memory.

16363460K bytes of physical memory.

7774207K bytes of virtual hard disk at bootflash:.

0K bytes of WebUI ODM Files at webui:.

Router#show platform software vmemory info

Memory Upgrade Limits:

Total System Memory:            15979 MB

Memory From Upgrade Licenses:   N/A(Smart License Enabled)

Memory From Feature Licenses:   N/A(Smart License Enabled)

Memory Available For Upgrade:

Available System Memory:        11883 MB

Available Upgrade Licensed Memory:      N/A(Smart License Enabled)

Available Feature Licensed Memory:      N/A(Smart License Enabled)

Current Memory Allocation:

Cisco IOSD:   2560 MB (default) + 0 MB upgrade

Data Plane:     1536 MB (default) + 0 MB upgrade

Router#show platform hardware qfp active infrastructure exmem statistics

QFP exmem statistics

Type: Name: DRAM, QFP: 0

Total: 268435456

InUse: 61156352

Free: 207279104

Lowest free water mark: 206885888

Type: Name: IRAM, QFP: 0

Total: 2097152

InUse: 218112

Free: 1879040

Lowest free water mark: 1879040

Type: Name: SRAM, QFP: 0

Total: 0

InUse: 0

Free: 0

Lowest free water mark: 0

Note: QFP exmem memory allocation depends solely on the amount of memory assigned to VM and the data plane template applied. It does not depend on memory add-on licenses installed.

Troubleshoot

There is currently no specific information available on how to troubleshoot this configuration.