Enabling NXDB and Installing the OVSDB Plugin

This chapter includes the following sections:

Licensing Requirements for NXDB

This table lists the license package required for using NXDB on the Cisco Nexus 9300 and 9300-EX Series switches.

License

Product ID

TP Services Package (TP_SERVICES_PKG)

N93-TP1K9=(SPARE)

For information on how to apply the license, see the instructions in the Cisco NX-OS Licensing Guide.

Prerequisites for NXDB

The following components are required for NXDB:

  • A Cisco Nexus 9300 or 9300-EX Series switch running Cisco NX-OS Release 7.0(3)I6(1) or a later release

  • Any NSXv controller that follows the OVSDB RFC 7047 and the Open vSwitch Manual

  • The OVSDB plugin that is compatible with your Cisco NX-OS release and Cisco device:

    Cisco NX-OS Release

    OVSDB Plugin Release

    JRE

    NSXv Release

    Cisco Switches

    7.0(3)I6(x)

    2.1.x

    jre-8u112-linux-x64.rpm

    6.3.x

    9300 and 9300-EX

    7.0(3)I7(1) or 7.0(3)I7(2)

    2.2.x

    jre-8u112-linux-x64.rpm

    6.3.x

    9300 and 9300-EX

    7.0(3)I7(3) or 7.0(3)I7(4)

    2.3.x

    jre-8u112-linux-x64.rpm

    6.3.x

    9300 and 9300-EX

    Note

    OVSDB plugin images are backward compatible with prior switch images. For example, 2.2.x will work with the 7.0(3)I6(x) switch image.

Installing the Cisco NX-OS Image on the Switch

Before you can enable NXDB, you must install a version of the Cisco NX-OS image that supports NXDB. For instructions, refer to the Cisco Nexus 9000 Series NX-OS Software Upgrade and Downgrade Guide.


Note

Cisco NX-OS Release 7.0(3)I6(1) and later releases support the NXDB feature.

What to do next

Enable NXDB on the switch.

Enabling NXDB on the Switch

You can enable the switch to be configured via JSON-RPC NX-API calls.

Procedure

Step 1

configure terminal 

switch# configure terminal
switch(config)#

Enters global configuration mode.

Step 2

feature nxapi 

switch(config)# feature nxapi

Enables the NX-API feature.

Step 3

feature nxdb 

switch(config)# feature nxdb

Enables NXDB on the switch, which allows the switch to be configured via JSON-RPC NX-API calls.

Step 4

nxapi use-vrf {default | management} 

switch(config)# nxapi use-vrf management

Specifies the VRF to use for NX-API. Choose the management option if the connection to the controller is through the management VRF. The default option specifies the default VRF.

Step 5

(Optional) copy running-config startup-config 

switch(config)# copy running-config startup-config

Copies the running configuration to the startup configuration.

What to do next

Assign switch resources for use by the external controller.

Assigning Switch Resources for Use by the External Controller

You can assign both VLANs and interfaces to the external controller.

Guidelines and Limitations for Assigning Switch Resources

VLANs that are assigned to the external controller have the following guidelines and limitations:

  • The assigned VLANs must not already exist on the system.

  • The assigned VLANs can be configured only as dedicated resources, which means that only the external controller can push down VLAN-related configurations.

  • The VLANs are either completely owned by the external controller or completely owned by the switch. If the VLAN is owned by the external controller, the switch cannot configure the port membership for that VLAN. If the VLAN is owned by the switch, any configuration that the controller sends down will be blocked.

Interfaces that are assigned to the external controller have the following guidelines and limitations:

  • The Ethernet and port-channel interfaces that are exposed to the external controller must be valid interfaces.

  • Virtual port channels (vPCs) are supported.

  • vPC domains should be configured with the delay peer-link timer (using the delay peer-link seconds command). The recommended value is 600 seconds but needs to be adjusted based on the scale.

  • The Ethernet and port-channel interfaces can be configured only as shared resources, which means that the configuration for these resources can be driven from both the switch CLI and the external controller.

  • If an interface is already assigned, it cannot be changed to an access mode interface.

  • OVSDB plugin 2.3.x and later versions support access ports and trunk ports with a native VLAN configuration assigned to the controller. For previous OVSDB plugin versions, an assigned interface is required to be a trunk port without a native VLAN configured.

  • FEX interfaces are not supported.

  • An assigned interface cannot be configured as a SPAN destination port, and a SPAN destination port cannot be configured as an assigned interface.

Default Behavior of Trunk Interface Commands

The default behavior varies for some trunk interface commands, depending on whether you are entering them on the switch or you are using them to assign interfaces to be used by the external controller.

  • switchport mode trunk—By default, this command allows all of the VLANs configured in the system to be brought up on this port. However, when a trunk interface is assigned for use by the external controller, only the VLANs that are not assigned are brought up on this port by default. All assigned VLANs are not brought up.

  • switchport trunk allowed vlan vlan—This command can now only accept the non-assigned VLANs. If you try to configure an assigned VLAN, an error message appears.

Assigning VLANs and Interfaces

In order for the switch to accept configurations from an external controller, you must identify the VLANs and interfaces whose configuration can be done from the external controller.


Note

You will specify the controller IP address later in Configuring the OVSDB Plugin Using CLI Commands.

Procedure

Step 1

configure terminal 

switch# configure terminal
switch(config)#

Enters global configuration mode.

Step 2

controller type l2-vxlan identifier number 

switch(config)# controller type l2-vxlan identifier 1
switch(config-ctrlr-type)#

Specifies the assigned interfaces and VLANs for a given controller.

Step 3

assign interface port-channel channel-number [shared] 

switch(config-ctrlr-type)# assign interface port-channel 100 shared

Assigns a vPC interface to the controller.

Note 

In vPC setups, you must assign an MCT port channel to the controller. However, the MCT port channel will not be visible in the NSX controller GUI.

Step 4

[no] assign vlan vlan-range dedicated 

switch(config-ctrlr-type)# assign vlan 501 dedicated

Assigns the VLANs that can be managed by the external controller.

Step 5

[no] assign interface {ethernet port/slot | port-channel port-channel-list} shared 

switch(config-ctrlr-type)# assign interface ethernet 1/4, ethernet 1/17 shared

Assigns the Ethernet or port-channel interfaces that can be managed by the external controller.

Assigned interfaces in a vPC pair are appended with the serial number of the device on the controller. For example, if eth1/1 and eth1/2 are assigned to a vPC pair with serial numbers SAL1951VJF5 and SAL1952VJF5, the ports are visible in the NSX controller GUI as follows: eth1/1_SAL1951VJF5 and eth1/2_SAL1952VJF5. However, vPC port-channel interfaces that are assigned to the controller will not have the serial number suffixes.

Note 

After running no assign then assigning back the VLANs in the controller context, the ovsdb-plugin will need to be restarted using the following command: guestshell run sudo ovsdb-plugin service restart .

Step 6

controller description string 

switch(config-ctrlr-type)# controller description externalcontroller

Describes or specifies a name for the external controller.

Step 7

(Optional) copy running-config startup-config 

switch(config-ctrlr-type)# copy running-config startup-config

Copies the running configuration to the startup configuration.

What to do next

Install the OVSDB plugin.

Installing the OVSDB Plugin

Once the switch has been configured to enable NXDB, you can install the OVSDB plugin.


Note

If this is not a new installation and you are upgrading the OVSDB plugin, see Upgrading the Cisco NX-OS Image and the OVSDB Plugin.

Before you begin

Locate the OVSDB plugin and digital signature key (GPG key) on Cisco.com. To do so, go to https://software.cisco.com/download/home, search for and select your Cisco switch in Select a Product (for example, 9372), and click NX-OS Other Software. Then download the public GPG key and the OVSDB plugin for NSXv controller integration files.

Procedure

Step 1

Install the digital signature key.

switch# run guestshell sudo rpm --import /bootflash/arm-Nexus9k-rel.gpg
Step 2

Destroy and create the Guest Shell with the following recommended configuration for CPU, memory, and hard disk size. 

switch# guestshell destroy
switch# guestshell resize cpu 18
switch# guestshell resize mem 2500
switch# guestshell resize rootfs 1200
switch# guestshell enable
Caution 

The guestshell destroy command removes the entire contents of the Guest Shell, so the switch admin should be aware that all previously installed packages will be removed and must be reinstalled (if needed) after enabling the Guest Shell later.

Note 

On vPC setups, the above set of commands must be entered for both the primary and secondary vPC.
Step 3

Copy the JRE RPM file and OVSDB plugin RPM file to the bootflash. 

switch# copy scp://user@scpserver.cisco.com//download/JRE-RPM-file bootflash:JRE-RPM-file

switch# copy scp://user@scpserver.cisco.com//download/OVSDB-plugin-RPM-file bootflash:OVSDB-plugin-RPM-file
Step 4

Access the Guest Shell prompt. 

switch# run guestshell
[guestshell@guestshell ~]$
Step 5

Install the JRE RPM file in the Linux prompt. 

[guestshell@guestshell ~]$ sudo rpm -i /bootflash/JRE-RPM-file
Step 6

Install the OVSDB plugin RPM file. 

[guestshell@guestshell ~]$ sudo rpm -i /bootflash/OVSDB-plugin-RPM-file
Step 7

Verify the installation of the JRE RPM file and OVSDB plugin RPM file. 

[guestshell@guestshell ~]$ sudo rpm -qa | grep jre
[guestshell@guestshell ~]$ sudo rpm -qa | grep ovsdb

What to do next

Configure the switch for VXLAN.

Configuring VXLAN on the Switch

In order for the switch to accept Layer 2 VXLAN configurations from an external controller, you must configure some VXLAN settings on the switch.

Procedure

Step 1

configure terminal 

switch# configure terminal
switch(config)#

Enters global configuration mode.

Step 2

feature vn-segment-vlan-based 

switch(config)# feature vn-segment-vlan-based

Configures the global mode for all VXLAN bridge domains.

Step 3

feature nv overlay 

switch(config)# feature nv overlay

Enables the VXLAN feature.

Step 4

interface loopback number and ip address ip-address

Non-vPC Standalone Switch 

switch(config)# loopback interface 0
switch(config-if)# ip address 1.1.101.1/32

vPC Primary Switch 

switch(config)# loopback interface 0
switch(config-if)# ip address 1.1.101.1/32
switch(config-if)# ip address 1.1.106.1/32 secondary

vPC Secondary Switch 

switch(config)# loopback interface 0
switch(config-if)# ip address 1.1.102.1/32
switch(config-if)# ip address 1.1.106.1/32 secondary

Configures the loopback interface and the VTEP IP address of the switch on the loopback interface.

A non-vPC standalone switch requires a loopback interface with only one IP address. In a vPC-based system, a primary IP address and a secondary IP address are required on the loopback address of both vPC switches (as shown in this example). The primary IP address must be unique on the two vPC switches, and the secondary IP address must be common to both. The secondary IP address becomes the VTEP IP address of the vPC VTEP.

Step 5

exit 

switch(config-if)# exit
switch(config)#

Exits interface configuration mode.

Step 6

interface nve number 

switch(config)# interface nve 1
switch(config-if-nve)#

Creates a VXLAN overlay interface that terminates VXLAN tunnels. The network virtualization endpoint (NVE) interface serves as a single logical interface for the VXLAN network ports.

Note 

The switch supports only one NVE interface.

Step 7

source-interface loopback number 

switch(config-if-nve)# source-interface loopback 0

Configures a loopback interface with a valid /32 IP address as the source interface on the switch. This /32 IP address must be known by the transient devices in the transport network and the remote VXLAN tunnel endpoints (VTEPs). This requirement is accomplished by advertising the address through a dynamic routing protocol in the transport network.

Step 8

auto-remap-replication-servers 

switch(config-if-nve)# auto-remap-replication-servers

Automatically remaps traffic to a different replication node when a replication node is added or goes down.

Step 9

host-reachability protocol controller controller-name 

switch(config-if-nve)# host-reachability protocol controller 1

Specifies that the external controller will distribute the host reachability information (such as the MAC addresses and IP addresses of the host) in the network.

Step 10

config-source controller 

switch(config-if-nve)# config-source controller

Enables the switch to receive configurations from the controller.

Step 11

show nve peers 

switch(config-if-nve)# show nve peers 
Interface Peer-IP          State LearnType Uptime   Router-Mac       
--------- ---------------  ----- --------- -------- -----------------
nve1      10.0.133.1       Up    CP        4d00h    n/a              
nve1      10.0.134.1       Up    CP        4d00h    n/a              
nve1      10.0.140.1       Up    CP        4d00h    n/a              
nve1      10.0.142.142     Up    CP        4d00h    n/a              
nve1      10.0.143.143     Up    CP        2d23h    n/a              
nve1      10.0.151.1       Up    CP        4d00h    n/a              
nve1      10.0.152.1       Up    CP        4d00h    n/a              
nve1      10.0.153.1       Up    CP        4d00h    n/a              
nve1      10.0.154.1       Up    CP        4d00h    n/a              
nve1      10.0.155.1       Up    CP        4d00h    n/a              
nve1      10.0.156.1       Up    CP        4d00h    n/a              
nve1      10.0.157.1       Up    CP        4d00h    n/a              
nve1      10.0.158.1       Up    CP        4d00h    n/a              
nve1      10.0.159.1       Up    CP        4d00h    n/a              
nve1      10.0.160.1       Up    CP        4d00h    n/a              
nve1      10.0.161.1       Up    CP        4d00h    n/a              
nve1      10.0.162.1       Up    CP        4d00h    n/a              
nve1      10.0.163.1       Up    CP        4d00h    n/a              
nve1      10.0.164.1       Up    CP        4d00h    n/a              
nve1      10.0.165.1       Up    CP        4d00h    n/a              
nve1      10.0.166.1       Up    CP        4d00h    n/a

Displays the status of the switch's NVE peers.

Step 12

show nve vni ingress-replication 

switch(config-if-nve)# show nve vni ingress-replication

Interface VNI      Replication List  Source      Up Time      
--------- -------- ----------------- -------     -------      

nve1      10001    10.0.140.1        CONTROLLER  03:31:56    

nve1      10002    10.0.134.1        CONTROLLER  03:31:58    

nve1      10003    10.0.133.1        CONTROLLER  03:31:56    

nve1      10004    10.0.140.1        CONTROLLER  03:31:58

Displays the mapping of VNI to ingress-replication peer list and uptime for each peer.

The ingress-replication peer list is a list of the VTEP IP addresses to which BUM packets need to be unicast-replicated. For NSXv, the list always contains a single entry, which maps to one of the replication nodes in the topology. This replication node is chosen on the basis of hashing the available VNIs to the available replication nodes.

What to do next

Enable BFD over VXLAN.

Enabling BFD over VXLAN

Follow these steps to enable BFD over VXLAN.

Procedure

Step 1

Enable the BFD feature. 

switch# configure terminal
switch(config)# feature bfd
Step 2

Enable TCAM carving of the redirect-tunnel region. 

switch(config)# hardware access-list tcam region redirect-tunnel 256

On the Cisco Nexus 9300 switch, access control lists (ACLs) are implemented in TCAM regions in the hardware. To redirect BFD packets received over VXLAN, you need to install ACL rules in the hardware to redirect these packets to the supervisor module. Certain TCAM regions are enabled by default, and certain regions are not. Because the redirect-tunnel TCAM region is not enabled by default and is responsible for implementing the redirect ACL for BFD packets, it is required for this configuration. For TCAM changes to take effect in the hardware, a reboot is required after enabling this configuration.

Note 

This step is required only for Cisco Nexus 9300 Series switches. Cisco Nexus 9300-EX Series switches do not require TCAM carving.

Step 3

Define a BFD control VLAN on the switch and map it to control VNI 0. The BFD control frames are encapsulated with VNI 0. 

switch(config)# vlan 3000
switch(config-vlan)# vn-segment 0
Note 

You must explicitly specify the BFD VLAN, and it cannot be one of the already assigned VLANs.

Step 4

Define a BFD control SVI with IP forwarding. This command is required to punt the BFD packet to the Supervisor upon receive. 

switch(config)# interface Vlan3000
switch(config-if)# no shutdown
switch(config-if)# ip forward

What to do next

Assign NXDB roles.

Assigning NXDB Roles

Network-admin users can assign roles that limit access to NXDB operations on the switch.

Cisco NX-OS supports two NXDB roles for users who are configured for remote use through TACACS+:

  • nxdb-admin—Allowed to execute get and set JSON-RPC NX-API calls from the external controller.

  • nxdb-operator—Allowed to execute only get JSON-RPC NX-API calls from the external controller.

When NXDB is enabled, the nxdb-admin role is automatically assigned to the permanent user (admin).

Network-admin users can assign the nxdb-admin or nxdb-operator role to other users as necessary.


Note

Representational State Transfer (REST) requests using credentials received from TACACS+ behave as expected.

Procedure

Step 1

configure terminal 

switch# configure terminal
switch(config)#

Enters global configuration mode.

Step 2

username user-id [password [0 | 5] password] role {nxdb-admin | nxdb-operator} 

switch(config)# username NewUser password 5 4Ty18Rnt role nxdb-operator

Configures a user account with the specified NXDB role. The user-id argument is a case-sensitive, alphanumeric character string with a maximum length of 28 characters. Valid characters are uppercase letters A through Z, lowercase letters a through z, numbers 0 through 9, hypen (-), period (.), underscore (_), plus sign (+), and equal sign (=). The at symbol (@) is supported in remote usernames but not in local usernames.

The default password is undefined. The 0 option indicates that the password is clear text, and the 5 option indicates that the password is encrypted. The default is 0 (clear text).

Step 3

(Optional) show user-account 

switch(config)# show user-account

Displays the NXDB role configuration for remote users who log in directly to the switch.

Step 4

(Optional) copy running-config startup-config 

switch(config)# copy running-config startup-config

Copies the running configuration to the startup configuration.

What to do next

Configure the OVSDB plugin.

Configuring the OVSDB Plugin

The OVSDB plugin is installed on both the primary and secondary switches in a vPC setup. The software remains dormant on the secondary switch until that switch becomes the operational primary switch. The primary switch establishes a connection to the external controller.

You must configure the OVSDB plugin on both the primary and secondary vPC switches, by entering Cisco NX-OS CLI commands or running a configuration script.

Configuring the OVSDB Plugin Using CLI Commands

You can use CLI commands to configure the OVSDB plugin.

Procedure

Step 1

Make sure that the controller IP address is reachable before you configure the OVSDB plugin.

Step 2

View information on how to configure the OVSDB plugin. 

switch# guestshell run sudo ovsdb-plugin config set -h
usage: ovsdb-plugin config set [-h]
                                         [--switch-description SWITCH_DESCRIPTION]
                                         [--keep-test-config]
                                         [--log-level {error,warn,info,debug,trace}]
                                         [--log-server ADDRESS]
                                         [--log-type {file,udp}]
                                         [--max-json-peers INT]
                                         [--run-in-switch]
                                         [--schema-version {1.3.99,1.3.0}]
                                         [--vrf NAME] [--xms INT] [--xmx INT]
                                         [-v]
                                         ct_addr1[,ct_addr2]
                                         sw_addr1[,sw_addr2] user1[,user2]
                                         pswd1[,pswd2] name

positional arguments:
  ct_addr1[,ct_addr2]   Controller cluster address in IP:PORT format. Port
                        defaults to 6632 if not included. To specify two or
                        more controllers, separate them with a comma (no
                        spaces). E.g. 10.21.1.10:6632,10.21.1.11:6632
  sw_addr1[,sw_addr2]   Switch address in IP:PORT format. Port defaults to 443
                        if not includes. In VPC mode, specify two switches by
                        separating them with a comma (no spaces). E.g.
                        10.21.1.10:443,10.21.1.11:443
  user1[,user2]         Switch username(s). To specify a different username
                        for a second switch separate with a comma (no spaces).
                        E.g. user1,user2. If no username is given for the
                        second switch, the first one will be used
  pswd1[,pswd2]         Switch password(s). To specify a different password
                        for a second switch, separate with a comma (no
                        spaces). E.g. pswd1,pswd2. If no password is given for
                        the second switch, the first one will be used
  name                  Switch name.

optional arguments:
  -h, --help            show this help message and exit
  --switch-description SWITCH_DESCRIPTION
                        Switch description.
  --keep-test-config    Keep the test config
  --log-level {error,warn,info,debug,trace}
                        Log level to use. Defaults to info
  --log-server ADDRESS  When --log-type is set to UDP, use this to specify the
                        remote where the logs will be sent. The address must
                        be in IP:PORT format. PORT defaults to 514 if not
                        included
  --log-type {file,udp}
                        The type of logging to use. When set to file, the path
                        is always PLUGIN_ROOT/log/ovsdb-plugin.log. Defaults
                        to file
  --max-json-peers INT  Maximum number of JSON peers. Defaults to 6 if not
                        included. Set to -1 to have the plugin auto-compute
                        the value based on the number of controllers
  --run-in-switch       Configures the plugin for running in the switch
  --schema-version {1.3.99,1.3.0}
                        Set the schema version to use. Defaults to 1.3.0
  --vrf NAME            Used only when --run-in-switch is set. This configures
                        the plugin to use the given VRF name when
                        communicating with the controller. Defaults to
                        management
  --xms INT             Set initial Java heap size in MB. Defaults to 2048
  --xmx INT             Set maximum Java heap size in MB. Defaults to 2048
  -v, --verbose         Show extended information
Step 3

Configure the OVSDB plugin to establish connectivity to the switch and the controller.

Generic Syntax: Configuration on Non-vPC Switches 

switch# guestshell run sudo ovsdb-plugin config set --run-in-switch --vrf vrf-name --log-level debug
--log-type file controller-ip-address:controller-port switch-mgmt-ip-address
switch-username switch-password switch-name
Note 

The VRF name configured here should match the VRF configured in the nxapi use-vrf command on the switch.

Example: Configuration on Non-vPC Switches 

switch# guestshell run sudo ovsdb-plugin config set --run-in-switch –-vrf default --log-level debug --log-type file :1.1.128.252:6640
1.1.103.1 admin pswd1234 N9K-TOR1

Generic Syntax: Configuration on vPC Switches

On a vPC system, the configuration command must be run on both the vPC primary and secondary switches.

switch# guestshell run sudo ovsdb-plugin config set --run-in-switch --vrf vrf-name --log-level debug
--log-type file controller-ip-address:controller-port local-switch-ip,remote-switch-ip
local-switch-username,remote-switch-username local-switch-password,remote-switch-password,
common-switch-name --switch-description common-switch-description

Example: vPC Primary Configuration 

switch# guestshell run sudo ovsdb-plugin config set --run-in-switch –-vrf default --log-level debug --log-type file 1.1.128.252:6640
1.1.104.1,1.1.105.1 admin,admin pswd1234,pswd1234 TOR1-TOR2 --switch-description TOR1-TOR2

Example: vPC Secondary Configuration 

switch# guestshell run sudo ovsdb-plugin config set --run-in-switch –-vrf default --log-level debug --log-type file 1.1.128.252:6640
1.1.105.1,1.1.104.1 admin,admin pswd1234,pswd1234 TOR1-TOR2 --switch-description TOR1-TOR2
Note 

For NSXv, the controller port must be 6640. For vPC setups, the switch-name must be identical on both the vPC primary and secondary switches.

By default, the schema version used by OVSDB is 1.3.0.

Step 4

Verify the OVSDB plugin configuration.

Example: Non-vPC Switch 

switch# guestshell run sudo ovsdb-plugin config show

Controllers:
  #1 addr           : 2.2.128.252:6640 
VPC                 : No
In switch           : Yes
VRF                 : default
Switches:
  #1 addr           : 1.1.24.2:443
  type              : STANDALONE
  user              : admin
  name              : SAL184333WH
  description       : 
Log type            : file
Log level           : trace
Log server          : -
TTY log path        : -
Maxi JSON peers     : 6
Min heap size       : 2048 MB
Max heap size       : 2048 MB
Schema              : 1.3.0

Example: vPC Primary Switch 

switch# guestshell run sudo ovsdb-plugin config show

Controllers:
  #1 addr           : 2.2.128.252:6640 
VPC                 : Yes
In switch           : Yes
VRF                 : default
Switches:
  #1 addr           : 1.1.24.1:443
  type              : LOCAL
  user              : admin
  name              : ovsdb-plugin
  description       : 
  #2 addr           : 1.1.24.4:443
    type            : REMOTE
    user            : admin
    name            : ovsdb-plugin
    description     : 
Log type            : file
Log level           : trace
Log server          : -
TTY log path        : -
Maxi JSON peers     : 6
Min heap size       : 2048 MB
Max heap size       : 2048 MB
Schema              : 1.3.0

Example: vPC Secondary Switch 

switch# guestshell run sudo ovsdb-plugin config show

Controllers:
  #1 addr           : 2.2.128.252:6640 
VPC                 : Yes
In switch           : Yes
VRF                 : default
Switches:
  #1 addr           : 1.1.24.4:443
  type              : LOCAL
  user              : admin
  name              : ovsdb-plugin
  description       : 
  #2 addr           : 1.1.24.1:443
    type            : REMOTE
    user            : admin
    name            : ovsdb-plugin
    description     : 
Log type            : file
Log level           : trace
Log server          : -
TTY log path        : -
Maxi JSON peers     : 6
Min heap size       : 2048 MB
Max heap size       : 2048 MB
Schema              : 1.3.0

If you want to fine tune the configuration of the OVSDB plugin, see Extended OVSDB Plugin Configuration.

Step 5

Make sure that the configured switch IP address and the controller IP address are pingable before moving to the next procedure.

What to do next

Create a custom certificate, for Cisco NX-OS 7.0(3)I7(5) and later releases. See Creating a Custom Certificate.

Obtain the necessary certificates and run the OVSDB plugin, for releases prior to Cisco NX-OS Release 7.0(3)I7(5). See Running the OVSDB Plugin.

Configuring the OVSDB Plugin Using a Configuration Script

The following example shows how to use a configuration script to configure the OVSDB plugin (running inside a guestshell container on the switch) in a non-vPC system: 

cd to directory: /usr/local/ovsdb/bin/
[admin@guestshell bin]$ sudo ./ovsdb-plugin user-input
Running in switch? (yes/no) [y] yes
Would you like to configure OVSDB plugin in Dual Switch mode? (yes/no) [n]no
Enter the switch address in IP:PORT format. Port defaults to 443 if not includes >>10.23.237.22
Enter the Switch username. >> admin
Enter the switch password. >>
Password: 
Enter the Controller cluster address in IP:PORT format. Port defaults to 6632 if not included. To specify two or more controllers, separate them with a comma (no spaces). E.g. 10.21.1.10:6632,10.21.1.11:6632 >> 10.23.237.33:6640
Enter the switch name >> ovsdb-plugin
Enter the schema version to use. Defaults to 1.3.0.
Choices are: 
1. 1.5.1
2. 1.3.0
Enter the option >>2
Would you like to set the parameter --vrf? [default] management
Would you like to configure optional parameters? (yes/no) [n]yes
Enter the switch description >> node02
Configure default log level to use? (error/warn/info/debug/trace) [debug] debug
Would you like to configure the type of logging to use ? Options are  (file/udp) [file] 
Configuration saved

The following example shows how to use a configuration script to configure the OVSDB plugin (running inside a guestshell container on the switch) in a vPC system: 

cd to /usr/local/ovsdb/bin/
[admin@guestshell bin]$ sudo ./ovsdb-plugin user-input
Running in switch? (yes/no) [y] yes
Would you like to configure OVSDB plugin in Dual Switch mode? (yes/no) [n]yes
Enter the local switch address in IP:PORT format. Port defaults to 443 if not includes >>10.23.237.21
Enter the remote-peer switch address in IP:PORT format. Port defaults to 443 if not includes >> 10.23.237.24
Enter the Switch username. >> admin
Enter the second switch username in the VPC setup >> admin
Enter the switch password. >>
Password: 
Enter the second switch password. >>
Password: 
Enter the Controller cluster address in IP:PORT format. Port defaults to 6632 if not included. To specify two or more controllers, separate them with a comma (no spaces). E.g. 10.21.1.10:6632,10.21.1.11:6632 >> 10.23.237.33:6640
Enter the switch name >> ovsdb-plugin
Enter the schema version to use. Defaults to 1.3.0.
Choices are: 
1. 1.5.1
2. 1.3.0
Enter the option >>2
Would you like to set the parameter --vrf? [default] management
Would you like to configure optional parameters? (yes/no) [n]yes
Enter the switch description >> node02
Configure default log level to use? (error/warn/info/debug/trace) [debug] 
Would you like to configure the type of logging to use ? Options are  (file/udp) [file] 
Configuration saved

Creating a Custom Certificate

For Cisco NX-OS Release 7.0(3)I7(5) and later releases, you must create and apply a custom certificate to run the OVSDB plugin. Although earlier releases can use a default certificate, we recommend that custom certificates be used.

Procedure

Step 1

Enable the Bash shell. 

switch# feature bash
Step 2

Create OpenSSL custom SSL key files. 

switch# run bash openssl req -nodes -x509 -new -keyout /bootflash/ssl.key -out /bootflash/ssl.crt -days 1000 -subj /C=US/ST=CA/L='San Jose'/O='Cisco Systems Inc.'/OU=dcnxos/CN=nxos/
switch# run bash openssl rsa -in /bootflash/ssl.key -out /bootflash/ssl.key

The default certification keys are not supported.

Step 3

Configure the switch NX-API certificate settings with the custom certificate keys. 

switch# nxapi certificate httpscrt certfile ssl.crt
switch# nxapi certificate httpskey keyfile ssl.key
switch# nxapi certificate enable
Step 4

Verify that the custom certificate is applied in the switch. 

switch# run bash cat /nginx_1_fe/conf/nginx.conf | grep ssl
        ssl                  on;
        ssl_certificate      /var/nginx/cert/server.crt_vdc_1;
        ssl_certificate_key  /var/nginx/cert/server.key_vdc_1;
        ssl_session_timeout  5m;
        ssl_prefer_server_ciphers on;
        ssl_protocols TLSv1.1 TLSv1.2;
        ssl_ciphers …

The certificate lines must include “_vdc_1” (or “_restore” if the switch has been reloaded). If “_default” is displayed, the custom certificate is not applied.

What to do next

Obtain the certificates and run the OVSDB plugin. See Running the OVSDB Plugin.

Running the OVSDB Plugin

You need to obtain a certificate to run the OVSDB plugin.

Procedure

Step 1

Perform one of the following to generate a certificate for the switch:

  • If you are not using vPCs, enter this command: 
    switch# guestshell run sudo ovsdb-plugin cert bootstrap
  • If you are using vPCs, do the following:

    1. On the secondary vPC, run the following command to generate the certificate: 

      switch# guestshell run sudo ovsdb-plugin cert bootstrap --receive

    2. On the primary vPC, run the following command, paste the certificate that is shown as part of Step a, and press Enter. 

      switch# guestshell run sudo ovsdb-plugin cert bootstrap --send secondary-ip-address
Note 

The bootstrap option needs to be used only for the first certificate download on the plugin. For subsequent certificate downloads, the cert reset option needs to be used.

Step 2

View the switch's certificate. 

switch# guestshell run sudo ovsdb-plugin cert show
Step 3

Use the information in the output of the previous command to configure the switch's certificate on the controller GUI.

Step 4

Start the OVSDB plugin so that it connects to the switch and controller. 

switch# guestshell run sudo ovsdb-plugin service start

What to do next

Verify the connection status for the OVSDB plugin.

Verifying the Connection Status for the OVSDB Plugin

The OVSDB plugin establishes a connection toward the external controller on one side and toward the switch running Cisco NX-OS on the other. To verify the success of these connections, enter the guestshell run sudo ovsdb-plugin service status command.

Example Output on vPC Primary Switch: 

switch# guestshell run sudo ovsdb-plugin service status --more

Status: Running

Connections

  Switches:

      #1 addr         : 2.2.14.1
         type         : Local
         vpc          : Enabled/Primary
         state        : Up
         config ready : Up
         nxapi        : Up
         websocket    : Up

      #2 addr         : 2.2.14.4
         type         : Remote
         vpc          : Enabled/Secondary
         state        : Up
         config ready : Up
         nxapi        : Up
         websocket    : Up

  Controllers:

      #1 addr         : 2.2.128.254
         state        : Up

      #2 addr         : 2.2.128.253
         state        : Up

      #3 addr         : 2.2.128.252
         state        : Up

Example Output on vPC Secondary Switch: 

switch# guestshell run sudo ovsdb-plugin service status --more

Status: Running

Connections

  Switches:

      #1 addr         : 2.2.14.4
         type         : Local
         vpc          : Enabled/Secondary
         state        : Up
         config ready : Up
         nxapi        : Up
         websocket    : Up

      #2 addr         : 2.2.14.1
         type         : Remote
         vpc          : Enabled/Primary
         state        : Up
         config ready : Up
         nxapi        : Up
         websocket    : Up

  Controllers:

      #1 addr         : 2.2.128.252
         state        : Down

Note

The controller connection state on the vPC secondary switch is shown as Down because only the plugin running on the vPC primary switch connects to the controller.

Example Output on a Non-vPC Switch: 

switch# guestshell run sudo ovsdb-plugin service status --more

Status: Running

Connections

  Switches:

      #1 addr         : 1.1.24.2
         type         : Local
         vpc          : Disabled
         state        : Up
         config ready : Up
         nxapi        : Up
         websocket    : Up

  Controllers:

      #1 addr         : 2.2.128.253
         state        : Up

      #2 addr         : 2.2.128.254
         state        : Up

      #3 addr         : 2.2.128.252
         state        : Up