Contents

Introduction

Background Information

Methodology

Practice

Introduction

This document describes how to troubleshoot Generic Routing Encapsulation (GRE) interface issues in an SD-WAN environment.

Background Information

In Cisco Viptela Solution, the use cases for GRE Interfaces include:

- Send traffic to ZScaler (HTTP-Proxy)  via vSmart Data-Policy or locally.

- Primary service GRE interface with default-back-up to the data center.

- Service chaining

There are cases, when the GRE interface may not come up and/or not working.

In those situations, check for

- GRE Interface is up/up via: show interface gre*

- GRE Keepalives via: show tunnel gre-keepalives

Methodology

If there is an issue, configure an Access Control List (ACL or access-list) to see if the GRE (47) packets are going out/in.

You are unable to see the GRE packets via TCP Dump, as the packets are generated by the fast path.

Sometimes, because of network address translation (NAT), GRE Keepalives can be dropped. In this case, disable the keepalive and see if the tunnel comes up.

Also, if the GRE Tunnel is constantly flapping and disabling keepalives, this keeps the interface up/up.

However, it has a drawback, where if there is a legitimate issue, it is hard to find out that GRE does not work.

See here in the document which shows an example.

This is a working GRE interface config

IN VPN0

vpn 0

interface gre1

ip address 192.0.2.1/30

tunnel-source      <SRC-IP>

tunnel-destination <DST-IP>

tcp-mss-adjust     1300

no shutdown

!

interface gre2

ip address 192.0.2.5/30

tunnel-source      <SRC-IP>

tunnel-destination <DST-IP>

tcp-mss-adjust     1300

no shutdown

!

!

IN Service side

vpn <SRVC-VPN>

service FW interface gre1 gre2

In Cisco SD-WAN solution based on vEdge routes, GRE interfaces working as Active-standby and not Active-Active.

At any given time, there is only GRE Interface which is in Up/Up state.

Practice

Create a policy for access-lists

vEdge# show running-config policy access-list

policy

access-list GRE-In

sequence 10

match

protocol 47

!

action accept

count gre-in

!

!

default-action accept

!

access-list GRE-Out

sequence 10

match

protocol 47

!

action accept

count gre-out

!

!

default-action accept

!

!

vEdge#

Create counters gre-in and gre-out and then you need to apply ACL to the interface (our tunnel rides over ge0/0).

The above ACL can be applied with the source address of the physical interface and destination address of the GRE endpoint.

vEdge# show running-config vpn 0 interface ge0/0

vpn 0

interface ge0/0

ip address 198.51.100.1/24

tunnel-interface

encapsulation ipsec

max-control-connections 1

allow-service all

no allow-service bgp

allow-service dhcp

allow-service dns

allow-service icmp

no allow-service sshd

no allow-service netconf

no allow-service ntp

no allow-service ospf

no allow-service stun

!

no shutdown

access-list GRE-In in

access-list GRE-Out out

!

!

vEdge#

Now you can see the counters for GRE packets in and out because these are in the fast path, one cannot see with tcpdump utility.

vEdge# show policy access-list-counters

COUNTER

NAME     NAME     PACKETS  BYTES

----------------------------------

GRE-In   gre-in   176      10736

GRE-Out  gre-out  88       2112

vEdge#

This is our GRE tunnel.

vEdge# show interface gre1

IF      IF      IF                                                               TCP

AF                ADMIN   OPER    TRACKER  ENCAP  PORT                              SPEED          MSS                 RX       TX

VPN  INTERFACE  TYPE  IP ADDRESS  STATUS  STATUS  STATUS   TYPE   TYPE     MTU   HWADDR             MBPS   DUPLEX  ADJUST  UPTIME      PACKETS  PACKETS

---------------------------------------------------------------------------------------------------------------------------------------------------------

0    gre1      ipv4  192.0.2.1/30 Up    Up      NA       null   service  1500  05:05:05:05:00:00  1000   full    1420    0:07:10:28  2968     2968

vEdge#

vEdge# show running-config vpn 0 interface gre1

vpn 0

interface gre1

ip address 192.0.2.1/30/30

tunnel-source-interface ge0/0

tunnel-destination 192.0.2.5/30

no shutdown

!

!

vEdge#

You can verify if the traffic is going on the GRE interface via show app cflowd flows command.

This is a sample example shows bi-directional traffic (both from ingress and egress):

vEdge# show app cflowd flows

TCP                                                                                  TIME    EGRESS  INGRESS

SRC   DEST         IP     CNTRL  ICMP                 TOTAL    TOTAL      MIN  MAX                             TO      INTF    INTF

VPN  SRC IP        DEST IP        PORT  PORT   DSCP  PROTO  BITS   OPCODE  NHOP IP      PKTS     BYTES      LEN  LEN   START TIME                EXPIRE  NAME    NAME

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------

10   203.0.113.1  203.0.113.11  61478 443    0     6      16     0       203.0.113.254 3399     286304     60   1339  Sun Apr  8 10:23:05 2018  599     gre1    ge0/6

10   203.0.113.11  203.0.113.1  443   61478  0     6      24     0       203.0.113.1262556     192965     40   1340  Sun Apr  8 10:23:05 2018  592     ge0/6   gre1

An example of disabling  keepalives (KA) on the GRE interface:

Default KA is 10 (hello-interval) and 3 (tolerance)

A KA of 0 0, disables the KA on the GRE interface.

vEdge# show running-config vpn 0 interface gre* | details

vpn 0

interface gre1

description         "Primary ZEN"

ip address <ip/mask>

keepalive 0 0

tunnel-source       <SRC-IP-Addr>

tunnel-destination  <DST-IP-Addr>

no clear-dont-fragment

mtu                 1500

tcp-mss-adjust      1300

no shutdown

!

A GRE Interface which is UP/Down shows as UP/UP (by passing the KA check).

See, TX counter here as it increases when KA is OFF.  It means, vEdge is TX the packets, but you don't see the increase in RX counter, which points to a remote issue.

vEdge# show interface gre*

IF      IF                                                              TCP

ADMIN   OPER    ENCAP  PORT                              SPEED          MSS                 RX         TX

VPN  INTERFACE  IP ADDRESS       STATUS  STATUS  TYPE   TYPE     MTU   HWADDR             MBPS   DUPLEX  ADJUST  UPTIME      PACKETS    PACKETS

---------------------------------------------------------------------------------------------------------------------------------------------------

### With KA ON

0    gre1       192.0.2.1/30  Up      Down    null   service  1500  cb:eb:98:02:00:00  -      -       1300    -           413218129  319299248

### With KA OFF

0    gre1       192.0.2.1/30  Up      Up      null   service  1500  cb:eb:98:02:00:00  100    half    1300    0:00:01:19  413218129  319299280