Cisco IOS Voice Troubleshooting and Monitoring Guide, Release 12.4
Troubleshooting Fax Applications
Downloads: This chapterpdf (PDF - 356.0KB) The complete bookPDF (PDF - 2.89MB) | Feedback

Troubleshooting Fax Applications

Table Of Contents

Troubleshooting Fax Applications

Fax Call Flow

Fax Pass-Through and Fax Pass-Through with Upspeed

Cisco Fax Relay

Cisco Fax Relay Fax Setup Phase

Cisco Fax Relay Data Transfer Phase

T.38 Fax Relay

H.323 T.38 Fax Relay

SIP T.38 Fax Relay

MGCP T.38 Fax Relay

T.37 Store-and-Forward Fax

Fax Relay

Identifying and Isolating the Problem

Checking Basic Connectivity

Normal Voice Connectivity Problems

Configuration Problems Related to Dial Peers

Other Basic Connectivity Problems Not Relating to Dial Peers

Fax Connectivity Problems Across the PSTN

Checking for Slips and Other Errors on Digital Interfaces

Checking Fax Interface Type

Ensuring That the Fax Codec is Loaded During the Fax Call

Disabling Fax Relay and Change Codec for Pass-Through

Checking for Packet Loss on the Voice Network

Disabling Fax Relay ECM (Cisco Proprietary VoIP Only)

Enabling T.38 Packet Redundancy (T.38 VoIP Only)

Setting the Fax NSF Command to All Zeroes

Achieving the Final Stages of Resolution

Debugging Fax Relay

T.30 Messages

Fax Relay Debug Commands

Fax Analyzers

Fax Detection


Troubleshooting Fax Applications


This chapter describes T.37 store and forward fax and T.38 fax gateway troubleshooting concepts. The applications are T.37 store and forward fax, T.38 fax relay for VoIP H.323, fax relay packet loss concealment, and T.37/T.38 fax gateways. The applications enable the Cisco gateways to send and receive faxes across packet-based networks, using modems or voice feature cards (VFCs).

Before beginning these troubleshooting procedures, check to ensure your fax machine and voice network are working. Try the following steps to help isolate the problem:

1. Plug the fax machine into a regular analog line and test the fax call while bypassing the voice network. If the problem persists, it could be the fax machine.

2. If the fax machine has a handset, connect the fax machine to the voice network, pick the handset up, and try to make a voice call. If the voice call works, you have verified that your voice network is operational. A voice call must be successful before a fax call can succeed.

This chapter contains the following topics:

Fax Call Flow

Fax Relay

Fax Detection

For information about configuring fax features, refer to the Cisco Fax Services over IP Application Guide.

Fax Call Flow

The following fax call flows are described:

Fax Pass-Through and Fax Pass-Through with Upspeed

Cisco Fax Relay

T.38 Fax Relay

T.37 Store-and-Forward Fax

Fax Pass-Through and Fax Pass-Through with Upspeed

Fax pass-through is the simplest technique for sending fax over IP networks, but it is not the default, nor is it the most desirable method of supporting fax over IP. T.38 fax relay provides a more reliable and error-free method of sending faxes over an IP network, but some third-party H.323 and SIP implementations do not support T.38 fax relay. These same implementations often support fax pass-through.

In fax pass-through mode, gateways do not distinguish between a fax call and a voice call. Fax communication between the two fax machines is carried in its entirety in-band over a voice call. Fax upspeed is similar to pass-through in the sense that the fax call is carried in-band over the voice call. The difference is that when you use upspeed, the gateways to some extent react to the fax call. Although relay mechanisms are not employed, with upspeed the gateways do recognize a CED fax tone and automatically change the voice codec to G.711 if necessary (thus the designation upspeed). Turn off echo cancellation (EC) and voice activity detection (VAD) for the duration of the call.

When a DSP is put into voice mode at the beginning of a VoIP call, the DSP is informed by the call control stack whether the control protocol can support pass-through or not. If pass-through is supported, the following events occur during a fax call:

1. For the duration of the call, the DSP listens for the 2100-Hz CED tone to detect a fax or modem on the line.

2. If the CED tone is heard, an internal event is generated to alert the call control stack that a fax or modem changeover is required.

3. The call control stack on the originating gateway instructs the DSP to send an NSE to the terminating gateway, informing the terminating gateway of the request to carry out a codec change.

4. If the terminating gateway supports NSEs, it responds to the originating gateway instruction and loads the new codec. The fax machines are able to communicate on an end-to-end basis with no further intervention by the voice gateways.

Fax pass-through call flow is shown in Figure 47.

Figure 47 Fax Pass-Through and Fax Upspeed Call Flow

Cisco Fax Relay

Cisco fax relay is the oldest method of supporting fax on Cisco IOS gateways and has been supported since Cisco IOS Release 11.3. Cisco fax relay uses Real-Time Transport Protocol (RTP) as the method of transport. In Cisco fax relay mode, gateways terminate T.30 fax signaling by spoofing a virtual fax machine to the locally attached fax machine. The gateways use a Cisco-proprietary fax-relay RTP-based protocol to communicate between them.

Cisco fax relay is the default operation and, in the absence of any explicit CLI on the dial peer, is used when a fax transmission is detected. If voice calls are being completed successfully between two routers, fax calls should also work. Events that occur during a Cisco fax relay call fall into the following call phases:

Cisco Fax Relay Fax Setup Phase

Cisco Fax Relay Data Transfer Phase

Cisco Fax Relay Fax Setup Phase

When a DSP is put into voice mode at the beginning of a VoIP call, the DSP is informed by the call control stack whether fax relay is supported and if it is supported, whether it is Cisco fax relay or T.38 fax relay. If Cisco fax relay is supported, the following events occur:

Initially a VoIP call is established as if it were a normal speech call. Call control procedures are followed and the DSP is put into voice mode, after which human speech is expected to be received and processed.


Note If a fax answer or calling tone (CED or CNG) is heard at any time during the call, the DSP does not interfere with the speech processing. It allows the tone to continue across the VoIP call leg.


A normal fax machine, after generating a CED or hearing a CNG, sends a DIS message with the capabilities of the fax machine.

The DSP in the Cisco IOS gateway attached to the fax machine that generated the DIS message (normally the terminating gateway) detects the HDLC flag sequence at the start of the DIS message and initiates fax relay switchover.

The DSP also triggers an internal event to notify the call control stack that fax switchover is required. The call control stack then instructs the DSP to change the RTP payload type to 96 and to send this payload type to the originating gateway.

When the DSP on the originating gateway receives an RTP packet with payload type set to 96, it triggers an event informing its own call control stack that a fax changeover has been requested by the remote gateway.

The originating gateway then sends an RTP packet to the terminating gateway with payload type 97 to indicate that the originating gateway has started the fax changeover. When the terminating gateway receives the payload type 97 packet, the packet serves as an acknowledgement. The terminating gateway starts the fax codec download and is ready for fax relay.

Once the originating gateway has completed the codec download, it sends RTP packets with payload type 96 to the terminating gateway. The terminating gateway responds with an RTP packet with payload type 97, and fax relay can begin between the two gateways. As part of the fax codec download, other parameters such as VAD, jitter buffers, and echo cancellation are changed to suit the different characteristics of a fax call.

Cisco fax relay fax setup is shown in Figure 48.

Figure 48 Cisco Fax Relay Fax Setup Call Flow

Cisco Fax Relay Data Transfer Phase

During fax relay operation, the T.30 analog fax signals are received from the PSTN or from a directly attached fax machine. The T.30 fax signals are demodulated by a DSP on the gateway and then packetized and sent across the VoIP network as data. The terminating gateway decodes the data stream and remodulates the T.30 analog fax signals to be sent to the PSTN or to a destination fax machine.

The messages that are demodulated and remodulated are predominantly the phase B, phase D, and phase E messages of a T.30 transaction. Most of the messages are passed across without any interference, but certain messages are modified according to the constraints of the VoIP network.

During phase B, fax machines interrogate each other's capabilities. They expect to communicate with each other across a 64-kbps PSTN circuit, and they attempt to make best use of the available bandwidth and circuit quality of a 64-kbps voice path. However, in a VoIP network, the fax machines do not have a 64-kbps PSTN circuit available. The bandwidth per call is probably less than 64 kbps, and the circuit is not considered a clear circuit.

Because transmission paths in VoIP networks are more limited than in the PSTN, Cisco IOS CLI is used to adjust fax settings on the VoIP dial peer. The adjusted fax settings restrict the facilities that are available to fax machines across the VoIP call leg and are also used to modify values in DIS and NSF messages that are received from fax machines.

The call flow of the Cisco fax relay data transfer phase is shown in Figure 49.

Figure 49 Cisco Fax Relay Data Transfer Call Flow

T.38 Fax Relay

T.38 provides an ITU-T standards-based method and protocols for fax relay. Data is packetized and encapsulated according to the T.38 standard. The encoding of the packet headers and the mechanism to switch from VoIP mode to fax relay mode are clearly defined in the specification. Annexes to the basic specification include details for operation under Session Initiation Protocol (SIP) and H.323 call control protocols.

T.38 uses data redundancy to compensate for packet loss. During T.38 call establishment, voice gateways indicate the level of packet redundancy that they incorporate in their transmission of facsimile User Datagram Packet Transport Layer packets (UDPTLs). The level of redundancy (the number of times that the packet is repeated) can be configured on Cisco IOS gateways.

There is work under way to implement T.38 fax switchover independently of the call control mechanisms. This is referred to as "bearer level signaling" and makes use of named signaling events (NSEs).

The following sections address call-control-initiated switchover mechanisms:

H.323 T.38 Fax Relay

SIP T.38 Fax Relay

MGCP T.38 Fax Relay

H.323 T.38 Fax Relay

The T.38 Annex B standard defines the mechanism that is used to switch over from voice mode to T.38 fax mode during a call. The ability to support T.38 must be indicated during the initial VoIP call setup. If the DSP on the gateway is capable of supporting T.38 mode, this information is indicated during the H.245 negotiation procedures as part of the regular H.323 VoIP call setup.

Once the VoIP call setup is completed, the DSP continues to listen for a fax tone. When it hears one, the DSP signals the receipt of fax tone to the call control layer, which then initiates fax changeover as specified in the T.38 Annex B procedures. The H.245 message flow shown in Figure 50 contains the following events:

1. The detecting terminating gateway sends a ModeRequest message to the originating gateway, and the originating gateway responds with a ModeRequestAck.

2. The originating gateway sends a closeLogicalChannel message to close its VoIP UDP port, and the terminating gateway responds with a closeLogicalChannelAck while it closes the VoIP port.

3. The originating gateway sends an openLogicalChannel message that indicates to which port to send the T.38 UDP information on the originating gateway, and the terminating gateway responds with an openLogicalChannelAck.

4. The terminating gateway sends a closeLogicalChannel message to close its VoIP UDP port, and the originating gateway responds with a closeLogicalChannelAck.

5. Finally the terminating gateway sends an openLogicalChannel message that indicates to which port to send the T.38 UDP stream, and the originating gateway responds with an openLogicalChannelAck.

6. T.38-encoded UDP packets flow back and forth. At the end of the fax transmission, either gateway can initiate another ModeRequest message to return to VoIP mode.

Figure 50 H.323 T.38 Fax Relay Call Flow

SIP T.38 Fax Relay

When the call control protocol is SIP, T.38 Annex D procedures are used for the changeover from VoIP to fax mode during a call. Initially, a normal VoIP call is established through the use of SIP INVITEs. The DSP needs to be informed that it can support T.38 mode while it is put into voice mode. During the call, when the DSP detects fax HDLC flags, it signals the detection of the flags to the call control layer, and the call control layer initiates a SIP INVITE mid-call to signal the desire to change the media stream.

The SIP T.38 fax relay call flow shown in Figure 51 contains the following events:

1. The terminating gateway detects a fax V.21 flag sequence and sends an INVITE with T.38 details in the SDP field to the originating gateway or to the SIP proxy server, depending on the network topology.

2. The originating gateway receives the INVITE message and sends back a 200 OK message.

3. The terminating gateway acknowledges the 200 OK message and sends an ACK message direct to the originating gateway.

4. The originating gateway starts sending T.38 UDP packets instead of VoIP UDP packets across the same ports.

5. At the end of the fax transmission, another INVITE message can be sent to return to VoIP mode.

Figure 51 SIP T.38 Fax Relay Call Flow

MGCP T.38 Fax Relay

MGCP T.38 fax relay conforms to ITU-T T.38, Procedures for Real-Time Group 3 Facsimile Communication over IP Networks, which determines procedures for real-time facsimile communication in various gateway control protocol (XGCP) applications.

MGCP-based T.38 fax relay has the following call flow:

1. A call is initially established as a voice call.

2. The gateways advertise capabilities in an SDP exchange during connection establishment.

3. If both gateways do not support T.38 fax relay, fax pass-through is used for fax transmission. If both gateways support T.38, they attempt to switch to T.38 upon fax tone detection. The existing audio channel is used for T.38 fax relay, and the existing connection port is reused to minimize delay. If failure occurs at some point during the switch to T.38, the call reverts to the original settings it had as a voice call. If this failure occurs, a fallback to fax pass-through is not supported.

4. Upon completion of the fax image transfer, the connection remains established and reverts to a voice call using the previously designated codec, unless the call agent instructs the gateways to do otherwise.

A fax relay MGCP event allows the gateway to notify the call agent of the status (start, stop, or failure) of T.38 processing for the connection. This event is sent in both call-agent-controlled and gateway-controlled mode.

T.37 Store-and-Forward Fax

T.37 provides an ITU-T standards-based method for store-and-forward fax operation. The fax transmission process is divided into distinct sending and receiving phases with the potential to store the fax between sending and receiving, if necessary.

Cisco recommends that you dedicate a mail server to fax mail and avoid the conflicting configuration requirements of traditional e-mail and fax-mail servers. Optimize each mail server for its individual functions—for example, fax messages should usually retry transmissions every 5 minutes whereas normal e-mail should retry every 30 minutes, and fax messages should give up after 3 to 4 hours whereas normal e-mail should not give up for 4 to 5 days.

A topology for T.37 store-and-forward fax is shown in Figure 52.

Figure 52 T.37 Store-and-Forward Fax Topology

For more information about T.37 store and forward fax, refer to the "Configuring T.37 Store-and-Forward Fax" chapter in the Cisco Fax Services over IP Application Guide.

Fax Relay

To troubleshoot T.38 fax relay, perform the following steps:

Make sure that you can make a voice call.

Make sure that the desired fax protocol was set using the fax protocol command on both the originating and terminating gateways.

Make sure that the fax protocol is configured as T.38 at the global configuration level or at the dial-peer configuration level for both the originating and terminating gateways.

Use the show call active {voice | fax} command to display information for the active call table.

Use the show dial-peer voice command to display configuration information for dial peers.

For H.323 gateways, use the debug cch323 all command to enable all H.323 debugging capabilities, or use one of the following commands to debug problems while making the call:

debug cch323 error

debug cch323 h225

debug cch323 h245

debug cch323 RAS

debug cch323 session

debug voip ccapi inout

debug vtsp session

For SIP gateways, use the debug ccsip all command to enable all SIP debugging capabilities, or use one of the following SIP debug commands:

debug ccsip calls

debug ccsip error

debug ccsip events

debug ccsip info

debug ccsip media

debug ccsip messages

debug ccsip states

For further details, see the following sections:

Identifying and Isolating the Problem

Checking Basic Connectivity

Checking for Slips and Other Errors on Digital Interfaces

Checking Fax Interface Type

Ensuring That the Fax Codec is Loaded During the Fax Call

Disabling Fax Relay and Change Codec for Pass-Through

Checking for Packet Loss on the Voice Network

Disabling Fax Relay ECM (Cisco Proprietary VoIP Only)

Enabling T.38 Packet Redundancy (T.38 VoIP Only)

Setting the Fax NSF Command to All Zeroes

Achieving the Final Stages of Resolution

Debugging Fax Relay

Identifying and Isolating the Problem

The first step to take when you troubleshoot any fax relay issue is to reduce the problem to its simplest form. Many issues arise in situations where multiple fax machines are not able to pass fax traffic. It is easiest to isolate two fax machines that are having problems and concentrate on a simple topology. Determine how these machines are connected to one another and resolve the issue between this pair first. In addition, you should draw a complete picture of the topology and determine how the fax machines are interconnected.

Troubleshooting one issue at a time minimizes confusion and allows for methodical troubleshooting. It is also possible that the solution for this problem resolves other fax relay problems in the network. Most fax relay problems result from poor voice configuration or network design. These lead to basic connectivity problems and physical line or packet loss and jitter problems.

After you have identified and isolated the problem, you need to verify the basic voice configuration and monitor the health of the network.

Checking Basic Connectivity

Basic fax connectivity issues can be the result of the following factors:

Normal Voice Connectivity Problems

Configuration Problems Related to Dial Peers

Other Basic Connectivity Problems Not Relating to Dial Peers

Fax Connectivity Problems Across the PSTN

Normal Voice Connectivity Problems

Confirm that normal voice calls can be completed before investigating fax connectivity. If there is no telephone attached, unplug the fax machine and connect a regular telephone. If normal voice calls do not connect, the issue might be network-related and you should troubleshoot the problem as a normal voice connectivity issue before proceeding with fax troubleshooting.

Configuration Problems Related to Dial Peers

Some configuration problems are associated with dial peers, for example:

Wrong Dial Peer Being Matched

Incorrectly Configured Dial Peers on One or Both Sides

Wrong Dial Peer Being Matched

After ensuring that voice calls can be successfully completed in both directions through the voice network, issue the show call active voice brief command and note the dial peers that are being matched with each voice call.


Note When you have VoIP trunks, you should be able to see all the call legs with the show call active voice brief command. In some versions of Cisco IOS Software Release 12.2, there is a bug in the show call active command. As a result, a fax call that comes through a VoIP trunk no longer appears. When you issue a show call active fax brief command, the call is listed. For more information about this bug, see Cisco bug IDs CSCdx50212 and CSCdv02561 (registered customers only).



Note Ensure that the configured dial peer is the peer that is being matched.


In the command output shown below, you can see that the outbound VoIP call leg is using peer ID 100.

Router# show call active voice brief

<snip> 

Total call-legs: 2
1218 : 51710253hs.1 +415 pid:400 Answer 400 active
dur 00:01:08 tx:3411/68220 rx:3410/68200
Tele 3/0/0:43: TX:68200/6820/0ms g729r8 noise:0 acom:2 
 i/0:-51/-44 dBm

1218 : 51710396hs.1 +272 pid:100 Originate 100 active
dur 00:01:09 TX:3466/69320 rx:3467/69340
IP 2.1.1.2:17092 rtt:56ms pl:64730/0ms lost:0/1/0 
 delay:69/69/70ms 
 g729r8

Total call-legs: 2

A common cause of fax relay problems is that the correctly configured dial peer is not the one being matched. It is also common that there is no particular incoming VoIP dial peer configured on the terminating gateway, and Cisco IOS Software selects the first appropriate (including default) VoIP dial peer as the incoming dial peer. Hence the parameters for this incoming dial peer might not match those of the outbound dial peer on the originating gateway.

It is not always necessary to have identical configurations on the outgoing and incoming VoIP dial peers. When you have a fax relay problem, though, make sure you have a dedicated incoming VoIP dial peer on the terminating router and that its configuration matches the configuration of the outgoing VoIP dial peer on the originating router. The following configuration for ISDN-connected routers is an example of specific, matching VoIP dial peers for the destination pattern "5..." outbound on the originating gateway and inbound on the terminating gateway.

Originating Gateway
Terminating Gateway

!--- Incoming POTS peer:

Dial-peer voice 1 pots 
Incoming called number. 
Direct-inward-dial 
Port 1/0:15

!--- Outgoing VoIP peer:

Dial-peer voice 2 voip
Destination-pattern 5...
Session target ipv4:1.1.1.1
Fax rate 144400
fax protocol t38 
 ls-redundancy 0
 hs-redundancy 0

!--- Outgoing POTS peer
:
Dial-peer voice 10 pots
Destination-pattern 5...
No digit-strip
Port 2/0:15

!--- Incoming VoIP peer:

Dial-peer voice 20 voip
Incoming called-number 5...
Fax rate 14400
fax protocol t38 
 Ls-redundancy 0
 Hs-redundancy 0

You can also check dial-peer matching by issuing the debug voip ccapi inout command. The debug output from this command (as shown below) includes an ssaSetupPeer message that lists all the dial-peers matching the called number. A ccCallSetupRequest message follows with the outbound peer option indicating the outgoing VoIP dial peer selected. When multiple VoIP dial peers are configured for the same destination, it is possible that the initial call setup could fail and another dial peer could be tried. In this case, another ccCallSetupRequest appears in the debug.

debug voip ccapi inout—Originating Gateway

ms-3640-13b# show call active voice brief

<snip> 

Total call-legs: 2
1218 : 51710253hs.1 +415 pid:400 Answer 400 active
dur 00:01:08 tx:3411/68220 rx:3410/68200
Tele 3/0/0:43: TX:68200/6820/0ms g729r8 noise:0 acom:2 
 i/0:-51/-44 dBm

1218 : 51710396hs.1 +272 pid:100 Originate 100 active
dur 00:01:09 TX:3466/69320 rx:3467/69340
IP 2.1.1.2:17092 rtt:56ms pl:64730/0ms lost:0/1/0 
 delay:69/69/70ms 
 g729r8

Total call-legs: 2

debug voip ccapi inout —Originating Gateway

.Jun 4 21:06:43.461: ssaSetupPeer cid(19) 
 peer list: tag(400) called number (5074) 
 
.Jun 4 21:06:43.461: ccCallSetupRequest 
 (Inbound call = 0x13, outbound peer =100, 
  dest=, params=0x62F1CC70 mode=0, *callID=0x62F1CFD8, 
  prog_ind = 0)

On the terminating voice gateway, the first line of the debug voip ccapi inout call trace (as shown below) is a cc_api_call_setup_ind message with a peer_tag option that refers to the incoming VoIP dial peer on the terminating gateway.

debug voip ccapi inout—Terminating Gateway

.Jun 4 21:06:43.461: cc_API_call_setup_ind 
 (vdbPtr=0x62F07650,
 callInfo={called=5074,called_oct3=0x80,
 calling=5075, calling_oct3=0x0,>calling_oct3a=0x83,
 calling_xlated=false, 
 subscriber_type_str=Unknown,fdest=1,
 peer_tag=400, prog_ind=0},callID=0x635F72D0)

Incorrectly Configured Dial Peers on One or Both Sides

After you confirm that the correct dial peer is being matched (in this case dial-peer 100 for the originating gateway and dial peer 400 for the terminating router), confirm in the configuration that the dial peer is configured correctly for fax. Here are some common errors to check for on both sides of the call:

Fax relay is disabled (that is, the fax rate disable command has been issued on the dial peer) while a low bandwidth codec has been in use.

The dial peer on one voice gateway is configured for Cisco fax relay but the other voice gateway is a Cisco AS5350/AS5400. Cisco AS5350/AS5400 universal gateways support only T.38 so the negotiation fails.

The default dial peer is being used inbound on the terminating gateway, and default parameters do not match those for the outbound dial peer on the originating gateway.

Incorrect compand type.

The companding type for the US is µ-law, for Europe and Asia it is a-law. You can issue the show voice call command to see which value is currently configured. If you are on a BRI or E1 port, the companding type on the router does not match the one on the connected device, and calls sometimes fail and sometimes connect, but the voice becomes heavily distorted so that the person becomes unrecognizable and a high low-pitch noise level appears.

In Cisco IOS Release 12.2(3) the compand-type command is missing on the BRI ports, leaving the companding type to the default value. For more information about this bug, see Cisco bug IDs CSCdv00152 and CSCdv01861 (registered customers only).

Other Basic Connectivity Problems Not Relating to Dial Peers

Here are some basic connectivity issues that are related to dial peers:

Cisco IOS software incompatibilities on gateway pairs. It is not always required that Cisco IOS software releases match, but you should check releases when problems occur.

Compressed Real-Time Transport Protocol (cRTP) has several known problems. Fixes are available for these problems and it makes sense to disable cRTP when problems occur. You can then decide whether a Cisco IOS software upgrade is an appropriate course of action.

On Cisco AS5300 voice gateways, ensure that the VCWare and Cisco IOS software are compatible.

Fax Connectivity Problems Across the PSTN

If voice calls work in both directions but fax calls are failing in at least one direction, check that normal faxing between these two machines works across the PSTN. In other words, ensure that the fax machines successfully transmit faxes to each other using the PSTN without traversing the VoX network. If they do not, the fax machines may have problems that need to be addressed before you consider fax relay problems.

Checking for Slips and Other Errors on Digital Interfaces

If there are any T1 or E1 digital connections used by the routers performing fax relay, make sure that they are error free. Fax relay is very sensitive to errors on digital interfaces, especially slips. The errors may not be noticeable on voice calls but they can cause faxes to fail.

show controller T1(E1) 1/0 Command

Router# show contr t1 1/0
T1 1/0 is up.
Applique type is Channelized T1
Cablelength is long gain36 0db
No alarms detected.
alarm-trigger is not set
Version info Firmware: 20010805, FPGA: 15
Framing is ESF, Line Code is B8ZS, Clock Source is Line.
Data in current interval (132 seconds elapsed):
0 Line Code Violations, 0 Path Code Violations
0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 
0 Degraded Mins
0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 
 0 Unavail Secs

The T1 or E1 controllers at both originating and terminating gateways should be error free, as shown above. If errors occur, repeat the show controller command several times during the call to see if the number of errors increases. The most common problem of slips is a synchronization problem resulting in clocking errors.

In packet voice networks, confirming that the router is clocking from the line is usually sufficient. If it is not, ensure the clock source line command is entered at the controller level. However, in VoATM or TDM networks where a clocking hierarchy is established and the routers may need to pass clock through the network, additional considerations are needed.

On Cisco modular acccess routers with AIM voice cards installed, the controller shows "controlled slips" unless you add the network-clock-participate and network-clock-select commands.

On the Cisco 7200VXR platform, the frame-clock-select command is required for the voice cards. This command is particularly important for the Cisco 7200VXR voice gateways because, by default, the internal TDM bus is not driven by the local oscillator. Since the E1 trunks are normally synchronized to the telephony network, the result is hidden clocking errors and intermittent fax transmission problems. More detail is available in Cisco bug ID CSCdv10359 (registered customers only).

Checking Fax Interface Type

On some platforms, including the Cisco 3660, Cisco AS5300, Cisco AS5350, Cisco AS5400, and Cisco AS5800 platforms, the router defaults to fax interface-type modem. The fax interface-type modem global configuration command forces fax calls to a modem (usually for T.37 Store and Forward fax) and not to a DSP. For Cisco fax relay to work, the fax call must be sent to a DSP, which means it must be configured by the issuing of the fax interface-type vfc command.

Router(config)# fax interface-type ?
modem Use modem card
vfc   Use Voice Feature Card

Router(config)# fax interface-type vfc
You must reload the router

Make sure you reload the router; otherwise the command does not take effect. Fax calls fail on the universal gateways listed above with Cisco fax relay (or T.38), so this is an important command to check.

The fax interface-type vfc command was not necessary in Cisco IOS software releases prior to Release 12.2. The problem is commonly seen when one of the voice gateways listed above is upgraded to Cisco IOS Release 12.2 or later.

Ensuring That the Fax Codec is Loaded During the Fax Call

Each fax machine displays the remote fax machine ID on its LCD screen at the completion of the fax negotiation phase. It is unlikely that the fax machines could complete negotiation if the fax codec has not been successfully downloaded. On the other hand if no remote fax machine ID is displayed, further debugging in this area is appropriate.

There are two ways to make sure that the voice gateways detect the fax transmission and successfully load the fax codec:

Issue the debug vtsp all command and the debug voip ccapi inout call trace.

Issue the show voice trace command. Show commands are less resource intensive on the router than debug commands and are preferable in production networks. Shown below is sample output from a show voice trace command on an ISDN interface:

BrisVG200gwy01# show voice trace 1/0:15
1/0:15 1  
1/0:15 2  
1/0:15 3
1/0:15 4
1/0:15 5
1/0:15 6
1/0:15 7
1/0:15 8
1/0:15 9
1/0:15 10 State Transitions: timestamp (state, event) -> ...
63513.792 (S_SETUP_REQUEST, E_TSP_PROCEEDING) ->
63515.264 (S_SETUP_REQ_PROC, E_TSP_ALERT) ->
63515.264 (S_SETUP_REQ_PROC, E_CC_BRIDGE) ->
63515.332 (S_SETUP_REQ_PROC, E_CC_CAPS_IND) ->
63515.332 (S_SETUP_REQ_PROC, E_CC_CAPS_ACK) ->
63515.348 (S_SETUP_REQ_PROC, E_CC_CAPS_IND) ->
63515.348 (S_SETUP_REQ_PROC, E_CC_CAPS_ACK) ->
63515.356 (S_SETUP_REQ_PROC, E_CC_CAPS_IND) ->
63515.356 (S_SETUP_REQ_PROC, E_CC_CAPS_ACK) ->
63518.656 (S_SETUP_REQ_PROC, E_CC_REQ_PACK_STAT) ->
63518.660 (S_SETUP_REQ_PROC, E_DSP_GET_VP_DELAY) ->
63518.660 (S_SETUP_REQ_PROC, E_DSP_GET_VP_ERROR) ->
63518.660 (S_SETUP_REQ_PROC, E_DSP_GET_RX) ->
63518.660 (S_SETUP_REQ_PROC, E_DSP_GET_TX) ->
63521.028 (S_SETUP_REQ_PROC, E_CC_REQ_PACK_STAT) ->
63521.028 (S_SETUP_REQ_PROC, E_DSP_GET_VP_DELAY) ->
63521.028 (S_SETUP_REQ_PROC, E_DSP_GET_VP_ERROR) ->
63521.028 (S_SETUP_REQ_PROC, E_DSP_GET_RX) ->
63521.028 (S_SETUP_REQ_PROC, E_DSP_GET_TX) ->
63524.128 (S_SETUP_REQ_PROC, E_TSP_CONNECT) ->

!--- Fax tone detected:

63529.352 (S_CONNECT, E_DSP_TONE_DETECT) ->
63529.356 (S_LFAX_WAIT_ACK, E_PH_CODEC_ACK) ->

!--- Fax codec being downloaded to DSPs:

Disabling Fax Relay and Change Codec for Pass-Through

In the previous troubleshooting steps you established that voice calls work, faxes work through PSTN, and all digital interfaces in the fax relay path are free from errors. The current step determines whether faxes can go through with fax relay disabled. For the VoIP/VoATM/VoFR dial peers, enter the following to disable the fax rate:

Router(config)# voice-port 2/0:15
Router(config-voiceport)# no echo-cancel enable
Router(config)# dial-p voice 3
Router(config-dial-peer)# fax rate disable
Router(config-dial-peer)# codec g711ulaw
Router(config-dial-peer)# no vad

Make sure these commands are entered on both gateways. These commands disable fax relay, disable echo cancellation, and force the call to use a high-bandwidth codec without VAD. The router then samples the tones as it would for a normal voice call, and with the high-bandwidth codec (G.711), the most precise sample possible is captured. The tone to be replayed on the other side is as accurate as possible. The caveat to this step is that since G.711 is a 64- kbps bandwidth codec, each call consumes up to 80 kbps (for VoIP) when additional transport protocol overhead is added.

If this test is positive, two things have been accomplished. First, if per-call bandwidth consumption is not a major issue for the network, there is now a potential fax pass-through workaround for the fax relay problem. Second, and more significantly, if bandwidth consumption is an issue, the problem has been isolated to the fax relay software and you should open a TAC case.

If this test fails, whatever is causing the fax calls to fail with fax relay is also probably causing the failures with this test. The problem might be that the network might have a large amount of jitter or packet loss.

Checking for Packet Loss on the Voice Network

Her is the easiest and most accurate way to determine if there is a packet loss:

1. Disable VAD on the VoIP dial peers.

2. Make a voice call between the same ports where the fax machines are connected. (Fax machines may be able to serve as ordinary phones or you might need to connect the handsets to the same ports where the fax machines are connected.)

3. When the call is connected, there are two main steps to take:

a. Issue the show voice dsp command. You can see in the output that one of the DSP channels has the configured codec loaded. Usually the column "TX/RX-PAK CNT" shows that the transmit and receive packet counters are equal, meaning that no packets are being lost. If the counters are not equal, packets might be getting lost. Type the show voice dsp command several times at 30-second intervals to determine if the difference increases and packets are being lost.

b. Issue the show voice call summary command to see which port (and timeslot if applicable) is allocated to the voice call. Type terminal monitor and issue the show voice call command with the voice port (and timeslot if applicable) to get the detailed DSP statistics. In the "***DSP VOICE VP_ERROR STATISTICS***" section of the output, look for the counters. They are usually 0 or below 20. If the any of the counters have higher than 20, investigate the packet loss.

If the network appears to be lossy, it is not reasonable to expect fax relay to work reliably. It is possible to disable ECM, but further investigation is probably needed to ensure QoS is provisioned end-to-end so that the voice and fax relay traffic has priority and is never lost during the congestion.

Disabling Fax Relay ECM (Cisco Proprietary VoIP Only)

For networks with packet loss and lots of jitter, disabling ECM can improve fax relay calls. Issue the command fax-relay ecm disable to turn off ECM, so that a larger amount of jitter and packet loss can be tolerated.

Issuing the fax-relay ecm disable command improves fax relay performance in lossy networks, but this command is also recommended for basic troubleshooting. Even if there is not a noticeable jitter problem in the network, this command can sometimes help determine fax relay problems. This command is available under VoFR and VoATM dial peers but currently works only for VoIP.


Note This command also activates the packet loss concealment feature.


Router(config-dial-peer)# dial-peer voice 3
Router(config-dial-peer)# fax-relay ECM disable

Enabling T.38 Packet Redundancy (T.38 VoIP Only)

If T.38 for VoIP is being used as the fax relay protocol, you can turn on the T.38 packet redundancy feature by configuring the following command under the appropriate dial peers on both gateways:

Router(config-dial-peer)# fax protocol t38 
 Ls-redundancy X Hs-redundancy Y

where X > 0 and Y = 0 (only make changes to Ls-redundancy)

If Cisco proprietary fax relay is in use, an alternative or additional option to disabling ECM is to change fax relay protocol to T.38 so that the T.38 packet redundancy feature can be tested. This feature might alleviate failure due to packet loss. However, T.38 packet redundancy significantly increases bandwidth usage, and it is preferable to eliminate the packet loss altogether, rather than alleviating the failures it causes.

Setting the Fax NSF Command to All Zeroes

The fax nsf command can be helpful for any fax machine that alters the NSF field during fax negotiation for proprietary encodings. This command allows the router doing fax relay to override the settings made by the fax machines trying to implement proprietary encodings. Before the fax nsf command was available, fax relay failed for such fax machines. Typically the fax nsf command is used to set the NSF field to all zeroes, forcing a standard fax negotiation from both sides. Using this command has been successful with certain brands of fax machines such as Harris and Lanier, and it is recommended when fax relay is failing.

Router(config-dial-peer)# fax NSF 000000

Achieving the Final Stages of Resolution

If the preceding troubleshooting steps do not resolve the fax relay issue, the problem might require more advanced troubleshooting. Listed below are additional steps to try before you open a case with the Cisco Technical Assistance Center (TAC). See the "Obtaining Technical Assistance" section on page xxvi for information about contacting TAC.

Learn the brands and models of the fax machines that are failing, and investigate those brands and models for known issues.

Sometimes there are CARE cases or bugs that address problems for a certain brand of fax machine. For example, a search in the Bug Toolkit (registered customers only) for a Pitney Bowes fax shows a bug with Pitney Bowes fax machines and Cisco fax relay (CSCdu78373 (registered customers only)). This bug is not in the Cisco IOS software but is an incompatibility with the Pitney Bowes proprietary fax signaling protocol. A problem arises when the fax devices on each side of a connection are Pitney Bowes 9920s or 9930s. The workaround is to disable the proprietary protocol on the fax machines or to disable fax relay and use a higher bandwidth codec.

Use search tools to look for known fax problems in the Cisco IOS software release where the problem is occurring.

In the previous step, searches were made for a specific fax brand in the hope of identifying a known issue between a certain fax brand and the Cisco fax relay code. The next step is to perform a generic search, because there could be a fax relay bug in the Cisco IOS software release installed.

For example, if fax relay using VoFR is not working in Cisco IOS Software Release 12.1(2)T, you can search for bugs using the Bug Toolkit on Cisco.com. In this example, you would use the following values:

Major version: 12.1

Revision: 2

Feature/component: VoFR

Keyword: fax

One of the bugs is Cisco bug ID CSCdr65984 (registered customers only), entitled "fax doesn't work for vofr." This bug causes all fax relays to fail for VoFR, and an upgrade is needed to a Cisco IOS software release in which this bug is no longer present.

Eliminate hardware faults.

In some cases it is easier to isolate the problem by excluding potential problem sources one by one. You can do this by replacing different hardware parts and using alternative IP connections between the gateways.

When extra hardware is available, the following steps can help.

Use different ports on the routers—If your configuration involves two gateways connected to the PBXs or PSTN with E1 or T1 and if you have the FXS ports available, try to connect the fax machines directly to the FXS ports on the voice gateways. This procedure helps you further isolate the problem by excluding the possibility of the E1 cards failing, problems on the telephony side, and E1 synchronization or cable problems.

Try different hardware—If you have another voice gateway with FXS ports available, try to connect it directly with the Ethernet crossover cable to each of the voice gateways and send a fax using the fax machine connected to the FXS port. This procedure helps determine if there are problems in the VoX network involving queuing, fragmentation, or prioritization.

Use debug commands on the router to determine what is happening—See the following section for details about debug commands that are useful for troubleshooting fax relay problems.

Debugging Fax Relay

The following topics are associated with fax relay debugging:

T.30 Messages

Fax Relay Debug Commands

Fax Analyzers

T.30 Messages

The debugs can be difficult to understand if you are not familiar with the messaging that occurs during a typical fax transmission. Figure 53 is a graphical representation of the basic T.30 transactions that occur for a single page fax transmission.

Figure 53 T.30 Transactions

Describing the details of these transactions is beyond the scope of this document, but listed below are definitions of the basic transactions that are seen during fax relay. The list is alphabetical for quick reference and includes messages that you are likely to see when you are debugging Cisco fax relay. For more in-depth information on this messaging or for information on messages that are not listed below, see the T.30 specification.

CED (called terminal identification)—A 2100 Hz signal that is transmitted by the terminating fax device upon answering a fax call. This signal temporarily disables echo cancellers that are present on the connection to prepare the line for data transmission.

CFR (confirmation to receive)—A response confirming that the previous messaging and training has been completed and that fax page transmission can begin.

CNG (calling tone)—An 1100-Hz tone that is on for 0.5 seconds and then off for 3 seconds. This signal identifies the fax terminal as a nonspeech device. The signal also indicates that the initiating fax terminal is awaiting the DIS signal from the terminating fax terminal.

CRP (command repeat)—A response that indicates that the previous command was received in error and needs to be repeated. (Optional)

CSI (called subscriber identification) —Can be used to provide the specific identity of the called fax terminal through its international telephone number. (Optional)

DCN (disconnect)—Ends the fax call and requires no response.

DIS (digital identification signal) — Identifies the capabilities of the called fax terminal.

DTC (digital transmit command) —The response to the capabilities identified by the DIS signal. Here is where the calling fax terminal matches its capabilities with those provided in the called fax terminal's DIS message.

EOM (end of message) — Indicates the end of a complete page of fax information.

EOP (end of procedure)— Indicates the end of a complete page of fax information and signals that no further pages are to be sent. The other device can proceed to the disconnect phase of the fax call.

FTT (failure to train) — Used to reject a training signal and request a retrain (retrains usually occur at lower modulation speeds).

MCF (message confirmation) —Indicates that a message has been satisfactorily received.

MPS (multipage signal)—Indicates the end of a complete page of fax information and signals that the receiver is ready for additional pages.

NSF (nonstandard facilities)— Can be used to identify specific capabilities or requirements that are not covered by the T-series specifications. (Optional)

RTN (retrain negative)— Indicates that a previous message has not been satisfactorily received. Retraining is needed to proceed (usually at a lower modulation speed).

RTP (retrain positive)— Indicates that a complete message has been received and that additional messages may follow after retraining.

TCF (training check)— Sent through the higher-speed T.4 modulation system (versus the 300-kbps V.21 modulation used for the previous T.30 signaling) to verify training and indicate the acceptability of sending fax pages at this transmission rate.

TSI (transmitting subscriber identification)— Identifies the transmitting (calling) fax terminal. (Optional)

Fax Relay Debug Commands

Useful fax relay debug commands include:

debug fax relay t30 all

debug vtsp all

debug vtsp vofr subframe 3

Additional Debug Commands

debug fax relay t30 all

The debug for Cisco fax relay is enabled with the debug fax relay t30 all command.

Router# debug fax relay t30 all
Debugging fax relay t30

Shown below is a copy of a debug from a failed fax relay session. This is a debug from the originating fax gateway running Cisco IOS Release 12.2(7a).


Router#
Dec 5 07:49:13.073: 1/2:62 1281347052 fr-entered (10ms)
Dec 5 07:49:17.985: 1/2:62 1281351950 fr-msg-det CRP
Dec 5 07:49:20.105: 1/2:62 1281354070 Fr-MSG-TX NSF
Dec 5 07:49:20.655: 1/2:62 1281354620 Fr-MSG-TX good crc,
 19 bytes
Dec 5 07:49:20.720: 1/2:62 1281354680 Fr-MSG-TX DIS
DEC 5 07:49:22.350: 1/2:62 1281356310 fr-msg-det TSI
DEC 5 07:49:23.045: 1/2:62 1281357000 fr-msg-det DCS
DEC 5 07:49:27.346: 1/2:62 1281361290 Fr-MSG-TX FTT
DEC 5 07:49:28.836: 1/2:62 1281362780 fr-msg-det TSI
DEC 5 07:49:29.531: 1/2:62 1281363470 fr-msg-det DCS
DEC 5 07:49:29.740: 1/2:62 1281363680 fr-msg-det bad crc,
 0 bytes
DEC 5 07:49:30.362: 1/2:62 1281364300 fr-msg-det bad crc,
 0 bytes
DEC 5 07:49:30.804: 1/2:62 1281364740 fr-msg-det bad crc,
 0 bytes
DEC 5 07:49:30.852: 1/2:62 1281364790 fr-msg-det bad crc,
 0 bytes
DEC 5 07:49:33.868: 1/2:62 1281367800 Fr-MSG-TX FTT
DEC 5 07:49:35.414: 1/2:62 1281369340 fr-msg-det TSI
DEC 5 07:49:36.113: 1/2:62 1281370040 fr-msg-det DCS
DEC 5 07:49:36.515: 1/2:62 1281370440 fr-msg-det bad crc,
 0 bytes
DEC 5 07:49:36.908: 1/2:62 1281370830 fr-msg-det bad crc,
 0 bytes
DEC 5 07:49:37.559: 1/2:62 1281371480 fr-msg-det bad crc,
0 bytes
DEC 5 07:49:37.784: 1/2:62 1281371700 fr-msg-det bad crc,
 0 bytes
DEC 5 07:49:37.900: 1/2:62 1281371820 fr-msg-det bad crc,
 0 bytes
DEC 5 07:49:40.133: 1/2:62 1281374050 Fr-MSG-TX FTT
DEC 5 07:49:41.888: 1/2:62 1281375800 fr-msg-det TSI
DEC 5 07:49:42.583: 1/2:62 1281376490 fr-msg-det DCS
DEC 5 07:49:43.173: 1/2:62 1281377080 fr-msg-det bad crc,
 0 bytes
DEC 5 07:49:44.937: 1/2:62 1281378840 fr-msg-det bad crc,
 0 bytes
DEC 5 07:49:45.386: 1/2:62 1281379290 fr-msg-det bad crc,
 0 bytes
DEC 5 07:49:46.941: 1/2:62 1281380840 Fr-MSG-TX FTT
DEC 5 07:49:48.503: 1/2:62 1281382400 fr-msg-det DCN
DEC 5 07:49:50.631: 1/2:62 1281384520 fr-end-dcn

This debug shows the T.30 events that are taking place in the DSP during fax relay. Remember that the debugs are taking place from the perspective of the DSP interacting with the fax device. Any Fr-MSG-TX or transmit message is being transmitted from the DSP to the connected fax device. Any message that the DSP says that it detects (an fr-msg-det message, is a message that it received from the connected fax device. Figure 54 illustrates the directional flow of the DSP messages when the debug fax relay t30 all command is issued.

Figure 54 DSP Message Flow

From the failed fax transaction shown in the debug above, you can see several badcyclic redundancy check (CRC) messages followed by a failure to train (FTT) message from the far side. From the debugs it looks like the problem involves the training signal. The bad crc errors and the FTT message returned from the other side indicate that the signal is corrupted or incompatible with the Cisco fax relay protocol. This debug is taken from a fax relay problem that occurs with a Lexmark Optra fax machine. The Lexmark is V.34-capable and attempts to connect at V.34 rates. V.34 is not supported in Cisco fax relay and the training errors shown here occur. See Cisco bug ID CSCdv89496 (registered customers only) for more details.

debug vtsp all

There are also other debug commands that might be useful for troubleshooting fax relay problems. These debugs might not be as easy to read or provide as much information as the T.30 debugs described above, but they can still be useful.

Voice Telephony Service Provider (VTSP) is an architecture for the interface between the Cisco IOS call control and a DSP endpoint connected to standard telephony equipment such as a PBX, fax, or central office via analog or digital interfaces.

For VoIP T.38 or fax relay, debug vtsp all can provide useful state information from the router. This debug command can be used to determine if the fax codec has been downloaded into the DSP.

debug vtsp vofr subframe 3

Another fax relay debug command that is helpful for fax using VoFR and VoATM is debug vtsp vofr subframe 3 . This command outputs FRF.11 frames that have an Annex D fax relay payload type. There is a significant amount of output from this command even with just one fax relay call, and the hexadecimal must be decoded (the FRF.11 specification is helpful for hexadecimal decoding).

Additional Debug Commands

To debug T.38 capabilities exchange issues, use the debug cch323 h245 command.

To debug DSP message exchanges between applications and the DSP, use the following debug commands:

debug voip ccapi inout

debug hpi all (on the Cisco 5300/2600/3600 and all other voice platforms using the TI c54x DSPs)

debug nextport vsmgr detail (on the NextPort DSP platforms (Cisco 5400, 5850))

Fax Analyzers

Sometimes it is necessary to go beyond the debugging capabilities of the Cisco voice gateways to resolve fax relay problems. You can use tools such as protocol analyzers and fax analyzers to see what is occurring during fax relay operation. Fax analyzers such as the Genoa ChannelProbe/FaxProbe by QualityLogic or the HP Telegra can be positioned between the fax device and the Cisco gateway to capture what is occurring. Protocol analyzers such as Sniffer and Domino can be helpful when you need to view the fax relay packets that are being exchanged between the routers.

The ability to resolve a complex problem sometimes requires using a combination of equipment—an analyzer to capture the fax traffic at each fax machine and a protocol analyzer to capture the fax relay packets. A single fax call is placed to reproduce the problem, and then the information is captured from the attached devices for analysis. Figure 55 shows where this test equipment is placed in the network.

Figure 55 Test Equipment Placement

Most of the fax analyzers have adequate help screens and documentation. The T.30 specification is also very helpful. For the protocol analyzers, decoding can be a little more difficult because sometimes the encodings are proprietary or the analyzer software does not have the specific decode needed. For fax relay using VoFR and VoATM, Cisco gateways use standards-based Annex D from the FRF11 specification. If the protocol analyzer cannot decode the frame, the frame can be manually decoded through use of this specification. With fax relay and VoIP, a Cisco proprietary format is used for the fax relay packets.

With fax analyzer and protocol analyzer information, you should be able to resolve fax relay problems. Few fax relay problems reach this point, and when they do, escalation and DE resources should already be involved for further assistance.

For more information about troubleshooting fax relay, refer to the Fax Relay Troubleshooting Guide, document 20227.

Fax Detection

Use the following tips to resolve problems that keep fax detection from working correctly:

On the router that you are using for the fax detection application, make sure that you have installed at least the minimum version of Cisco IOS software that is listed in the Cicso Fax Services over IP guide.

Before configuring fax detection, make sure that your voice application is functional by putting a series of calls through.

Before configuring fax detection, make sure that your fax application is functional by sending a series of faxes.

After configuring fax detection, issue the debug voip ivr script command to display debug information from the fax detection script. Put through a series of voice calls and fax calls to ensure correct operation. The debug output that is displayed when you put calls through is indispensable for diagnosing failing calls and finding the source of a problem. It is the only way to verify that parameters are set to the values that you want and that they are actually taking effect. Mistakes such as typing errors in command-line interface (CLI) parameters (for example, typing "moode" for "mode") are not recognized as errors by Cisco IOS. They are accepted without complaint when typed, yet they do not produce the desired effect during operation. It is only by watching the debug output during operation that you find these mistakes.

Make sure that you have configured different DTMF digits for fax and for voice. If you configure both to be the same number, you are not notified immediately, as you would be with other Cisco IOS command errors. You find this error only if the debug voip ivr script command is enabled before a failing call comes in.