Data Flow from Logger to Historical Data Server
Assuming a full-duplex, fault-tolerant implementation, data is sent from CallRouter A to Logger A and from CallRouter B to Logger B.
The Logger Central Database forwards (replicates) historical data to corresponding historical tables on the Historical Database Server in summary intervals. These data populate the historical interval and daily reports.
Two Administration & Data Servers are typically set up as HDS machines. A similar fault-tolerant strategy applies to the HDS—when the primary HDS fails, the Administration Client automatically switches over to use the backup HDS.
Each Historical Data Server (HDS) is connected to a single Logger.
Recovery and Replication
The recovery key is the base key for all historical data tables. This key is always incremented by 1 before a new record is inserted into any historical table.
In a duplex configuration, the Logger that finishes initializing first is designated the primary Logger (although both the Loggers are always active and working in parallel). The recovery key is always initialized by the primary Logger. The recovery key is based on the current GMT date and time and always has a value greater than any previous value generated. This helps the recovery process to keep the Loggers in sync.
The replication process may have a latency of about one to five minutes because the Logger replicates data table-by-table on the HDS.
Each historical table on the Logger Central Database has two corresponding temporary tables that act as buffers to incoming historical data. As they have minimal indexes, the temporary tables speed up the process of inserting data into the corresponding actual table in the Logger Central Database.
As the incoming historical data is written to the corresponding temporary tables by the Logger, the Recovery process reads the data from the temporary tables and performs a bulk insert operation of up to 2000 records into the actual historical tables.
In a duplex configuration, the recovery process keeps the historical data on the two Loggers in sync, using the recovery keys. The historical data between the Loggers is synced directly using actual tables; temporary tables are not used by the recovery process.
The Replication process is responsible for replicating data that has been committed to the Logger Central database to the HDS database.
The Replication mechanism consists of two processes: the Replication Server Process that runs on the Logger and the Replication Client Process that runs on the Distributor on which HDS has also been installed.
The Replication Client sends a request to the Replication Server requesting historical data that have associated Recovery Keys higher than those currently on corresponding historical tables. The Replication server sends the requested data back as a set of 2000 records each time.
The Replication server reads the historical data from the actual tables on the Logger and sends it to the Replication Client, which writes the historical data to the actual corresponding tables in the HDS database. Temporary tables are not used to replicate the data from the Logger to the HDS.
Possible Points of Delay or Inconsistency
If the Logger connected to the HDS goes offline, the HDS does not connect to a different Logger. For example, if the HDS is connected to Logger B and Logger B fails, the HDS does not connect to Logger A. When Logger B comes back up, it recovers data from Logger A and begins to receive current historical information. Once the Logger has recovered all of the data from Logger A, it begins to replicate this data to the HDS.
If reports are run from this HDS for recent intervals while the Logger is offline or while the Logger is in the process of recovering or replicating data, you might not see data for those intervals in reports. This situation is temporary, and you will see the data once the replication process for the tables used by the reports is complete. If you are using a fault-tolerant system with two HDS Administration & Data Servers, you can run reports using the backup HDS while the primary HDS is not receiving data.
If the HDS goes offline and you are using a fault-tolerant system with two HDS Administration & Data Servers, you can run reports using the backup HDS. When the HDS comes back up, it recovers data from the last HDS data backup and also replicates data from the Logger for the most recent data not available in the backup.
The recovery data replication is faster than regular Logger-HDS data replication. Once the HDS has recovered to its typical Logger-HDS latency of one to five minutes, data replication proceeds as usual.
If you are not using a fault-tolerant system, you will not see data in historical reports until the HDS is restored. You might also notice missing data as the replication process is in progress. This situation is temporary and you will see the data once the replication process for the tables utilized by the reports is complete.