Overview
This topic contains an example logs from Influx DB to Postgres migration process.
[root@iot-fnd log]# /opt/cgms-tools/log/influx2postgres.log
====== sample log =========
2025-08-05 17:09:17 INFO Influx2PostgresMigrator:102 - ##### Migration of records from Influx to Postgres process: Started 2025-08-05T15:09:17.144Z #####
2025-08-05 17:09:27 INFO Influx2PostgresMigrator:119 - parallelize : true
2025-08-05 17:09:27 INFO Influx2PostgresMigrator:131 - Total number of measurements to migrate: 6
2025-08-05 17:09:27 INFO Influx2PostgresMigrator:132 - Measurements : [events, historicalnameddata, historicalnameddata_1_hr, historicalnameddata_da, metricshistory, mhistory_1_hr]
2025-08-05 17:09:39 INFO Influx2PostgresMigrator:157 - Measurement: events has a retention period of 90 days.
2025-08-05 17:09:40 INFO Influx2PostgresMigrator:162 - Successfully updated retention period for postgres table: events
2025-08-05 17:09:40 INFO Influx2PostgresMigrator:157 - Measurement: historicalnameddata has a retention period of 7 days.
2025-08-05 17:09:40 INFO Influx2PostgresMigrator:162 - Successfully updated retention period for postgres table: historical_named_data
2025-08-05 17:09:40 INFO Influx2PostgresMigrator:157 - Measurement: historicalnameddata_1_hr has a retention period of 90 days.
2025-08-05 17:09:40 INFO Influx2PostgresMigrator:162 - Successfully updated retention period for postgres table: historical_named_data_hr
2025-08-05 17:09:40 INFO Influx2PostgresMigrator:157 - Measurement: historicalnameddata_da has a retention period of 90 days.
2025-08-05 17:09:40 INFO Influx2PostgresMigrator:162 - Successfully updated retention period for postgres table: historical_named_data_da
2025-08-05 17:09:52 INFO Influx2PostgresMigrator:157 - Measurement: metricshistory has a retention period of 30 days.
2025-08-05 17:09:52 INFO Influx2PostgresMigrator:162 - Successfully updated retention period for postgres table: metrics_history
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:157 - Measurement: mhistory_1_hr has a retention period of 30 days.
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:162 - Successfully updated retention period for postgres table: metrics_history_hr
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:178 - Parallel migration enabled at measurements level. Using maximum of 20 read threads and 32 write threads.
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:246 - Starting migration for measurement: events
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:246 - Starting migration for measurement: historicalnameddata
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:246 - Starting migration for measurement: historicalnameddata_1_hr
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:246 - Starting migration for measurement: mhistory_1_hr
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:246 - Starting migration for measurement: metricshistory
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:246 - Starting migration for measurement: historicalnameddata_da
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:249 - Migration for measurement: mhistory_1_hr started at 2025-08-05T15:10:00.811Z
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:249 - Migration for measurement: historicalnameddata_1_hr started at 2025-08-05T15:10:00.812Z
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:249 - Migration for measurement: metricshistory started at 2025-08-05T15:10:00.816Z
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:249 - Migration for measurement: historicalnameddata started at 2025-08-05T15:10:00.820Z
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:249 - Migration for measurement: events started at 2025-08-05T15:10:00.824Z
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:249 - Migration for measurement: historicalnameddata_da started at 2025-08-05T15:10:00.830Z
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:263 - Max ID value in Postgres table metrics_history_hr before migration: 64
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:263 - Max ID value in Postgres table events before migration: 149
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:263 - Max ID value in Postgres table historical_named_data_da before migration: 146748
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:263 - Max ID value in Postgres table historical_named_data_hr before migration: 3471494
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:263 - Max ID value in Postgres table historical_named_data before migration: 14524653
2025-08-05 17:10:00 INFO Influx2PostgresMigrator:263 - Max ID value in Postgres table metrics_history before migration: 431
2025-06-21 15:48:36 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:48:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:48:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:48:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 4000/132270 records copied (3.02%)
2025-06-21 15:48:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:48:36 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:48:46 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:48:46 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:48:46 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:48:46 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 10000/132270 records copied (7.56%)
2025-06-21 15:48:46 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:48:46 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:48:56 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:48:56 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:48:56 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:48:56 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 20000/132270 records copied (15.12%)
2025-06-21 15:48:56 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:48:56 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:49:06 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:49:06 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:49:06 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:49:06 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 28000/132270 records copied (21.17%)
2025-06-21 15:49:06 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:49:06 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:49:16 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:49:16 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:49:16 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:49:16 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 34000/132270 records copied (25.70%)
2025-06-21 15:49:16 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:49:16 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:49:26 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:49:26 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:49:26 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:49:26 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 40000/132270 records copied (30.24%)
2025-06-21 15:49:26 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:49:26 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:49:36 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:49:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:49:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:49:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 50000/132270 records copied (37.80%)
2025-06-21 15:49:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:49:36 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:49:46 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:49:46 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:49:46 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:49:46 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 54000/132270 records copied (40.83%)
2025-06-21 15:49:46 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:49:46 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:49:56 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:49:56 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:49:56 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:49:56 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 64000/132270 records copied (48.39%)
2025-06-21 15:49:56 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:49:56 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:50:06 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:50:06 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:50:06 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:50:06 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 70000/132270 records copied (52.92%)
2025-06-21 15:50:06 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:50:06 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:50:16 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:50:16 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:50:16 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:50:16 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 80000/132270 records copied (60.48%)
2025-06-21 15:50:16 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:50:16 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:50:26 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:50:26 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:50:26 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:50:26 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 84000/132270 records copied (63.51%)
2025-06-21 15:50:26 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:50:26 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:50:36 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:50:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:50:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:50:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 90000/132270 records copied (68.04%)
2025-06-21 15:50:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:50:36 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:50:46 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:50:46 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:50:46 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:50:46 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 100000/132270 records copied (75.60%)
2025-06-21 15:50:46 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:50:46 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:50:56 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:50:56 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:50:56 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:50:56 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 108000/132270 records copied (81.65%)
2025-06-21 15:50:56 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:50:56 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:51:06 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:51:06 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:51:06 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:51:06 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 114000/132270 records copied (86.19%)
2025-06-21 15:51:06 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:51:06 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:51:16 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:51:16 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:51:16 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:51:16 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 120000/132270 records copied (90.72%)
2025-06-21 15:51:16 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:51:16 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:51:26 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:51:26 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:51:26 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:51:26 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 130000/132270 records copied (98.28%)
2025-06-21 15:51:26 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:51:26 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:51:29 INFO Influx2PostgresMigrator:331 - Migration completed for measurement: events in 183003 ms
2025-06-21 15:51:36 INFO Influx2PostgresStatsLogger:124 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:51:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data: 5060/5060 records copied (100.00%)
2025-06-21 15:51:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_hr: 9960/9960 records copied (100.00%)
2025-06-21 15:51:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] events: 132270/132270 records copied (100.00%)
2025-06-21 15:51:36 INFO Influx2PostgresStatsLogger:129 - [MIGRATION] historical_named_data_da: 380/380 records copied (100.00%)
2025-06-21 15:51:36 INFO Influx2PostgresStatsLogger:132 - ############################################ INFLUX TO POSTGRES ###########################################
2025-06-21 15:51:39 INFO Influx2PostgresMigrator:182 - ##### Migration of records from Influx to Postgres process: Completed 2025-06-21T13:51:39.535Z Total time taken: 00:03:14 #####