Herstel procedure voor ultra-M AutoVNF

Inleiding

In dit document worden de stappen beschreven die vereist zijn om een AutoVNF in een Ultra-M te herimplementeren. De AutoVNF is verantwoordelijk voor de installatie van de Virtual Network Function Manager (VNFM) en Virtual Network Function (VNF). 

Voorcontroles

1. Meld u aan bij de OpenStack Platform Director (OSPF) en controleer de status van virtuele machine (VM's) in het gezondheidsrapport.

[stack@labucs300-ospd ~]$ cat /var/log/cisco/ultram-health/*.report | grep -i xxx

[stack@labucs300-ospd ~]$ cat /var/log/cisco/ultram-health/ultram_health_uas.report
---------------------------------------------------------------------------------------------------------
 VNF-ID/VNFD-ID           | UAS Node | Status| Error Info, if any         
---------------------------------------------------------------------------------------------------------
10.10.10.40/LABPGW300-UAS | autovnf  | :-)   | LABPGW300-UAS:(alive)
                          |          |       |   labucs300-UAS-LABPGW300-UAS-core-UAS2-2:(alive)
                          |          |       |   labucs300-UAS-LABPGW300-UAS-core-UAS2-1:(alive)
10.10.10.40/LABPCF300-UAS | autovnf  | :-)   | LABPCF300-UAS:(alive)
                          |          |       |   labucs300-UAS-LABPCF300-UAS-core-UAS1-2:(alive)
                          |          |       |   labucs300-UAS-LABPCF300-UAS-core-UAS1-1:(alive)
10.10.10.45/LABPCF300-UGP | vnf-em   | :-)   | LABPCF300-UGP:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-EM1-3:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-EM1-2:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-EM1-1:(alive)
10.10.10.45/LABPCF300-ESC | esc      | :-)   | LABPCF300-ESC:(alive)
                          |          |       |   LABPCF300-LABPCF300-ESC-core-ESC1-1:(alive)
                          |          |       |   LABPCF300-LABPCF300-ESC-core-ESC1-2:(alive)
10.10.10.45/LABPCF300-UGP | vnf      | :-)   | LABPCF300-UGP:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-CF-VDU-1:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-CF-VDU-0:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-SF-VDU-3:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-SF-VDU-2:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-SF-VDU-1:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-SF-VDU-0:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-SF-VDU-6:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-SF-VDU-5:(alive)
                          |          |       |   LABPCF300-LABPCF300-UGP-core-LABPCF300-SF-VDU-4:(alive)
10.10.10.48/LABPGW300-UGP | vnf-em   | :-)   | LABPGW300-UGP:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-EM2-2:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-EM2-3:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-EM2-1:(alive)
10.10.10.48/LABPGW300-ESC | esc      | :-)   | LABPGW300-ESC:(alive)
                          |          |       |   LABPGW300-LABPGW300-ESC-core-ESC2-1:(alive)
                          |          |       |   LABPGW300-LABPGW300-ESC-core-ESC2-2:(alive)
10.10.10.48/LABPGW300-UGP | vnf      | :-)   | LABPGW300-UGP:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-SF-VDU-4:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-SF-VDU-5:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-SF-VDU-6:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-SF-VDU-0:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-SF-VDU-1:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-SF-VDU-2:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-SF-VDU-3:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-CF-VDU-0:(alive)
                          |          |       |   LABPGW300-LABPGW300-UGP-core-LABPGW300-CF-VDU-1:(alive)
---------------------------------------------------------------------------------------------------------

2. Controleer de AutoVNF-status.

[stack@labucs300-ospd ~]$ source *core
[stack@labucs300-ospd ~]$ nova list | grep LABPGW300-UAS-core-UAS2
| 8608fda4-b763-4753-95ff-2e07852098e3 | labucs300-UAS-LABPGW300-UAS-core-UAS2-1 | ACTIVE | - | Running | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.15; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.7 |
| 19f4496c-3907-4ea5-84c9-e5a6ef222392 | labucs300-UAS-LABPGW300-UAS-core-UAS2-2 | ACTIVE | - | Running | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.17; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.16

labucs300-UAS-LABPGW300-UAS-core-UAS2-2 -> VNFM2-UAS-VIP Primary
labucs300-UAS-LABPGW300-UAS-core-UAS2-1 -> Secondary 

3. Meld u aan bij AutoIT en controleer de logbestanden met de hartslag.

ubuntu@labucs300-autoit-2:~$ grep "'restarting'" /var/log/cisco/uas/heartbeat.log
2021-02-22 01:41:42,808 - 192.0.2.15: Notify Event: {'action': 'restart', 'source': 'heartbeat', 'event': 'restarting', 'ip': '192.0.2.15'}
2021-02-22 01:45:42,251 - 192.0.2.15: Notify Event: {'action': 'restart', 'source': 'heartbeat', 'event': 'restarting', 'ip': '192.0.2.15'}
2021-02-23 01:43:36,013 - 192.0.2.15: Notify Event: {'action': 'restart', 'source': 'heartbeat', 'event': 'restarting', 'ip': '192.0.2.15'}
2021-02-23 01:45:55,785 - 192.0.2.15: Notify Event: {'action': 'restart', 'source': 'heartbeat', 'event': 'restarting', 'ip': '192.0.2.15'}
2021-02-24 01:45:19,680 - 192.0.2.15: Notify Event: {'action': 'restart', 'source': 'heartbeat', 'event': 'restarting', 'ip': '192.0.2.15'}

ubuntu@labucs300-autoit-2:~$ cd /var/log/cisco/uas
ubuntu@labucs300-autoit-2:/var/log/cisco/uas$ grep "Rebooting Instance" uas_USPCHBWorker.log
2019-06-26 18:26:13,088 - Rebooting Instance: 19f4496c-3907-4ea5-84c9-e5a6ef222392
2019-06-29 03:45:12,710 - Rebooting Instance: 19f4496c-3907-4ea5-84c9-e5a6ef222392
2020-07-17 00:46:25,800 - Rebooting Instance: 19f4496c-3907-4ea5-84c9-e5a6ef222392
2020-07-18 00:47:13,347 - Rebooting Instance: 19f4496c-3907-4ea5-84c9-e5a6ef222392
2020-07-18 05:11:11,133 - Rebooting Instance: 19f4496c-3907-4ea5-84c9-e5a6ef222392
2020-07-18 5:16:07,333 - Rebooting Instance: 19f4496c-3907-4ea5-84c9-e5a6ef222392 

AutoVNF-back-up

AutoVNF is verantwoordelijk voor het verzorgen van individuele VNFM en VNF. AutoDeployment stuurt de configuratie die vereist is om zowel VPN als VNF te concretiseren naar AutoVNF en AutoVNF om deze handeling te doen. Om VNFM op te halen,

AutoVNF praat rechtstreeks met VIM/openstack en zodra VNFM is geactiveerd, maakt AutoVNF gebruik van VNFM om VNF aan te halen.

AutoVNF heeft 1:1 redundantie en in UltraM instelling, 2 AutoVNF VMs die in HA-modus draaien.

Back-uplijn:

• Configuratie uitvoeren
• ConfD CDB DB
• AutoVNF-documenten (van elk geval van AutoVNF)
• Syslog-configuratie

Belangrijk: De back-ups moeten worden uitgevoerd voordat ze op de ingestelde POD/site worden geactiveerd en naar de reserveserver worden geüpload. 

1. Stel ha_debug in op zowel het primaire als het secundaire AutoIT in /opt/uas_baseconfig.txt.

Opmerking: De ha_debug vlag is allemaal van kapitaal. De functie is AAN of UIT.

ubuntu@labucs300-autoit-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 172.16.181.7
ha-secret: f99d04acb84807c4c6c6c0eaad392b5c
orch-ips: 172.16.181.5,172.16.181.8
orch-intf: eth0
prov-ha-vip: 172.16.181.13
prov-intf: eth0
profile: AUTOIT
ha_debug: ON

ubuntu@labucs300-autoit-1:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 172.16.181.7
ha-secret: f99d04acb84807c4c6c6c0eaad392b5c
orch-ips: 172.16.181.5,172.16.181.8
orch-intf: eth0
prov-ha-vip: 172.16.181.13
prov-intf: eth0
profile: AUTOIT
ha_debug: ON

2. Stop de AutoIT-service op Primair AutoIT met deze opdrachten:

ssh   ubuntu@ < AutoIT Floating IP>
sudo -i
service autoit stop

Dit om te voorkomen dat AutoIT automatisch herstel van Ultra Automation Services (UAS) in gang zet wanneer de shutdown op latere stappen wordt gestart.

ubuntu@labucs300-autoit-2:~$ sudo -i
root@labucs300-autoit-2:~#
root@labucs300-autoit-2:~# service autoit status
autoit start/running, process 25001
root@labucs300-autoit-2:~# service autoit stop

3. Stel ha_debug in op ON zowel voor Primaire als voor Secundaire AutoVNF (UAS) in het bestand /opt/uas_baseconfig.txt.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: ON

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ ssh ubuntu@192.0.2.15
ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-1:~$ cat /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: ON

4. Sluiting van de secundaire UAS van de OspD met instemming.

 . corerc ; openstack server stop <VMName>

labucs300-UAS-LABPGW300-UAS-core-UAS2-2 -> VNFM2-UAS-VIP Primary
labucs300-UAS-LABPGW300-UAS-core-UAS2-1 -> Secondary

[stack@labucs300-ospd ~]$ nova list | grep LABPGW300-UAS-core-UAS2
| 8608fda4-b763-4753-95ff-2e07852098e3 | labucs300-UAS-LABPGW300-UAS-core-UAS2-1 | ACTIVE | - | Running | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.15; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.7 |
| 19f4496c-3907-4ea5-84c9-e5a6ef222392 | labucs300-UAS-LABPGW300-UAS-core-UAS2-2 | ACTIVE | - | Running | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.17; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.16

[stack@labucs300-ospd ~]$. corerc ; openstack server stop labucs300-UAS-LABPGW300-UAS-core-UAS2-1

5. Stop de uas-confd en autovF diensten op de UAS met deze opdrachten:

service uas-confd stop
service autovnf stop

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ sudo -i 
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service uas-confd status
uas-confd start/running, process 1305
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service autovnf status
autovnf start/running, process 24208
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service uas-confd stop
uas-confd stop/waiting
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service autovnf stop
autovnf stop/waiting

6. Neem een back-up van de UAS conf-database en kopieer deze naar een reserveserver met deze opdrachten:

cd /opt/cisco/usp/uas/confd-latest/var/confd/
tar -cvf <pod>_<VNF>_UAS_cdb_backup.tar cdb/

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# cd /opt/cisco/usp/uas/confd-latest/var/confd/
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# tar -cvf Autovnf_cdb_backup.tar cdb/
cdb/
cdb/O.cdb
cdb/C.cdb
cdb/aaa_init.xml
cdb/A.cdb
root@labucs300-uas-LABPGW300-uas-core-uas2-2:/opt/cisco/usp/uas/confd-latest/var/confd# ll Autovnf_cdb_backup_cdb_backup.tar 
total 1612
drwxr-xr-x 3 root root 4096 Jan 24 2017 ..
drwxr-xr-x 2 root root 4096 Jan 24 2017 log
drwxr-xr-x 8 root root 4096 Oct 11 11:30 webui
drwxr-xr-x 2 root root 4096 Oct 19 19:18 candidate
drwxr-xr-x 2 root root 4096 Oct 23 13:00 rollback
drwxr-xr-x 2 root root 4096 Oct 28 17:00 cdb
drwxr-xr-x 3 root root 4096 Oct 28 17:00 state
drwxr-xr-x 8 root root 4096 Oct 31 18:00 .
-rw-r--r-- 1 root root 1617920 Oct 31 18:00 Autovnf_cdb_backup.tar

7. Start de ua-confd en autovF services opnieuw op de UAS met deze opdrachten:

service uas-confd start
service autovnf start

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service uas-confd start
uas-confd start/running, process 13852
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service autovnf start
autovnf start/running, process 13853

8. Start secundaire UAS opnieuw vanaf spatie met deze opdrachten:

. corerc ; openstack server start <VMName>

[stack@labucs300-ospd ~]$. corerc ; openstack server start labucs300-UAS-LABPGW300-UAS-core-UAS2-1

9. Controleer of zowel de primaire als de secundaire UAS in de opdracht levend zijn.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ sudo -i
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# confd_cli -u admin -C
Welcome to the ConfD CLI
admin connected from 127.0.0.1 using console on labucs300-uas-LABPGW300-uas-core-uas2-2
labucs300-uas-LABPGW300-uas-core-uas2-2#show uas 
uas version 6.2.0
uas state active
uas external-connection-point 192.0.2.8
INSTANCE IP STATE ROLE 
-----------------------------------
0.0.0.0 error CONFD-Secondary
192.0.2.15 alive CONFD-Secondary
192.0.2.17 alive CONFD-Primary

10. Wijzig ha_debug in OFF op zowel Primair als Secundair AutoVNF (UAS) in het bestand /opt/uas_baseconfig.txt.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: OFF
ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ ssh ubuntu@192.0.2.15
ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-1:~$ cat /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: OFF

11. Start de automatische service opnieuw op Primaire AutoIT met de opdracht voor startvertraging van de servicetechnicus.

ubuntu@labucs300-autoit-2:~$ sudo -i
root@labucs300-autoit-2:~# service autoit start

12. Controleer dat zowel Primaire als Secundaire AutoIT met onze opdracht tot leven komt.

ubuntu@labucs300-autoit-2:~$ sudo -i
root@labucs300-autoit-2:~# confd_cli -u admin -C 
Welcome to the ConfD CLI
admin connected from 127.0.0.1 using console on labucs300-autoit-2
labucs300-autoit-2#show uas
uas version 6.2.0
uas state active
uas external-connection-point 172.16.181.7
INSTANCE IP STATE ROLE 
-----------------------------------
172.16.181.5 alive CONFD-Secondary 
172.16.181.8 alive CONFD-Primary

13. Stel ha_debug in op OFF voor zowel primaire als secundaire AutoIT in het bestand /opt/uas_baseconfig.txt.

ubuntu@labucs300-autoit-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 172.16.181.7
ha-secret: f99d04acb84807c4c6c6c0eaad392b5c
orch-ips: 172.16.181.5,172.16.181.8
orch-intf: eth0
prov-ha-vip: 172.16.181.13
prov-intf: eth0
profile: AUTOIT
ha_debug: OFF

ubuntu@labucs300-autoit-1:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 172.16.181.7
ha-secret: f99d04acb84807c4c6c6c0eaad392b5c
orch-ips: 172.16.181.5,172.16.181.8
orch-intf: eth0
prov-ha-vip: 172.16.181.13
prov-intf: eth0
profile: AUTOIT
ha_debug: OFF

14. Verzamel logbestanden van de UAS en breng ze over naar een reserveserver.

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# cd /opt/cisco/usp/uas/confd-latest/var/confd/
root@labucs300-uas-LABPGW300-uas-core-uas2-2:/opt/cisco/usp/uas/confd-latest/var/confd# cd /opt/cisco/usp/uas/scripts/
root@labucs300-uas-LABPGW300-uas-core-uas2-2:/opt/cisco/usp/uas/scripts# sudo ./collect-uas-logs.sh 
Dumping output for show transaction in file /tmp/uas-logs/transactions.txt
Dumping output for show log in file /tmp/uas-logs/transactions.txt
Dumping output for show running-config in file /tmp/uas-logs/confd_output.txt
Dumping output for show uas in file /tmp/uas-logs/confd_output.txt
Dumping output for show usp in file /tmp/uas-logs/confd_output.txt
..........

15. Meld u aan bij de secundaire AutoVNF en herhaal de vorige stap om de logbestanden te verzamelen en naar de reserveserver over te brengen.

16. Maak een back-up van de systeemconfiguratie op de primaire en secundaire AutoVPN-VM's en breng deze over naar de reserveserver. De bestanden bevinden zich in deze bestanden:

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# ls /etc/rsyslog.d/00-autovnf.conf 
/etc/rsyslog.d/00-autovnf.conf
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# ls /etc/rsyslog.conf
/etc/rsyslog.conf

17. Inschakelen van automatische service met de servicetechnicus start de opdracht op primaire AutoIT.

ubuntu@labucs300-autoit-2:~$ sudo -i
root@labucs300-autoit-2:~#
root@labucs300-autoit-2:~# service autoit start
autoit start/running, process 25001

18. Stel de ha_debug vlaggenmodus in op OFF in de /opt/uas_baseconfig.txt op Primaire AutoVNF en AutoIT.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: OFF

ubuntu@labucs300-autoit-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 172.16.181.7
ha-secret: f99d04acb84807c4c6c6c0eaad392b5c
orch-ips: 172.16.181.5,172.16.181.8
orch-intf: eth0
prov-ha-vip: 172.16.181.13
prov-intf: eth0
profile: AUTOIT
ha_debug: OFF

19. Bevestig uas-confd en autovann-diensten die op UAS-netwerken actief zijn.

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service uas-confd status
uas-confd start/running, process 1305
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service autovnf status
autovnf start/running, process 24208

AutoVFN opnieuw implementeren

1. Meld u aan bij AutoDeployment en noteer de UAS-instantie.

ubuntu@labucs300-autodeploy-2:~$ sudo su 
root@labucs300-autodeploy-2:/home/ubuntu# confd_cli -u admin -C
Welcome to the ConfD CLI
admin connected from 127.0.0.1 using console on labucs300-autodeploy-2
labucs300-autodeploy-2#show nsr
 nsr         LABSGW300-instance
 nsd         LABSGW300
 vnfr        [ LABPCF300-LABPCF300-ESC LABPCF300-LABPCF300-UGP ]
 vnf-package [ usp_6_2_b8 ]
 vim-artifact vim_art_rack
nsr LABPGW300-instance
 nsd         LABPGW300
 vnfr        [ LABPGW300-LABPGW300-ESC LABPGW300-LABPGW300-UGP ]
 vnf-package [ usp_6_2_b8 ]
 vim-artifact vim_art_rack
nsr labucs300-UAS-instance
 nsd         labucs300-UAS
 vnfr        [ labucs300-UAS-LABPCF300-UAS labucs300-UAS-LABPGW300-UAS ]
 vnf-package [ usp_6_2_b8 ]
 vim-artifact vim_art_rack

2. Deactiveren AutoVNF vanaf AutoDeployment met de opdracht nsd-id <nsd-id> vnfd <vnfd-id>.

ubuntu@labucs300-autodeploy-2:~$ /opt/cisco/usp/uas/confd-6.3.1/bin/confd_cli -u admin -C
Welcome to the ConfD CLI
admin connected from 10.10.10.10 using ssh on labucs300-autodeploy-2
labucs300-autodeploy-2#nsd:deactivate nsd-id labucs300-UAS vnfd [LABPGW300-UAS]
transaction-id 1560431372-357328 

3. Bevestig de status van de transactie.

labucs300-autodeploy-2#show transaction 
                                                           DEPLOYMENT                                                 STATUS 
TX ID                               TX TYPE                ID            TIMESTAMP                        STATUS      DETAIL 
------------------------------------------------------------------------------------------------------------------------------
1560431372-357328                   activate-ns-deployment labucs300-UAS 2019-06-13T13:09:32.357355-00:00 in-progress - 
1560431372-357328/1560431373-102024 activate-ns-deployment labucs300-UAS 2019-06-13T13:09:33.102041-00:00 in-progress - 

4. Controleer de boeken van de transactie. In dit geval worden de gegevens300-UAS-transacties: 1560431372-357328 en 1560431372-357328/1560431373-102024.

labucs300-autodeploy-2#show log 1560431372-357328 | display xml
<config xmlns="http://tail-f.com/ns/config/1.0">
<log xmlns="http://www.cisco.com/usp/nfv/usp-transaction">
<tx-id>1560431372-357328</tx-id>
<log>
2019-06-13 13:09:33,367 - Send Deployment notification for: labucs300-UAS-instance
2019-06-13 13:09:33,375 - Deployment activate-ns-deployment: labucs300-UAS started
2019-06-13 13:09:33,378 - Adding NSR: labucs300-UAS-instance
2019-06-13 13:09:33,385 - Start pipeline of 1 tasks
2019-06-13 13:09:33,390 - Scheduling Task: labucs300-UAS
2019-06-13 13:09:33,400 - Waiting for all workers to finish the transactions
2019-06-13 13:15:00,006 - Deployment activate-ns-deployment: labucs300-UAS succeeded
2019-06-13 13:15:00,020 - Send Deployment notification for: labucs300-UAS-instance
2019-06-13 13:09:33,437 - Send Deployment notification for: labucs300-UAS-instance-deploy
2019-06-13 13:09:33,441 - Deployment activate-ns-deployment: labucs300-UAS started
……………

labucs300-autodeploy-2#show log 1560431372-357328/1560431373-102024 | display xml
<config xmlns="http://tail-f.com/ns/config/1.0">
<log xmlns="http://www.cisco.com/usp/nfv/usp-transaction">
<tx-id>1560431372-357328/1560431373-102024</tx-id>
<log>
2019-06-13 13:09:33,437 - Send Deployment notification for: labucs300-UAS-instance-deploy
2019-06-13 13:09:33,441 - Deployment activate-ns-deployment: labucs300-UAS started
2019-06-13 13:09:33,446 - Adding NSR: labucs300-UAS-instance, VNFR: labucs300-UAS-LABPCF300-UAS, vlrs: None
2019-06-13 13:09:33,453 - Adding NSR: labucs300-UAS-instance, VNFR: labucs300-UAS-LABPGW300-UAS, vlrs: None
2019-06-13 13:09:33,463 - VNF deployment pre-check success(all-not-present)
2019-06-13 13:09:33,472 - VNF-Package deployment pre-check success(all-not-present)
2019-06-13 13:09:33,481 - VIM-Artifact deployment pre-check success
2019-06-13 13:09:33,487 - Skipping VIM-Orch pre-deployment, since VIM-Orch is not defined
2019-06-13 13:09:33,496 - Skipping VIM pre-deployment, since VIM is not defined
2019-06-13 13:09:33,499 - NS pre-check success
2019-06-13 13:09:33,503 - Copying '/home/ubuntu/usp-6_2_b8.iso' to '/var/cisco/isos/labucs300-UAS_usp_6_2_b8'
2019-06-13 13:09:53,359 - Updated path to URL 'http://172.16.181.14:5000/isos/labucs300-UAS_usp_6_2_b8'

5. Wacht tot de transactie is voltooid en bevestig de staat.

labucs300-autodeploy-2#show transaction 
                                                           DEPLOYMENT                                            STATUS 
TX ID                               TX TYPE                ID            TIMESTAMP                        STATUS DETAIL 
------------------------------------------------------------------------------------------------------------------------------
1560431372-357328                   activate-ns-deployment labucs300-UAS 2019-06-13T13:09:32.357355-00:00 success - 
1560431372-357328/1560431373-102024 activate-ns-deployment labucs300-UAS 2019-06-13T13:09:33.102041-00:00 success - 

6. Activeer AutoVPN vanaf AutoDeployment met de opdracht nsd-id <nsd-id> vnfd <vnfd-id>.

ubuntu@labucs300-autodeploy-2:~$ /opt/cisco/usp/uas/confd-6.3.1/bin/confd_cli -u admin -C
Welcome to the ConfD CLI
admin connected from 10.253.110.47 using ssh on labucs300-autodeploy-2
labucs300-autodeploy-2#nsd:activate nsd-id labucs300-UAS vnfd [LABPGW300-UAS]
transaction-id 1560431371-357330 

7. Controleer de status van de transactie en verzamel logbestanden van de transactie met deze opdrachten:

show transaction
show log <transaction-id> | display xml
show log <transaction-id> | display xml

8. Wacht tot de transacties zijn voltooid. De opdracht tonen de staat van de transacties.

Terug zetten

1. Stel bij Primair AutoIT de ha_debug vlaggenmodus in op ON in /opt/uas_baseconfig.txt.

ubuntu@labucs300-autoit-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 172.16.181.7
ha-secret: f99d04acb84807c4c6c6c0eaad392b5c
orch-ips: 172.16.181.5,172.16.181.8
orch-intf: eth0
prov-ha-vip: 172.16.181.13
prov-intf: eth0
profile: AUTOIT
ha_debug: ON

2. Bij primaire automatische IT-stopdienst. Deze stap is om te voorkomen dat AutoIT UAS automatisch herstelt.

ubuntu@labucs300-autoit-2:~$ sudo -i
root@labucs300-autoit-2:~#
root@labucs300-autoit-2:~# service autoit status
autoit start/running, process 25001
root@labucs300-autoit-2:~# service autoit stop

3. Stel de ha_debug vlaggenmodus in op /opt/uas_baseconfig.txt op Primaire UAS.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: ON

4. Stel de ha_debug vlaggenmodus in op ON in /opt/uas_baseconfig.txt.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-1:~$ cat /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: ON

5. Op OSPF-stop secundaire UAS-server met de opdracht OpenStack.

labucs300-UAS-LABPGW300-UAS-core-UAS2-2 -> VNFM2-UAS-VIP Primary
labucs300-UAS-LABPGW300-UAS-core-UAS2-1 -> Secondary

[stack@labucs300-ospd ~]$. corerc ; openstack server stop labucs300-UAS-LABPGW300-UAS-core-UAS2-1

6. Op primaire UAS stop de ua-vormige service.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ sudo -i 
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service uas-confd status
uas-confd start/running, process 1305
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# service uas-confd stop
uas-confd stop/waiting

7. Op primaire UAS kopieert u het geback-upte archief van CDB naar directory /opt/cisco/usp/uas/confd-last/var/confd/.

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# cp Autovnf_cdb_backup.tar to /opt/cisco/usp/uas/confd-latest/var/confd/

8. Op Primaire UAS verwijdert u bestanden onder een CBD-map.

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# cd /opt/cisco/usp/uas/confd-latest/var/confd/ ; rm cdb/*

9. Op Primaire UAS extraheert u bestanden uit het CDB-reservekopiebestand.

root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# cd /opt/cisco/usp/uas/confd-latest/var/confd/ ; tar -xvf <archive_backup_tar_file>

10. Herstart primaire UAS op OSPD met opdrachten voor OpenStack.

[stack@labucs300-ospd ~]$ source *core
[stack@labucs300-ospd ~]$ nova list | grep LABPGW300-UAS-core-UAS2
| 8608fda4-b763-4753-95ff-2e07852098e3 | labucs300-UAS-LABPGW300-UAS-core-UAS2-1 | ACTIVE | - | Running | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.15; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.7 |
| 19f4496c-3907-4ea5-84c9-e5a6ef222392 | labucs300-UAS-LABPGW300-UAS-core-UAS2-2 | ACTIVE | - | Running | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.17; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.16

labucs300-UAS-LABPGW300-UAS-core-UAS2-2 -> VNFM2-UAS-VIP Primary
labucs300-UAS-LABPGW300-UAS-core-UAS2-1 -> Secondary

[stack@labucs300-ospd ~]$ nova reboot --hard 19f4496c-3907-4ea5-84c9-e5a6ef222392
Request to reboot server <Server: auto-testautovnf1-uas-2> has been accepted.

11. Wacht tot de primaire UAS op is. Controleer de status op UAS op Primair UAS na de herstart. De primaire toestand leeft terwijl de secundaire toestand onbekend is.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ sudo -i
root@labucs300-uas-LABPGW300-uas-core-uas2-2:~# confd_cli -u admin -C
Welcome to the ConfD CLI
admin connected from 127.0.0.1 using console on labucs300-uas-LABPGW300-uas-core-uas2-2
labucs300-uas-LABPGW300-uas-core-uas2-2#show uas 
uas version 6.2.0
uas state active
uas external-connection-point 192.0.2.8
INSTANCE IP STATE ROLE 
-----------------------------------
192.0.2.15 unknown CONFD-Secondary 
192.0.2.17 alive CONFD-Primary

12. Start op OSPF-niveau secundaire UAS met de OpenStack-opdracht.

[stack@labucs300-ospd ~]$. corerc ; openstack server start labucs300-UAS-LABPGW300-UAS-core-UAS2-1

13. Controleer op het OSPD-overleg of de staten van de primaire en secundaire UAS actief zijn.

[stack@labucs300-ospd ~]$ openstack server list | grep labucs300-UAS-LABPGW300
| 19f4496c-3907-4ea5-84c9-e5a6ef222392 | labucs300-UAS-LABPGW300-UAS-core-UAS2-2 | ACTIVE | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.17; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.16 | labucs300-UAS-usp_6_2_b8-core-uas |
| 8608fda4-b763-4753-95ff-2e07852098e3 | labucs300-UAS-LABPGW300-UAS-core-UAS2-1 | ACTIVE | labucs300-UAS-LABPGW300-ORCH-NW=192.0.2.15; labucs300-UAS-LABPGW300-MGMT-NW=192.0.2.7 | labucs300-UAS-usp_6_2_b8-core-uas |

14. Controleer op primaire UAS of de toestand van de primaire en secundaire UAS nog in leven is.

labucs300-uas-LABPGW300-uas-core-uas2-2#show uas 
uas version 6.2.0
uas state active
uas external-connection-point 192.0.2.8
INSTANCE IP STATE ROLE 
-----------------------------------
192.0.2.15 alive CONFD-Secondary
192.0.2.17 alive CONFD-Primary

15. Start de autoIT-dienst bij primaire auto.

ubuntu@labucs300-autoit-2:~$ sudo -i
root@labucs300-autoit-2:~# service autoit start

16. Controleer dat de SSH-sessie (Secure Shell) enkele minuten lang wordt voortgezet in Primaire en secundaire UAS.

17. Stel op Primaire UAS de ha_debug vlaggenmodus in op /opt/uas_baseconfig.txt.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: OFF

18.  Stel bij Secundaire UAS de ha_debug vlaggenmodus in op /opt/uas_baseconfig.txt.

ubuntu@labucs300-uas-LABPGW300-uas-core-uas2-1:~$ cat /opt/uas_baseconfig.txt
ha: true
ha-vip: 192.0.2.8
ha-secret: d5a9fee60ddae4fe357677dcf1320e51
orch-ips: 192.0.2.15,192.0.2.17
orch-intf: eth0
profile: AUTOVNF
ha_debug: OFF

19. Stel op Primair AutoIT de ha_debug vlaggenmodus in op /opt/uas_baseconfig.txt.

 ubuntu@labucs300-autoit-2:~$ vi /opt/uas_baseconfig.txt
ha: true
ha-vip: 172.16.181.7
ha-secret: f99d04acb84807c4c6c6c0eaad392b5c
orch-ips: 172.16.181.5,172.16.181.8
orch-intf: eth0
prov-ha-vip: 172.16.181.13
prov-intf: eth0
profile: AUTOIT
ha_debug: OFF

20. Controleer de 00-autovnf.conf- en rsyslog.conf-bestanden op AutoVNF en herstel deze uit de vorige back-up.

ubuntu@autoit-tb1-autovnf1-core-avf-1:~#sudo su
root@autoit-tb1-autovnf1-core-avf-1:~#ls /etc/rsyslog.d/00-autovnf.conf
00-autovnf.conf

root@autoit-tb1-autovnf1-core-avf-1:~#/home/ubuntu#ls /etc/rsyslog.conf
rsyslog.conf

nacontroles

Controleer op OSPF of beide AutoVNF's actief zijn en controleer het Ultra-M gezondheidsrapport.

[stack@labucs300-ospd ~]$ cat /var/log/cisco/ultram-health/*.report | grep -i xxx

[stack@labucs300-ospd ~]$ cat /var/log/cisco/ultram-health/ultram_health_uas.report