The Enterprise requires an operator to add, modify, and/or delete a user with information based on APN and IP pools. The tool generates and applies the required configuration to add, modify, or delete an APN in the CUPS environment.

The following operations can be performed:

• Enterprise Addition: A new APN is added with required number of IPv4/IPv6 pools.

• Enterprise Modification: IP pools can be added/deleted for an existing APN.

• Enterprise Deletion: An APN will be deleted.

In 21.20.13 and later releases, the IOB tool also supports the onboarding of one or more virtual APNs in one operation. As part of this operation, one or more existing APNs can be modified to reference these new virtual APNs. Similarly, the tool also supports deleting the set of virtual APNs onboarded together and simultaneously removing existing references to those APNs from other APNs.

On ASR 5500, Enterprise addition consists of adding a new APN. For CUPS, along with the APN configuration, we must include the correct UP Group and SGi context configuration.

The IOB tool takes inputs from the Provisioning tool, chooses the best suited UP Group and SGi context for the APN, and configures the CP and UP. The IOB tool also allows modification of the APN configuration (adding/deleting the IP pools) and Deletion of an APN.

If onboarding multiple APNs, the APN configuration section must list:

• The configuration for all onboarding APNs and

• APNs referring to them (in case of virtual APNs)

In the preceding scenario, all the APNs get onboarded onto the same UP group and SGi context.

The IOB tool is shipped as Linux executable. All dependencies, like Pexpect and connection management library, are packaged into the standalone .exe file.

The tool is shipped with StarOS images and signed with the same keys that are used for StarOS VPC-SI image.

The executable tool requires the following environment:

• RedHat Enterprise Linux 7.6 (or CentOS equivalent) 64-bit installation

• OpenSSL version 1.0.2.k-fips

• The following shared libraries are installed under /lib64 (these are typically present in a standard RHEL or CentOS installation):

  • libdl.so.2

  • libz.so.1

  • libc.so.6

  • ld-linux-x86-64.so.2

• Read, write, execute permissions for /tmp directory. While executing, the tool creates a temporary directory under /tmp , extracts sections of the executable to this temporary directory and executes the sections.

• Sufficient disk space for the tool and the log files (current usage is approximately 10 MB)

• IP connectivity to CPs and UPs on which onboarding is to be done. Password-based SSH is used for connections.

Pre-processing step helps in understanding the status of the CUPS system where the onboarding operation is being performed. In the pre-processing stage, following checks are performed irrespective of the operation:

• Verify if all CP and UP management IPs are reachable:

  • Ping Active/Standby management IPs of all the CPs.

  • Ping Active/Standby management IPs of all the UPs.

• Collect the resources information (APN, IP Pools, VRF, and Context) based on the output of:

  • show ip user-plane verbose

  • show cups-resources session summary

• Add Operation:

  • On Control Plane node, following checks are performed:

    • Verifies that the VRF, APN, and IP pool to be onboarded is not configured in the system. If onboarding one or more virtual APNs, then the APNs that refer to these virtual APNs must be already present on the system. The tool uses the presence of the following configuration in the APN to distinguish these APNs.

      virtual-apn gcdr apn-name-to-be-included Gn

      So, given an input configuration with one or more APNs, then any APN that is already present in the system must include the preceding configuration. Otherwise, the tool assumes that the APN isn’t present and hence fails the preaudit step.

    • Verifies that there is no configuration difference between Active/Standby CPs using show srp info .

  • After context and UP Group selection, on User Plane node, the following pre-processing checks are performed on all the UPs of the selected UP Group:

    • Verifies that the VRF to be onboarded doesn't exist in the system. If it exists, then the pre-processing fails and onboarding is aborted.

    • Verifies that there is no configuration difference between Active/Standby UPs using show srp info .

    • Verifies if SGi context is mapped in the UP Groups.

• Modify Operation:

  • On Control Plane node, following checks are performed:

    • Verifies that the VRF to be modified exists in the system.

    • Verifies that the APN to be modified exists in the system.

    • Verifies that the IP pools, deleted as part of modify operation, exists in the system. Any IP pool that is added as part of modify operation, doesn't exist in the system.

    • Verifies that there is no configuration difference between Active/Standby CPs using show srp info .

• Delete Operation:

  • On Control Plane node, following checks are performed:

    • Verifies that the VRF to be deleted exists in the system.

    • Verifies that the APN(s) to be deleted exists in the system.

    • Verifies that there is no configuration difference between Active/Standby CPs using show srp info .

  • On User Plane node, following checks are performed:

    • Verifies that the VRF to be deleted exists in the system.

    • Verifies that there is no configuration difference between Active/Standby UPs using show srp info .

On successful pre-processing, the tool performs the Add/Modify/Delete operation as per the input and applies the configuration on CP and UP. For ICSR setups, the configuration is applied on both Active and Standby CP and UPs.

• Add operation: The algorithm chooses the right SGi context and UP group for the APN to be added.

  • On Control Plane node, following steps are performed:

    • The chosen SGi context and the UP Group are added to the APN configuration, which goes as input to the tool. In case of onboarding virtual APNs, only the onboard virtual APNs get updated with UP Group and IP context. The APNs that refer to them (which is already present in the system) just gets updated with any virtual-apn preference .. configuration that is present in the input file.

    • The updated configuration is then applied to the CP node.

  • On User Plane node:

    • The IOB tool replaces the dummy SGi context with chosen context, and applies the resulting configuration to all the UPs in the chosen UP Group.

    • Applies VRF configurations to all the UPs in the UP Group.

  • For any failure scenario, the IOB tool attempts to roll back to the previous configuration.

• Modify Operation: Configuration is modified to add or delete the IP pools.

  • On Control Plane node:

    • For the given APN configuration, IP pool configuration is modified to add/delete the IP pools. If any IP pools are deleted, then prior to deletion, the tool:

      • Busyouts the pool.

      • Clears existing subscribers for that pool per pace-out interval. The pace-out interval is calculated based on the size of the pool.

        For IPv6 pools, the formula is:

        Pace-out interval = (2^(64 - pool size) * 2 - 2) / 500

        So, a /48 pool will get a pace-out interval of (2^(64 - 48) * 2 - 2) / 500 = (2^16 * 2 - 2) / 500 = 131070 / 500 = 262 seconds

        For IPv4 pools, the formula is:

        Pace-out interval = (2^(32 - pool size) * 2 - 2) / 500

        So, a /21 pool will get a pace-out interval of (2^(32 - 21) * 2 - 2) / 500 = (2^11 * 2 - 2) / 500 = 4094 / 500 = 8 seconds

  • For any failure scenario, the IOB tool attempts to roll back to the previous configuration.

• Delete Operation: Deletes the APN.

  • On Control Plane node:

    • IP pools and VRFs, associated with the APN, are deleted.

      Prior to deleting any APN, the IOB tool verifies if any user is attached to the given APN. If any user exists, it exits from the tool and displays an error message "Please clear the subscribers then run the DELETE_ENTERPRISE else it will delete the APN".

    • APN configuration is deleted.

    • Deleting the virtual APNs removes only the virtual APNs and references to the virtual APNs. The APNs that refer to them are expected to remain in the system. Otherwise, post audit will fail.

  • On User Plane node, VRF configurations are deleted.

    The IOB tool doesn't rollback to the previous configuration on a failure. It, however, tries to delete as much of the relevant configuration as possible to minimize the amount of manual clean-up required.

After the configurations are pushed to CP and UP, checks are performed to validate configuration changes.

• Add Operation:

  • On Control Plane node, following checks are performed:

    • Verifies configured VRF with show ip vrf vrf_name : To verify if the VRF configuration is applied in the CUPS system.

    • Verifies that the chosen context is shown with show configuration apn apn_name : To verify if the context has been associated with the APN that is added. This verification takes place for each APN that is onboard. If there are virtual APNs onboard, then this verification takes place only for each virtual APN.

    • Verifies that the chosen UP Group is shown under show configuration apn apn_name : To verify if the UP Group has been associated with the APN that is added.

    • If there are virtual APNs onboard, then the tool verifies that all the references to the virtual APNs from other APNs as per the input configuration (virtual-apn preference <preference> apn <virtual apn> and so on) are present and correct.

    • Saves configuration using save configuration file_path / file_name : After successful addition of new enterprise, checks if the respective configuration files are stored in the given path as mentioned in "CUPSinfo.txt" file.

    • Synchronize configuration on CPs with filesystem synchronize : After successful addition of new enterprise, verifies the file synchronization.

    • Verifies that there is no configuration difference between CPs using show srp info : SRP validation in ICSR setup: After successful addition of new enterprise, the IOB tool checks for SRP validation with "Primary" and "secondary" status, "Last Peer Configuration Error", "Connection State", along with "Number of Sessmgrs".

  • On User Plane node, following checks are performed:

    • Verifies configured VRF with show ip vrf vrf_name : To verify the VRF configuration applied in the CUPS system.

    • Verifies Route Distinguisher using show ip vrf vrf_name : To verify the Route Distinguisher configuration applied in the CUPS system.

    • Save configuration using save configuration file_path / file_name .

    • Invoking SRP validation using srp validate-configuration : Verifies that there is no configuration difference between UPs using show srp info : SRP validation in ICSR setup.

  • For any failure scenario, the IOB tool attempts to roll back to the previous configuration.

• Modify Operation:

  • On Control Plane node, following checks are performed:

    • Verifies that the modified changes are applied to the CUPS system.

    • Verifies that the changes to IP pool are reflected in the system

    • Saves configuration using save configuration file_path / file_name .

    • Invoking SRP validation using srp validate-configuration : Verifies that there is no configuration difference between UPs using show srp info : SRP validation in ICSR setup

  • For any failure scenario, the IOB tool attempts to roll back to the previous configuration.

• Delete Operation:

  • On Control Plane node, following checks are performed:

    • This verification takes place for each APN after the delete operation. The delete operation for the virtual APN removes only the references to the virtual APN and retains the APNs that refer to them. Removal of the latter APNs causes the post processing to fail.

    • Verifies if the VRF configuration is deleted from the CUPS system.

    • Saves configuration using save configuration file_path / file_name.

    • Invoking SRP validation using srp validate-configuration : Verifies that there is no configuration difference between UPs using show srp info : SRP validation in ICSR setup

  • On User Plane node, following checks are performed:

    • Verifies if the VRF configuration is deleted from the CUPS system.

    • Saves configuration using save configuration file_path / file_name.

    • Invoking SRP validation using srp validate-configuration : Verifies that there is no configuration difference between UPs using show srp info : SRP validation in ICSR setup.

The Add operation configures a new APN for the enterprise customer. The tool also supports onboarding multiple APNs in one operation, provided they share the same SGi context and VRF configuration. In this case, the onboarding APNs may or may not share the IP pool information (supports both the conditions). All the onboarding APNs map to the same SGi context and UP group in the preceding scenario. The algorithm chooses the right SGi context and UP Group, and maps them to the APN by taking the system parameters into consideration.

Algorithm Logic:

• Check for System Limits (done against CP limits mentioned in System Limits). In case of onboarding virtual APNs, the tool only considers the virtual APNs as new APNs for APN limit calculation. The APNs referring to the virtual APNs are already present in the system and are hence already included in the current count of APNs in the system.

• Rank UP Groups based on number of APNs configured with low numbers on top.

• Sort the SGi contexts based on the number of VRFs configured in ascending order.

• Exclude VIP UP Groups and Contexts from the list.

• Pick a UP Group from top of the list (least-used):

  • Get a Context that is mapped to UP Group (if no Contexts are mapped, pick from the sorted list).

  • Check the number of VRFs, IPv4, IPv6 pools, and total pool size.

  • Choose the Context if checks fall within thresholds; else, repeat for next Context.

  • Pick suitable Context within limits; if none found, exit algorithm.

  • For this UP Group, iterate through UPs and check total IP pool limits.

  • If successful, choose the UP Group and Context.

• Iterate through all UP Groups.

• At each step, while checking against the thresholds, print error messages.

• Prepare the configuration with chosen Context and UP Group and apply.

Modify operation allows the onboarded Enterprise customer to increase/decrease the subscribers by Adding more IP pools or deleting the existing IP pools.

Delete operation removes a previously onboarded Enterprise. During this operation, the IOB tool cleans up the IP pools, VRFs, and APNs that are used for the Enterprise.

To delete an Enterprise, the following procedure must be followed as there may be active subscribers on the system:

• Busyout the IP Pools: This is performed to block the new subscribers. Invoke IOB tool and perform the Busyout operation using the MODIFY operation.

• Clear Subscribers: The Provisioning tool clears the active subscribers.

• Delete the Enterprise: Invoke the IOB tool and perform the enterprise removal using the DELETE operation.

The IOB tool expects passwords in the "CUPSInfo.txt" input file to be RSA encrypted and converted to base64 format. Encryption is done using OpenSSL (currently, version 1.0.2.k is supported) commands and RSA public key. The IOB tool must be provided the path to the corresponding RSA private key so that it can decrypt the passwords. The decrypted passwords are stored only in the IOB tool's RAM. The detailed steps for encryption and decryption are described below:

1. Verify that the OpenSSL with the correct version is installed on the target machine:

   • "openssl version" should indicate that the version is 1.0.2.k-fips.

2. Generate RSA private and public key pair:

   1. RSA private key:

      
      openssl genpkey -algorithm RSA -out private_key.pem -pkeyopt rsa_keygen_bits:4096 
      

      Where:

      • "private_key.pem" represents the generated private key file in PEM format. This is used for decryption and has to be stored securely.

      • 4096 is the key length in bits. Either 2048 or 4096 can be used. Multiple passwords may need to be encrypted and so, 4096 is recommended. Generally, the larger the key size, the larger the size of data it can encrypt. However, it also takes longer to encrypt/decrypt.

   2. RSA public key:

      
      openssl rsa -pubout -in private_key.pem -out public_key.pem 
      

      Where:

      • "private_key.pem" is the private key generated in Step (a).

      • "public_key.pem" is the file that contains the corresponding public key.

3. For each password that needs to be encrypted, do the following:

   1. Type the password in plaintext in a text file using an editor. Don't hit enter at the end of the line. It should have just the password in a single line. In this example, the file is named as "pp1".

   2. Execute:

      
      openssl pkeyutl -encrypt -inkey public_key.pem -pubin -in pp1 -out encrypted_pp1 
      

      Where:

      • "public_key.pem" is the public key generated in Step 2b.

      • "pp1" is the file containing the single password in plaintext.

      • "encrypted_pp1" contains the password in encrypted form.

      Delete "pp1" created in Step 3a to avoid accidental exposure.

   3. "encrypted_pp1" contains the key in raw binary form. Convert it to base64 as follows:

      
      base64 encrypted_pp1 
      

   4. The above command (Step 3c) will output the base64 encoded encrypted password to the terminal. Copy and paste this into the "CUPSinfo.txt" file that contains the credentials supplied to the IOB tool. While copying, make sure to remove any line breaks or spaces. The entire password should be a single line.

   5. "encrypted_pp1" can be deleted at this point.

Note	Step 3 must be performed for each password, one at a time, using the same public key/private key pair for all the passwords.

After "CUPSinfo.txt" file is updated with all the encrypted base64 passwords, the IOB tool is ready to be run. When running the script, specify an additional parameter: -k absolute path to private_key.pem created in Step 2a> .

The application is compiled to create a standalone .exe. The application can be run on a RedHat Enterprise Linux machine.

The Onboarding Application can be run with below syntax:

./intelligent_onboarding -o <OP_Type_Parameter_File> -i <CUPS_Info_File> -k 
<Path_to_Pvt_Key_file> [ -l <Path_to_store_logfiles> ] [ -p ] [ --context_selection_from_cp ] [ -v ]

Options:

• -o: [Mandatory] Provide the input parameter file specific to the operation being invoked.

  After successful onboarding, the IOB tool deletes the file.

• -i: [Mandatory] This option is used for "CUPSinfo.txt" file which has the details of CUPS system.

• -k: [Mandatory] Absolute path to the private key file. The tool uses this to decrypt the previously encrypted passwords. This private key file must correspond to the public key that is used to encrypt the passwords.

• -p: [Optional] When included, few pre-audit and post-audit checks are bypassed to reduce the time taken for Add/Modify/Delete operation.

• -l: [Optional] Provide absolute path to store the logs.

  When this keyword is not specified, the log files are created in the directory from which the IOB tool is invoked.

• --context_selection_from_cp: [Optional] When specified, the tool bases its context selection solely on the list of contexts available on the CP. The tool assumes that the selected context is also available on the UPs and does not validate this. This is an optimization. The default behavior is to examine contexts configured on CP and UP and select from contexts common to both.

• -v: [Optional] Displays the version of the IOB executable.

  If IOB tool is executed without the -v option, the version is displayed that is similar to:

        ###############################################################
        #                                                             #
        #             WELCOME TO ENTERPRISE ONBOARDING                #
        #                   Version 21.20.9.private                   #
        #                                                             #
        ###############################################################

  NOTE: The version is displayed in the log file and terminal output as well.

The following table depicts the maximum limits on ASR 5500 and CUPS.

Note

The IOB tool allows onboarding (OpType: ADD_ENTERPRISE) multiple APNs provided all those APNs share the "CP_SGi_Context" and "UP_VRF_Config" section of the input file. The APNs may potentially use different IP pool groups, but all those pool groups must be present in a single context in the "CP_SGi_Context" section of the input file. Also, the APNs must share a single VRF. In such a case all those APNs onboard to the same UP group and SGi context. The tool supports deletion (OpType: DELETE_ENTERPRISE) of multiple APNs provided that all APNs share the "UP_VRF_Config" and "CP_SGi_Context" sections of the input configuration. The tool deletes the VRF and pools at the end of the operation. The intended use case for multiple APN deletes is to delete APNs that were onboarded together. The APNs onboarded together must be deleted together - the tool does not support separate deletion of APNs that were onboarded together. The tool does not support modification (OpType: MODIFY_ENTERPRISE) of multiple APNs in one operation. Only one APN can be modified at a time. The CUPSinfo.txt file is considered as the primary UP information. If any UP Groups are added in the system, but are not present in the file, then they are excluded from onboarding.

The following list of error codes is available in support of Enterprise Onboarding in CUPS feature.