NFVO Management Function Validation Test Suite

Virtualized Network Functions (VNF) are managed and orchestrated by the NFV MANO through the NFVI. It allows a VNF to be instantiation, management, scaled up or down and in or out, and terminated when no longer required.

The efficiency of VNF life cycle depends on how successfully VNFs of different types are created and managed. For instance, a Network Operator (ISP) often receive upgrades, bug fixes and enhancements for any of the different NFV components.  Even a tinny modification can cause a major instability issue when not detected at early stage by the System Test and validation with a thorough test suite.

Validation of the VNF Lifecycle Management encompasses all major NFV components like NFVI, VNF Manager, VIM, and the VNF function.

When validating a fully populated NFV system, the customer should also consider validating each component separately. Some of them are listed below:

• Validating the NFVI and VIM to ensures a stable and high performance infrastructure
• NFVI Bench marking for performance and fault management
• VNF validation to focus on actual function or service that a VNF provides
• MANO validation focuses on the feature set and scalability aspect of the NFVO and VNF Manager.
• VNF Lifecycle management for a system wide

In this post we will be discussing some differentiate test cases for NFV Orchestration (NFVO) Management Function Validation which includes following categories.

Test Category

• Software Image Management
• VNF Package Management
• VNF Lifecycle Management
• Fault Management
• Performance Management
• NS Lifecycle Management

Software Image Management

NFVO architecture supports the capability to distribute the software images to one or more VIMs. These software images are stored in a image repository and different management operations likes adding new Image, deleting existing image, retrieving information of existing image, updating mata data information shall be verified as part of validation for this.

Following test cases are some of the examples for Software Image management.

• • Test to verify that the NFVO can add a software image to the image repository managed by the VIM
  • Test to verify that the NFVO can retrieve the information of a software image from the image repository managed by the VIM
  • Test to verify that the VNFM can retrieve the information of a software image from the image repository managed by the VIM
  • Test to verify that the NFVO can update the metadata of a software image in the image repository managed by the VIM
  • Test to verify that the NFVO can delete a software image from the image repository managed by the VIM

VNF Package Management

A VNF Package is a tar-archive that contains all the information required for managing the lifecycle of a VNF. These VNF packages are kept within VNF catalogs. A VNF catalog is basically a collection of VNFs with a template to determines the type of VNF managed. The catalog includes deployment specifications, KPIs, and recipes for VNF management functions applicable to the VNF type. NFVO can perform operation like On-board/delete, Enable/Disable, Query, Fetch, Subscribe/Notify changes as part of VNF package management.

Following test cases are some of the examples for VNF package management.

• • Test to verify that a VNF Package can be successfully on-boarded to the VNF catalogue managed by the NFVO
  • Test to verify that an on-boarded VNF Package can be successfully deleted from the VNF catalogue managed by the NFVO
  • Test to verify that an on-boarded VNF Package that has been in deletion pending state can be successfully aborted the deletion by the NFVO

VNF Lifecycle Management

VNF Lifecycle management includes procedures like VNF instantiation, querying VNF instance information, healing or recovery, scaling in/out and termination.

Following test cases are some of the examples for VNF Life cycle management.

• • Test to verify that a VNF be successfully instantiated when a “instantiate VNF” operation is triggered by the EM
  • Test to verify that the VNF instance’s information can be queried successfully by the EM
  • Test to verify that the VNF information can be updated successfully by the EM
  • Test to verify that a VNF/VNFC instance be successfully started when an “operate VNF” operation is triggered by the EM
  • Test to verify that a VNF/VNFC instance be successfully stopped when an “operate VNF” operation is triggered by the EM
  • Test to verify that a VNF can be successfully healed when VNF healing request is sent out by the EM/VNF
  • Test to verify that a VNF can be successfully scaled out by adding VNFC instances triggered by a EM/VNF request
  • Test to verify that a VNF can be successfully scaled in by removing VNFC instances triggered by a EM/VNF request
  • Test to verify that a VNF be successfully terminated when a “terminate VNF” operation is triggered by the EM

Fault Management

The fault management includes the generation and sending alarms for hardware (Storage, compute, networking) and software (application, VNFs, OS) failures notification and event in NFVI environment and clearing them after recovery or healing.

Following test cases are some of the examples for fault management.

• • Test to verify that a fault alarm notification propagates to the NFVO when a virtualised resource that is required for the NS connectivity fails
  • Test to verify that a fault clearance notification propagates to the NFVO when a failed virtualised resource that is required for the NS connectivity is recovered
  • Test to verify that a VNF fault alarm notification propagates via the VNFM to the NFVO when a VNF fault is triggered by a failed virtualised resource
  • Test to verify that a VNF fault alarm clearance notification propagates via the VNFM to the NFVO when a VNF fault is cleared by resolving a failed virtualised resource

Performance Management 

Performance Management includes monitoring the network services (NS) and VNF instances performance metrics using scheduled Job and notifications based on defined thresholds.

Following test cases are some of the examples for Performance management.

• • Test to verify that the performance metrics of a virtualised resource that is required for a NS instance connectivity can be monitored using PM jobs and notifications
  • Test to verify that the performance metrics of a virtualised resource that is required for a NS instance connectivity can be monitored using PM jobs and thresholds
  • Test to verify that the monitoring of performance metrics of a virtualised resource that is required for a NS instance connectivity can be stopped by deleting PM jobs
  • Test to verify that a threshold created for a virtualised resource that is required for a NS instance connectivity can be deleted
  • Test to verify that the performance metrics of a virtualised resource that is allocated to a VNF instance inside a NS instance can be monitored using PM jobs and notifications
  • Test to verify that the performance metrics of a virtualised resource that is allocated to a VNF instance inside a NS instance can be monitored using PM jobs and thresholds
  • Test to verify that the monitoring of performance metrics of a virtualised resource that is allocated to a VNF instance inside a NS instance can be stopped by deleting PM jobs
  • Test to verify that a threshold created for a virtualised resource that is allocated to a VNF instance inside a NS instance can be deleted

NS Lifecycle Management

A Network Service (NS) is a composition of Network Functions (VNF) and defined by its functional and behavioral specification. Life cycle management of a Network service includes instantiation, Scaling in/out, updates, healing in case of failure and termination.

Following test cases are some of the examples for Performance management.

• Network Service Instantiation
  • Test to verify that a standalone NS can be successfully instantiated
  • Test to verify that a NS referencing an existing nested NS can be successfully instantiated
• Network Service Scaling Out
  • Test to verify that the NS can be successfully scaled out by adding VNF instances triggered by an operator action
  • Test to verify that the NS can be successfully scaled out by adding VNF instances triggered automatically by a VNF indicator
  • Test to verify that the NS can be successfully scaled out by adding VNF instances triggered automatically by a VIM KPI
  • Test to verify that a VNF in a NS can be successfully scaled out by adding VNFC instances when triggered by a NFVO operator
  • Test to verify that a VNF in a NS can be successfully scaled out by adding VNFC instances when triggered automatically by a VNF indicator
  • Test to verify that a VNF in a NS can be successfully scaled out by adding VNFC instances when triggered automatically by a VIM KPI
• Network Service (NS) Scaling IN 
  • Test to verify that the NS can be successfully scaled in by removing VNF instances triggered by an operator action
  • Test to verify that the NS can be successfully scaled in by removing VNF instances triggered automatically by a VNF indicator
  • Test to verify that the NS can be successfully scaled in by removing VNF instances triggered automatically by a VIM KPI
  • Test to verify that a VNF in a NS can be successfully scaled in by removing VNFC instances from an existing VNF triggered by an operator action
  • Test to verify that a VNF in a NS can be successfully scaled in by removing VNFC instances triggered automatically by a VNF indicator
  • Test to verify that a VNF in a NS can be successfully scaled in by removing VNFC instances triggered automatically by a VIM KPI
• Network Service (NS) Update
  • Test to verify the capability to start a VNF instance inside a NS instance
  • Test to verify the capability to stop a VNF instance inside a NS instance
  • Test to verify that one or more VNFs can be instantiated and the instances added to a running NS instance
  • Test to verify that one or more VNF instances can be removed from a running NS instance
  • Test to verify that one or more shared VNF instances can be added to a running NS instance
  • Test to verify that one or more shared VNF instances can be removed from a running NS instance
  • Test to verify that one or more virtual links (VL) can be added to a running NS instance
  • Test to verify that one or more virtual links (VL) can be removed from a running NS instance
  • Test to verify that the deployment flavour of one or more VNF instances in a NS instance can be changed
• Network Service (NS) Healing and Termination
  • Test to verify that VNF instances inside the NS can be successfully healed when partial NS healing (VNF healing) is triggered by an operator action
  • Test to verify that a NS can be successfully healed when complete NS healing is triggered by an operator action
  • Test to verify that a standalone NS instance can be successfully terminated
  • Test to verify that a NS instance referencing an existing nested NS can be successfully terminated

References:

• ETSI GR NFV-TST 007: Network Functions Virtualisation (NFV); Testing; Guidelines on Interoperability Testing for MANO
• VNF Lifecycle Management 

Related Post:

• NFV Management and Orchestration [MANO] key and performance indicators [KPIs]
• Network Function Virtualization (NFV)
• Virtual Network Function (VNF) Definition, Architecture and Design
• Virtual Network’s Most Common Definitions
• Open Stack, its components and deployment models