Software Testing
Mainframe Testing - Complete Tutorial
Before learning mainframe testing concepts, lets learn What is a Mainframe? The mainframe is a...
What is SOA Testing? Tutorial with Example
- Details
What is SOA Testing?
SOA (Service Oriented Architecture) Testing is a Testing of SOA architectural style in which the application components are designed to communicate via communication protocols typically over a network.
In this tutorial, you will learn-
- What is SOA?
- What is Service?
- SOA Testing
- Strategy for SOA Testing
- SOA testing methods
- Challenges in SOA testing
- SOA Testing Tools
- SOA Testing Use Cases
What is SOA?
SOA is a method of integrating business applications and processes together so as to meet the business needs.
In Software Engineering, SOA provides agility and flexibility to business processes. The changes to the process or application can be directed to a particular component without affecting the whole system.
The software developers in SOA either develop or buy chunks of programs called SERVICES.
What is Service?
- Services can be a functional unit of application or business process, which can be reused or repeated by any other application or process.
(For example, in the above image, Payment Gateway is a service which can be reused by any e-commerce site. Whenever a payment needs to do, the e-commerce site calls/Requests the Payment Gateway service. After payment is done on a gateway, a response is sent to the e-commerce website)
- Services are easy to assemble and easy to reconfigure components.
- Services can be compared to building blocks. They can construct any application needed. Adding and removing them from the application or business process is easy.
- Services are defined more by the business function they perform rather than as chunks of code.
Web Services
Web services are independent application components, which are available over the web.
They can be published, found and can be used on the web. They can communicate through the internet.
- The Service Provider publishes the service to the internet.
- The Client searches for a particular web service from the Web Service Registry
- A URL and the WSDL for the required web service is returned.
>> Using the WSDL and the URL, the communication between the service provider and the requestor happens through SOAP messages. <<
- When a consumer calls a web service, an HTTP connection will be established to the provider.
A SOAP message is created to instruct the provider for invoking the required web service logic.
- The response received from the provider is a SOAP message which will be embedded into the HTTP response. This HTTP response is the data format which is understandable by the consumer application.
Example
A home page of a Website and a Search Engine displays everyday weather report. Instead of coding the weather report section all over, a Service of weather report can be bought from a vendor and integrated into the pages.
SOA Testing
SOA consists of various technologies. Applications built using SOA has various services which are loosely coupled.
SOA Testing should focus on 3 system layers
Services Layer
This layer consists of the services, services exposed by a system derived from business functions.
For example –
Consider a Wellness Website which consists of
- Weight Tracker
- Blood Sugar Tracker
- Blood Pressure Tracker
Trackers display the respective data and date they are entered. Services layer consists of the services which gets the respective data from the Database–
- Weight Tracker service
- Blood Sugar Tracker service
- Blood Pressure Tracker service
- Login Service
Process Layer
Process Layer consists the processes, collection of services which are part of a single functionality.
The processes might be a part of user interface (for ex – A search engine), a part of an ETL tool (for getting data from the database).
The main focus in this layer will be in user interfaces and process.
The user interface of the weight tracker and its integration with the Database is the primary focus.
Below functions will be of consideration
- Adding new data
- Editing existing data
- Creating new tracker
- Deleting data
Consumer Layer
This layer mainly comprises of user interfaces.
Based on the layer, the testing of an SOA application is distributed into three levels.
- Service level
- Interface level
- End to End level
- Top Down approach is used for Test Designing.
- Bottom Up approach is used for Test Execution.
Strategy for SOA Testing
Test Planning Approach,
- The complete architecture of the application should be understood by the SOA Testers.
- The application needs to be broken down into independent services (Service, which has their own request and response structure and does not depend on any other service to form response).
- The application structure needs to be reorganized into three components – Data, Services, and front-end applications.
- All the components need to be carefully analyzed, and business scenarios should be chalked out.
- The business scenarios should be classified as common scenarios and application specific scenarios.
- A Traceability Matrix should be prepared, and all test cases should be traced to business scenarios.
Test Execution Approach
- Each service component should be tested.
- Integration Testing of the service components should be done to validate the data flow through the services and data integrity.
- System Testing of the complete model should be done to validate the data flow between front-end application and database.
- Performance Testing should be done for fine tuning and optimum performance.
SOA Testing Methods
1) Business scenario driven data based testing,
- Various business aspects related to the system should be analyzed.
- Scenarios should be developed based on the integration of
- Various Web services of the application
- Web services and application.
- Data set up should be done based on the above scenarios.
- Data set up should be done so as to cover end to end scenarios as well.
2) Stubs
- Dummy interfaces will be created to test services.
- Various inputs can be provided through these interfaces, and the outputs can be validated.
- When an application uses an interface to an external service, which is not under test (third party service), a stub can be created during Integration Testing.
3) Regression testing
- Regression Testing on the application should be done when there are multiple releases so as to ensure the stability and availability of the systems.
- A comprehensive regression test suite will be created covering the services which form an important part of the application.
- This test suite can be reused in multiple releases of the project.
4) Service Level Testing
Service Level Testing includes testing the component for functionality, security, performance and interoperability.
Each and Every Service needs to be first tested independently.
5) Functional Testing
Functional Testing should be done on each service to
- Ensure that service delivers the right response to each request.
- Right errors are received for requests with Invalid data, bad data, etc.
- Check for each request and response for each and every operation the service has to perform in run time.
- Validate the fault messages when an error occurs at the server, client or network level.
- Validate that the responses received are in the right format.
- Validate that the data received on the response corresponding to the data requested.
6) Security Testing
Security testing of the web service is an important aspect during service level testing of the SOA application; this ensures the safety of the application.
The following factors need to be covered during testing:
- Industry Standard defined by WS-Security testing should be abided by the Web Service.
- Security measures should work flawlessly.
- Encryption of data and Digital signatures on the documents
- Authentication and Authorization
- SQL Injection, Malware, XSS, CSRF, other vulnerabilities are to be tested on the XML.
- Denial of Service attacks
7) Performance Testing
Performance Testing of the service needs to be done since the services are reusable and multiple applications might be using the same service.
The following factors are considered during testing:
- 8) Performance and functionality of the service need to be tested under heavy load.
- The performance of the service needs to be compared while working individually and within the application, it is coupled with.
- Load testing of service should be performed
- to verify response time
- to check for bottlenecks
- to verify the utilization of CPU and memory
- to predict scalability
9) Integration level testing
- Service level testing ensures proper working of only the services individually, it does not guarantee the working of the coupled components.
- Integration Testing is done focusing mainly on the interfaces.
- This phase covers all the possible business scenarios.
- The Non-Functional testing of the application should be done one more time in this phase. Security, compliance, and Performance Testing ensure the availability and stability of the system in all aspects.
- The communication and network protocols should be tested to validate the consistency of the data communication between the services.
10) End to End testing
This phase ensures that the application confirms to the business requirements both functionally and non-functionally.
The below items are ensured to be tested during the end to end testing
- All services working as expected after integration
- Exception handling
- User Interface of the application
- Proper data flow through all the components
- Business process
Challenges in SOA testing
- Lack of interfaces for Services
- Testing process spans across multiple systems thus creating complex data needs
- The application is a collection of various components which tends to change. The need for Regression Testing is more frequent.
- Due to Multilayer architecture, it is difficult to isolate defects.
- Since the service will be used in different interfaces, it is difficult predicting load, hence making performance test planning cumbersome.
- SOA is a collection of heterogeneous technologies. Testing of an SOA application requires people with different skill sets which in turn increase the planning and execution costs.
- Since the application is an integration of multiple services, security testing has its own share of woes. Validation of authentication and authorization is pretty much difficult.
SOA Testing Tools
There are many SOA testing tools available in the market to help testers in testing SOA applications. Here are some of the popular SOA Testing Tools:
1) SOAP UI
"SOAP UI" is an open source Functional testing tool for Services and API Testing.
- Desktop application
- Supports multiple protocols – SOAP, REST, HTTP, JMS, AMF, JDBC
- Web services can be developed, inspected and invoked.
- Can also use for load testing, Automation Testing, and security testing
- Stubs can be created by MockServices
- Web Service requests and tests can be generated automatically through its web service client.
- Have inbuilt reporting tools
- Developed by SmartBear
2) iTKO LISA
"LISA" is a product suite which provides a functional testing solution for distributed systems like SOA.
- Can also use for regression, integration, load and Performance Testing.
- Developed by iTKO (CA Technologies)
- Can be used to design and execute tests.
3) HP Service Test
"Service Test" is a functional testing tool, which supports both UI and shared services testing
- Both functional and performance test of services can be done by a single script.
- Integrated with HP QC.
- The massive amount of service and data can be managed.
- Supports interoperability testing by simulating JEE, AXIS, and DotNet client environments.
- Developed by HP.
4) Parasoft SOA Test
SOA Test is a testing and analysis tool suite developed for API and API applications testing.
- Supports Web Services, REST, JSON, MQ, JMS, TIBCO, HTTP, XML technologies.
- Functional, Unit, Integration, Regression, Security, Interoperability, Compliance and Performance Testing are possible.
- Stubs can be created using Parasoft Virtualize, which are intelligent than SOAP UI.
- Developed by ParaSoft
SOA Testing Use Cases
Consider an e-commerce Website, which contains the below functions and Sub-functions:
Order Processing
PHASE 1
In the first phase of SOA testing i.e. Test Strategy Phase, the application is broken into Services and business functions.
Let us consider below are the Services in the application.
- Create Order
- Check Customer Status
- Change Order Status
- Check Order Status
- Check Inventory
Business functions being the same as the functions of the Website.
Note: The test strategy document would contain the list of the service and the functions which have to be tested.
PHASE 2
Test Planning phase. Test cases are written for each level.
- End to End level. The test cases are written for each business use case and flow.
Below are the example of test cases
- Create an order with the active user.
- Create an order with an inactive user.
- Create an order with the available product with order quantity < available quantity.
- Create an order with the available product with order quantity > available quantity.
- Create an order with multiple items
- Cancel an order completely.
- Cancel order partially.
- Integration level. Test cases are written for integration of database and user interface.
Below are example test cases.
- Create a new order with a single item. Verify that the order is created on the database.
- Create a new order with a single item. Verify that the price calculated for the order is correct.
- Create a new order with a single item. Verify that the quantity of the available product is less by the order amount.
- Verify that the status of the order displayed on the UI is same as that on the database.
- Cancel the order and verify that the status of the order is modified on the database.
- For the first time payment, verify that the payment details entered on the UI is saved on the database.
- For returning payments, verify that the payment details on the database are displayed on the UI.
- Service level. Each service is tested for all the data conditions.
Below are a few examples.
| No. | Order Details | Order Condition |
|---|---|---|
| 1 | Create Order. No. of Items = 1 | Quantity on order < Quantity on database |
| 2 | Create Order. No. of Items > 1 | Quantity on Order < Quantity on database. |
| 3 | Create Order No. of Items = 1 | Quantity on Order > Quantity on database |
| 4 | Check Order status | Status on database = Active |
| 5 | Check Order status | Status on database = Shipped |
| 6 | Check Order status | Status on database = Cancelled |
| 7 | Check Order status | Order id = Invalid |
| 8 | Check product availability | Quantity of product >0 |
| 9 | Check product availability | Quantity of product =0 |
| 10 | Check product availability | Product id = invalid |
PHASE 3 – Test Execution
Test Execution uses bottom-up approach i.e. service level testing is done first, then Integration level and at last End to End testing.
1) Service level
Let us consider that Soapui tool is considered for testing the application.
The WSDL and URL are browsed into the test window of SOAP.
The request for each service will be displayed on the request window.
By modifying the data as per the service level test cases, requests are created for each test case.
Test Case | Request | Expected response |
|---|---|---|
Create Order. No. of Items = 1Quantity on order < Quantity on db | <productid>x2</productid><quantity>2</quantity> | <orderid>o3251</orderid><message>Successful</message> |
Create Order.No. of Items > 1Quantity on Order < Quantity on db | <productid>y1</productid><quantity>1</quantity><productid>y2</productid><quantity>3</quantity> | <orderid>o3251</orderid><message>Successful</message> |
Create OrderNo. of Items = 1Quantity on Order > Quantity on db | <productid>x23</productid><quantity>200</quantity> | <orderid>null</orderid><message>Unsuccessful</message> |
Check Order statusStatus on database = Active | <orderid>o9876</orderid> | <status>Active</active><message>Successful</message> |
Check Order statusStatus on database = Shipped | <orderid>o9656</orderid> | <status>Shipped</active><message>Successful</message> |
Check Order statusOrder id = Invalid | <orderid>y5686</orderid> | <status>null</active><message>Unsuccessful</message> |
Check product availabilityQuantity of product >0 | <productid>d34</productid> | <quantity>34</quantity><available>yes</available><message>Successful</message> |
Check product availabilityQuantity of product =0 | <productid>y34</productid> | <quantity>0</quantity><available>no</available><message>Successful</message> |
Check product availabilityProduct id = invalid | <productid>sder</productid> | <message>Unsuccessful</message> |
2) Integration Level
The integration level test cases are executed on the User interface and database.
- Create an order with a single item –
- A user opens the website.
- Goes to place an order.
- Selects a valid product and quantity and saves the order.
- A message saying the Order is placed successfully should be displayed.
- A user opens database and checks if the details of the order are same as that entered on the website.
3) End to End level
The business flows and use cases are executed on the User interface.
- Create an order with multiple items –
- A user opens a website.
- Goes to place an order.
- Enquires about a valid product and quantity adds them to cart.
- Other valid products are added with valid quantities and the order is saved. Payment is done through a new payment method and order is placed.
- A message saying "Order placed successfully" should be displayed.
- A tester should validate that the whole flow is done without skewing of data.
Conclusion:
By sketching the right strategy for testing, resources, tools, and compliance to provide good service, SOA testing can deliver completely and perfectly tested application.
