Data warehousing Testing

PadmaOctober 28, 2023

0 192 3 minutes read

Data warehousing Testing is significant for data warehouse systems that will make to work correctly and also efficiently. There are three levels of testing performed on the data warehouse.

They are:

– Unit testing

– Integration testing

– System testing

Table of Contents

. Unit Testing

In unit testing, every component will be individually tested. Each module that is procedure, program, SQL Script, UNIX shell will be tested. This test is performed by the developer.

.Integration Testing

In this integration testing, the various modules of the application are brought together than tested for the inputs. It will be performed to test where the various components will be done well after integration.

.System testing

This system testing the entire data warehouse application will be tested together. The main intention of system testing is to check whether the application in whole works correctly together. System testing is done by the testing team. The size of the entire data warehouse will be very large, it will be very usually possible to do minimal system testing before the test plan that will be done.

Testing the database:

The Database will be tested in three ways:

Testing the database manager and monitoring tools- These tools will include to test the database manager and the monitoring tools that they should be used in the creation, running and the management of test database.

Testing database feature:

The list of feature we have to test by:

Querying in parallel

Creating the index

Data load in parallel

Testing database performance:

This query execution will play a important role in the data warehouse performance measures. There will be sets of fixed queries that need to be run regularly and they will be tested. To test queries one will go through the user requirement document and understand the business completely. It always takes the time to test the most awkward queries that the business will be likely to ask against many index and aggregation strategies.

The future scenarios of data warehousing:

When the size of the open database will be grown approximately double of its magnitude in the previous years, it will show the significant value that it contains. As the size of the database will grow this estimates of what it will constitute a very large database that follows to grow. The hardware and software that is available at present will not allow keeping a large data online. For example, a Telco call may record will require 10TB of data to be kept online which will of size of one month’s record. This record will contain textual information and may be multimedia data. Multimedia data will not be easily manipulated as text data. This searching the multimedia data wills not an easy task, whereas textual information can be retrieved by the relational software available today. While it will be complex to build and run the data warehouse systems that are increasing in size. The prescribed size of the data warehouse is in increasing order. These users will also require accessing the system. Along with the growth of the internet there will be a requirement of users to access the data online. Therefore the data warehouse is very typical from what is created today.

Goals of Data warehouse testing:

1. Data completeness-

We can ensure that all the data from many sources will be loaded into a data warehouse. The testing team may validate if all the data warehouse records are loaded against the source database and also flat files by the following below samples

The total number of records that are counted, uploaded from the source system that should match the total records loaded into Data warehouse.
By comparing the data loaded into each field of data warehouse with the source system data fields.

2. Data transformation-

When we are uploading the source data to the data warehouse. There are few fields that will be directly be loaded with the source data but there are few fields that will be loaded with the transformed as per the business logic.