QA Tutorials

DATA WAREHOUSE TESTING

SOFTWARE TESTING

The data warehouse testing is also called ETL testing.

Click here to read first part of article

The data warehouse testing or ETL testing includes the following techniques:

  1. Data transformation testing: Here the data from various sources are collected and verified to transform as per  the business rules .

2.  Data transfer count testing: counting the target of records loaded in the data warehouse sources should match with the expected count.

3. Data transfer testing: Verifying all the data which is collected is loaded properly in the data warehouse without any loss or truncating.

4. Data Quality testing: here the quality of the data is tested it makes sure that improper and invalid data is reported and replaced with proper default values.

5. Performance Testing: Here it verifies that the data is loaded in data warehouse within the prescribed time slots to improve its performance and scalability.

6. Production data testing: Validating or checking the data in production process against the data which is in sources.

7. Data Integration Testing: Verifying that all data from the sources are loaded properly and checked in each point and then transformed properly.

8. Software Migration Testing: In this testing it is made sure that the data from the data warehouse is working efficiently in the new environment or platform.

9. Data and Constraint case check: In this type of testing data type, length, constraints are checked.

10. Data integrity testing: here it is checked for any duplicate data in the target systems.

Database testing can be often confused with data warehouse testing. Database testing is done on smaller volumes of normalised data to validate the changes that affect the data from the software application. Data warehouse testing is performed on huge volumes of data that is not normalised.

Check your understanding:

1. Identify the possible challenges in data warehouse testing.

Facebook Comments
Tags

Related Articles

34 thoughts on “DATA WAREHOUSE TESTING”

  1. The challenges in data warehouse testing are:
    • Unavailability of inclusive test bed at times
    • Lack of proper flow of business information
    • Loss of data might be there during the ETL process
    • Existence of many ambiguous software requirements
    • Existence of apparent trouble acquiring and building test data
    • Production sample data is not a true representation of all possible business process
    • Incompatible & duplicate data.

  2. Challenges in data warehouse testing:
    .Loss of data (ETL, extract, load, transform)
    . Duplicate data
    .Lack of proper flow of business information
    .Difficulty extracting data from other source
    Difficulty of building test data
    .Unavailability of inclusive test bed at times
    .Volume and complex data

  3. The challenges in data warehouse testing
    -Loss of data
    -incorrect data
    -data not working efficiently in new environment
    -Unable to load /transform
    -duplication of data

  4. Challenges in data warehouse testing:
    Loss of data during ETL
    Duplicate data and incompatibility
    Data volume and complexity is huge
    Lack of proper flow of business information
    Errors occur while extracting data from different sources

  5. Challenges in data warehouse testing are:
    – Data loss during the ETL process.
    – Incorrect, Incomplete or duplicate data.
    – Due to huge volume of data and contains historical data, testing is complex.
    – Testers normally don’t have the access to see job schedules in the ETL tool.
    – Tough to build test cases because of high and complex volume.
    – Involves various complex sql concepts for data validation.
    – Unstable Testing environment delay the development and testing of a process.
    – Testers normally don’t have an idea of end-user report and business flow of information.

  6. Because data warehouse testing uses a huge volume of data, there are challenges that comes with it: Below are the sample challenges:
    Data loss
    duplicate data
    tough to generate and build test cases because volume is too high and complex
    unstable testing environment that will results to delay of development and testing

  7. Challenges in datawarehouse testing:
    – complexity of data
    – high volumes of data
    -duplicate data issues
    – loss of data
    – data should be able to work in new environments.

  8. Data warehouse testing has many challenges:
    – Dealing with large volumes of data
    – Counting the target records in the large databases
    – Time consuming
    – Data transfers errors, data loss may be frequent
    – Checking for duplicate data

  9. – Loss of data ETL, extract, load, transform
    – Difficulty of building test data
    – Incorrect data
    – Unable to load /transform
    – Data volume and perplexity is huge
    – Lack of proper flow of business information
    – Transferring complex Data Structure

  10. Possible challenges in data warehouse testing:
    Not possible to test real time data.
    Data will be Huge to visually identify the issues if any.
    Difficult to cover all test case senarios.
    Time consuming.
    Difficult to test on real time production environment.

  11. The few challenges in ETL or data warehouse testing are:
    Loss of data during transfer
    duplicating data
    Proper storage of large amounts of data
    Eliminating invalid data
    loading of data in prescribed time
    performance of transformed data in new environment

  12. The challenges in data warehouse testing are:
    -loss of data
    -incorect data
    -duplication of data
    -unable to load, transform
    -data not working efficiently
    -data volume and complexity.

  13. The main challenges of Data warehouse testing includes:
    • Data loss during testing.
    • Duplicate data and Incompatibility.
    • Lack of inclusive test bed.
    • Testers have no benefits to execute ETL jobs by their own.
    • Data volume and complexity is huge.
    • Inefficient in procedures and business process.
    • Inconvenience securing and building test data.
    • Absence of business course information.

  14. Identify the possible challenges in data warehouse testing.

    1. Data warehouse testing deals with large volumes of data from multiple system, a small change may have impact on multiple places and it may affect the project schedule.
    2.Since it deals with huge volumes of data complete testing may not be possible due to time and budget constraints.
    3.Due to security issues sufficient data may not be available from various systems for testing

  15. Lots of challenges are involved in data warehousing testing which are as follows:
    1.Errors occurs while extracting data from different sources.
    2.Loss of data collected and stored.
    3.Time consuming.
    4.Lack of flow of business information.
    5.Duplicate data.
    6.Inefficiently working of data from data warehouse in new environment.
    7.Huge volume of data.
    8.Budget and cost involved.

  16. Unavailability of inclusive test bed at times
    Lack of proper flow of business information
    Loss of data might be there during the ETL process
    Ambiguous software requirements
    Trouble acquiring and building test data
    Production sample data is not a true representative of all possible business processes
    Not having a good knowledge of SQL coding skills
    Certain testing strategies are time consuming

  17. Data warehousing testing is done with large volume of databases to assure that data that has been loaded from source to destination after business transformation is accurate. some challenges in this testing are:
    1. Performing data completeness checks for transformed columns is very tricky.
    2. Certain testing strategies used are time consuming.
    3. Unavailability of inclusive test bed at times.
    4. Lack of proper flow of business information.
    5. Loss of data might be there during ETL Process.
    6. Incorrect, incomplete or duplicate data.
    7. Unstable testing environment delay the development and testing of a process.
    8.sometimes the testers are not provided with the source to target mapping information.
    9. Tough to generate and build test cases, as data volume is too high and complex.
    10. Loss of data during theETL PROCESS.

  18. Some Challenges of ETL or Data Warehouse Testing are:
    – Ambiguous or duplicate data
    – Very large volumes of data
    – Tester privileges to perform data transfer
    – test data availability for the business function
    – missing business flow information
    – loss off data during ETL process

  19. Identify the possible challenges in Data Warehouse Testing.
    Loss of data during ETL
    Duplicate data and incompatibility
    Data volume and complexity is huge
    Lack of proper flow of business information
    Errors occur while extracting data from different sources

  20. Some of the challanges in data ware house testing:
    – verification of data loading for performance and scalability
    -Ensuring data working efficiency in new environment
    – Duplicacy of data
    – Loss of data during ETL process
    -time conumption
    – challanges in transformation and loading of data
    – Lack of proper flow of business information

  21. As we know that Data warehouse testing must deal with a huge volume of data to be tested. On like ordinary testing which identifies the defeats and validates the quality, and correctness of the client’s requirements. Because Data warehouse testing are done by more expects tester, there is a credibility of data testing integrity. However, there many possible challenges which faces the data warehouse testing:
    1. Due to the large volume of information to be tested, the possibility of data loss is real
    2. Duplication of data is also real
    3. Incompatibility of data is common
    4. Business decision making can be delayed or become erroneous due incomplete or missing information.

  22. Unstabilized source systems, This is because any bug in the source systems potentially injects unwarranted defects in data warehouse. Disparate data sources add to data inconsistency
    Prioritizing performance . Data warehouses should be built for performance rather than tuned for performance.
    Setting realistic goal
    Performance by design Performance is a consequence of design. So performance goals can be best addressed at the time of designing. If that’s not done, meeting up performance criteria can be an overwhelming challenge.
    Like anything in data warehousing, performance should be subjected to testing
    Because of such high dependencies, regression testing requires lot of planning. Time consuming.
    Reconciliation is complex
    the challenges of making a newly built data warehouse acceptable to the users. No matter how good or great you think your data warehouse is, unless the users accept and use it wholeheartedly the project will be considered as failure. In fact, most of the data warehouse projects fail in this phase alone.
    . Inconsistent data, duplicates, logic conflicts, and missing data all result in data quality challenges.
    b. Lack of proper flow of business information
    c. Loss of data might be there during the ETL process
    d. Existence of many ambiguous software requirements
    e. Unstable testing environment
    f. Fault in business process and procedures
    g. Volume and complexity of data are very huge
    h. Trouble acquiring and building test data
    i. Cost
    j. Performance
    k. User expectation

  23. DATA WAREHOUSE TESTING

    CHALLENGES IN DATA WAREHOUSE TESTING:
    1. Validity of data & source, duplicate, incorrect, etc
    2. Time consuming in testing
    3. Volume of data
    4. Incorrect data

  24. 1. Identify the possible challenges in data warehouse testing.
    ~Testers donot have the privilege to execute the ETL jobs on their own
    ~Incompatible and duplicate data
    ~Unavailability of inclusive test bed
    ~faults in business process and procedures
    ~volume and complexity of huge data
    ~trouble acquiring and building test data
    ~missing business flow information
    ~loss of data during ETL process

  25. Some of the important ETL Testing Challenges are:

    Unavailability of inclusive test bed at times
    Lack of proper flow of business information
    Loss of data might be there during the ETL process
    Existence of many ambiguous software requirements
    Existence of apparent trouble acquiring and building test data
    Production sample data is not a true representation of all possible business processes
    Some of the important issues with Data Warehouse Testing are:
    Data Warehouse/ETL testing requires SQL programming: This has become a major issue as most of the testers are manual testers and have limited SQL coding skills, thus making data testing very difficult
    Performing Data completeness checks for transformed columns is tricky
    Certain testing strategies used are time consuming.

  26. Some issues/possible errors that could occur with Data warehousing:
    The common skills required by the data warehouse testers are sql basics, knowledge of database, etc. However, if the data warehouse testers have insufficient skills that could lead to incorrect interpretations of the data.
    Errors could occur during the Data transformation testing where the data from various sources are collected and verified to transform as per the business rules.
    During Data transfer testing the data could be loaded incorrectly. There could be loss of data due or truncation.
    Improper and invalid data might not be reported or replaced with the incorrect default values.
    In the Software Migration Testing, the data from the data warehouse might not be working efficiently in the new environment or platform.
    During the Data integrity testing: duplicate data could be missed in the target systems.

  27. Possible challenges are :
    1.Ambiguous or duplicate data
    2.Very large volumes of data
    3. test data availability for the business function
    4. missing business flow information
    5.loss off data during ETL process
    6.Certain testing strategies used are time consuming.
    7.Improper flow of business information.
    8.Loss of data might be there during ETL Process.
    9.Incorrect, incomplete or duplicate data

  28. Data loss during the ETL process.
    Incorrect, incomplete or duplicate data.
    The system contains historical data, so the data volume is too large and extremely complex to perform ETL testing in the target system.
    Tough to generate and build test cases, as data volume is too high and complex
    ETL testing involves various complex SQL concepts for data validation in the target system.

  29. 1.Unavailability of inclusive test bed at times
    2. Lack of proper flow of business information
    3. Loss of data might be there during the ETL process
    4. Existence of many ambiguous software requirements
    5. Existence of apparent trouble acquiring and building test data
    6. Production sample data is not a true representation of all possible business processes

  30. Data Warehouse testing/ETL testing: The Data warehouse testing is one of the different types of testing performed when the project involves the huge volume of data.
    Challenges:
    • Incompatible and duplicate data
    • Loss of data during ETL process
    • Unavailability of the inclusive testbed
    • Testers have no privileges to execute ETL jobs by their own
    • Volume and complexity of data are very huge
    • Fault in business process and procedures
    • Trouble acquiring and building test data
    • Unstable testing environment
    • Missing business flow information

  31. Possible Challenges in data warehouse testing are
    1.Data loss during ETL testing.
    2.Duplicate/incorrect/incomplete data
    3.Production sample data is not a true representation of all possible business processes
    4.Data volume and complexity is huge.
    5.Since there is lot of data for testing in various tables and business logic is complex hence complete testing
    may not be practically possible due to time and budget constraints whereas client may expect the testing team to use all the available test data for testing

Leave a Reply

Your email address will not be published. Required fields are marked *

Check Also

Close
Close