Data Warehouse & ETL testing

Never ever test manually again! Using BiG EVAL you automate the test processes for your data warehouse system and the ETL or ELT processes.

Use Case DWH und ETL testen

BiG EVAL takes away the manual effort from testing your ETL or ELT processes and your data warehouse. This is done by comparing and checking test data between the source systems, the staging area, the data warehouse and all components in between.
Exactly the way in which you would do it manually – but automated.

Unit tests

During development, individual pieces of code are tested for their correct functioning without regard to their dependencies. Thus can check, e.g. single data transformation tasks or even MDX formulas.

Integration tests

Do the individual components of the system work together properly? This can only be verified by providing them in a test environment and reviewing them across the board.

Data plausibility check

This process checks if the data is accurate and credible. By involving the business user you increase the value of this test method.

Acceptance testing

This type of testing has two objectives with respect to the user. On the one hand, it ensures that the data provided to the user meet the expectations. On the other hand, this ensures that the tools provided to the user meet the expectations.

Regression testing

Testing a function over and over again increases the certainty that no errors will creep in during development that affect an already tested part of the system. Adhering to short release cycles in agile projects is not possible without regression testing.

In addition, regression testing in the productive operation of the system can be very useful to regularly monitor the quality of the data, thereby providing an early warning system for any problems that occur.

Performance Tests

This testing process is an important aspect of user acceptance and therefore can not simply be ignored. Does loading of data lead to a slowdown of the systems? Does the parallel queriy of data lead to a slowdown of the systems?

Testmethods

Choose out of a predefined set of testmethods and -algorithms or create your own by using a database query language, C# or even R.

Vorgefertigte und benutzerdefinierte Testmethoden

Specific tests for your data

Compare number of records

Compare the number of data records in the source systems with those in the data warehouse for a first overview.

Check business-keys

Which records are missing or too much respectively duplicated you only learn when you review business keys.

Validate source data

Do not allow erroneous data to be loaded into your data warehouse by checking that the data provided is error-free, even during staging.

Check scopes of validity

Would you load a customer record into your data warehouse that has a date of birth around 1850? Validate information that is subject to restrictions early.

Timeline tests

Be sure to check, if your implementation of historicized data in Slowly Changing Dimensions (SCD 2) is correct.
Are there any gaps? Are there any overlaps or date rotators?

Distribution tests

Classifiable data should be checked for their distribution. Were around 40% of your sales made on product group A and only 10% on product group B?

Check aggregates and KPI's

You can mathematically check business logic in the data warehouse or in analysis systems such as OLAP cubes and, if necessary, compare them with values from your source systems.

Plausibilize data

If you involve business users to your tests, you'll benefit of their knowledge. Check, for example, whether the November and December sales differ in a credible manner.

Performance testing

Define reference queries or record real queries from your users to constantly monitor their performance.

End-to-End Data Validation

The technology-neutral concept of BiG EVAL allows validation of key performance indicators starting in your source systems up to your analysis-model and reporting layer. Let’s see a video to see how that’s done.