Release Notes - v15.3
🚀 New Feature: Data Quality Check for DB Pipelines
Section titled “🚀 New Feature: Data Quality Check for DB Pipelines”We are excited to introduce Data Quality Check for Database (DB) connections — a new capability designed to improve data reliability and pipeline monitoring.
Why This Feature is Needed
Section titled “Why This Feature is Needed”Data inconsistencies between source and destination systems can impact reporting accuracy and downstream analytics.
The new Data Quality Check feature helps identify:
- Mismatched record counts between source and destination systems
This enables teams to proactively monitor pipeline health and maintain data integrity.
Where to Find It
Section titled “Where to Find It”For DB pipelines, a new 4th step — “Quality Check” has been added to the pipeline configuration flow.
Users can:
- Enable or disable the quality check
- Configure thresholds
- Schedule checks as needed
How It Works
Section titled “How It Works”The feature compares data between the:
- Source system
- Destination system
If the variance exceeds the configured threshold, an automated email notification is triggered to alert users about the discrepancy.
Detailed History & Monitoring
Section titled “Detailed History & Monitoring”A dedicated History page is now available to help users monitor past quality check runs.
From the History page, users can:
- View detailed comparison statistics
- Track discrepancies over time
- Analyse quality check execution results
Additionally, a new Schema button is available on the right side of the listing page.
The Schema comparison feature helps users compare:
- Source table schema
- Destination table schema