Blog (18) Brochures (4) Customer Success Stories (3) Customer Feedback (13) eBooks (3) Events (1) FAQs News (26) Newsletters (1) PPT Slideshow (1) Pre Installation Checklist (1) Training (1) Videos (9) Webinar (3) Whitepapers (4)

iCEDQ your Gatekeeper for Data Issues

The Challenges: Today’s organizations have thousands of data integration (ETL) processes constantly moving silos of data from various operational and/or external data sources to downstream applications.

Since the downstream system doesn’t have control over incoming data or the process, it can cause serious data issues due to:

The quality of the data depends on the upstream systems
The ETL jobs may not process the data correctly

Introducing iCEDQ

iCEDQ (Integrity Check Engine for Data Quality) is a technical and business facing auditing and monitoring platform that acts as a gatekeeper for all the incoming and outgoing data.

1. Auditing Rules Engine
To avoid such data issues iCEDQ provides an in-memory Audit Rules Engine based on Validation and Reconciliation concepts.

Validate Data

The validation rules can monitor the incoming data as well as outgoing data from the system.

Reconcile Data for ETL Process Quality

Data reconciliation techniques can be applied to verify data processed by the ETL jobs. If the data doesn’t reconcile, it sends an alert of issues with the process that populated the data.

2. Monitoring & Alerts for Ops
The iCEDQ rules can be embedded in your data flow and as the data processes are executed, the auditing rules will validate them. These results are then live streamed into a control room dashboard so that the responsible party can take action. Based on the severity of the issues users are alerted or the workflow can be automatically terminated.

3. Provide the Metadata for further Investigation	4. Workflow & Support for a ticketing system	5. Management Reports & Compliance
When an issue is identified, just being informed is not enough. The system provides additional information about the exact data element(s) where the issue exists and who should be consulted to resolve it.	Some data issues will take longer to fix than a day or week. For those, a workflow to generate tickets and follow-up can track the life cycle until the issue is resolved or rejected.	Management reports provide both the health of the data on a given day as well as the trends over time. Additionally, it satisfies compliance requirements such as BCBS-239 with the proof checks and controls on data.

While data processing or ETL is highly technical development-oriented work, data auditing is very agile and evolves over time. A rule-based auditing system solves both operational as well as compliance (BCBS-239) requirements.

Bottom Line:

iCEDQ is an effective gatekeeper helping organizations control their upstream data quality as well as data integration (ETL) process issues. It goes beyond the technical challenges and provides a business-friendly platform to manage the complete lifecycle for data issues.