Discipline · Established

Data Engineering

Building reliable systems for data

What does Data Engineering mean?

The discipline of collecting, transforming, storing and serving data for operational and analytical use.

Keep exploring

Data PipelineAn automated flow that ingests data, applies processing and delivers it to a destination.
Data IngestionThe process of collecting data from sources and loading or streaming it into a platform.
Batch ProcessingWork that runs on accumulated records at scheduled or triggered intervals.
Stream ProcessingContinuous computation over an ordered flow of records with low delay.
Change Data Capture (CDC)A technique that detects inserts, updates and deletes in a source system and forwards those changes elsewhere.
Data LakehouseAn architecture that keeps flexible object storage while adding table management, governance and analytics features.
Data MartA curated subset of data designed for a department, subject area or use case.
Operational Data Store (ODS)A current, integrated store used for operational reporting or as an intermediate step between source systems and analytics.