Big Data Engineer

These professionals engineer resilient Extract, Transform, Load (ETL) and ELT pipelines using distributed processing frameworks like Apache Spark, Apache Kafka, Apache Flink, and Python. They architect scalable cloud data warehouses and lakehouses across Snowflake, Google BigQuery, and Databricks, enforcing schema evolution, partition pruning, and data compaction strategies to optimize query performance and storage costs. They orchestrate complex workflow DAGs using Apache Airflow and Dagster, ensuring automated backfills, idempotent execution, and zero data duplication.

Furthermore, Data Engineers collaborate closely with machine learning engineers and analytics teams to construct real-time feature stores, data catalogs, and data quality observability frameworks. They enforce strict data governance and security controls, implementing data masking, role-based access, and automated data retention pipelines complying with the Kenya Data Protection Act. Working across leading telecommunication corporations, fintech startups, commercial banking institutions, and multinational tech firms in Nairobi, they engineer the data infrastructure powering the continent digital transformation.

Starts the day reviewing Airflow DAG execution graphs to ensure overnight billing and transaction batch runs completed without schema validation failures. Midday optimizes a PySpark aggregation job that processes 50 million daily mobile money records, reducing runtime by 45%. Afternoon designs a Kafka streaming topic with Avro schema registry for real-time fraud alerts. Wraps up the day writing dbt data tests to verify financial reconciliation consistency between core banking and reporting lakes. You coordinate closely with cross functional colleagues to align on immediate deliverables, double checking equipment readiness and standard operating protocols. Throughout the shift, you document system performance metrics, consult with operational supervisors on technical exceptions, and uphold rigorous occupational health and environmental safety protocols.

In the afternoon, you execute core deliverables, consult with cross functional colleagues, document operational outcomes, and ensure that all equipment and handover notes are fully updated before completing the shift. Before completing the shift, you reconcile physical logs, review operational checklists with the oncoming team, archive incident reports, and brief project supervisors on task completion.

Data Engineers design, construct, operationalize, and maintain the scalable data pipelines, distributed storage architectures, and ingestion systems that convert raw, messy enterprise data into clean, structured foundations for analytics and artificial intelligence. Operating at the foundational software engineering core of data-driven enterprises, they build batch and real-time streaming architectures capable of ingesting billions of events from mobile applications, payment gateways, IoT sensors, and operational databases. In Kenya, where mobile money transaction volumes, fintech credit engines, and e-commerce platforms generate petabytes of continuous telemetry, these engineers keep analytical systems alive.

Core responsibilities include design distributed data lakehouse architectures and partition storage layouts., operational coordination, and ensuring strict adherence to Kenyan statutory frameworks and quality standards.

Key organizations hiring for this role include Safaricom, KCB Bank Kenya, Cellulant, Cellulant Kenya, alongside expanding opportunities in Nairobi, Mombasa, Kisumu, and regional infrastructure projects.

Data Engineer Specialist, Senior Data Engineer, Data Engineer Lead, Data Engineer Consultant

High demand in Safaricom, Equity Bank, KCB, Absa, and fintech companies. Due to the rapid growth of digital lending and mobile payments in Kenya, data engineers are among the most competitively compensated technology professionals in Nairobi. Employment opportunities concentrate across major industrial, commercial, and financial hubs including Nairobi, Mombasa, Kisumu, Nakuru, and Eldoret, alongside specialized regional project sites. Remuneration varies between private contracting firms and institutional employers, with certified practitioners enjoying substantial premiums, overtime allowances, and supervisory advancement opportunities. Continuous industrial modernization across Kenya sustains robust demand for certified practitioners with verifiable technical competence and practical operational excellence.