Data Pipeline Architect

About: We are seeking a Data Pipeline Architect to design and implement scalable and efficient ETL/ELT pipelines that support analytics and machine learning workloads across the organization, fostering a culture of data-driven decision making.

Responsibilities:Design and implement end-to-end data pipelines for batch and real-time streaming use cases, ensuring seamless data flow and integration across various systems and platforms.Define and enforce standardized data modeling standards and governance policies, ensuring data consistency, accuracy, and reliability.Collaborate with cross-functional teams to identify business requirements and design data solutions that meet organizational needs.Mentor junior engineers and drive best practices in data reliability, ensuring that data pipelines are secure, scalable, and maintainable.Develop and maintain technical documentation, ensuring that data pipelines and solutions are well-documented and easily accessible.Stay up-to-date with emerging technologies and trends in data engineering, ensuring that the organization remains competitive and innovative.

Requirements:A minimum of 5+ years of experience in data engineering, with a strong background in designing and implementing scalable and efficient data pipelines.In-depth expertise in Apache Spark, Airflow, Kafka, and cloud platforms (AWS/GCP), with the ability to design and implement complex data architectures.Strong SQL and Python skills, with experience in data warehousing and lakehouse architectures.Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams.Experience with data governance policies, data modeling standards, and data reliability best practices.