Senior PySpark ETL Engineer

Job title: Senior PySpark ETL Engineer
Location : Chennai
Experience : 10-12 years
Education Qualifications: Any Graduate

Role Summary:

• The Senior PySpark ETL Engineer is responsible for designing, building, optimizing, and operating scalable data pipelines using Apache Spark (PySpark).
• This role focuses on high-volume batch processing, with optional exposure to streaming data, ensuring performance, reliability, data quality, and cost efficiency across enterprise data platforms.
• The position requires strong Python expertise, hands-on experience with Spark, and deep knowledge of SQL and data modeling.
• It also demands the ability to take full ownership of data pipelines, managing them end-to-end in a production environment.

Mandatory Requirements:

• 10–12 years of overall IT experience, with strong focus on data engineering and ETL
• Minimum 3+ years of hands-on experience with PySpark / Apache Spark in production environments
• Strong experience in designing and implementing ETL / ELT pipelines at scale
• Excellent knowledge of SQL and relational database concepts
• Experience in handling large datasets in distributed environments
• Strong ownership mindset with excellent problem-solving skills and ability to independently manage production pipelines
• Hands-on experience with AWS EMR / Spark on Kubernetes / S3 and orchestration tools such as Airflow / Databricks / Azure Data Factory (ADF)