Sr Data Engineer

Job Description
Experience Required - 6+ Years

Must Have Technical/Functional Skills
• Strong hands-on experience in Big Data technologies and distributed data processing frameworks.
• Proficiency in Python for data engineering, scripting, and performance optimization.
• Extensive experience with Apache Spark (PySpark/Spark SQL) for large-scale data transformation and processing.
• Solid experience in designing and developing ETL/ELT pipelines for batch and real-time data processing.
• Experience working with Google Cloud Platform (GCP) services such as BigQuery, Cloud Storage, Dataflow, Dataproc, Composer, and Pub/Sub.
• Strong understanding of data warehousing concepts, dimensional modeling, and data lake architectures.
• Experience with CI/CD pipelines and DevOps practices for data engineering workflows.
• Proficiency in version control systems (Git) and build/release management.
• Experience with workflow orchestration tools such as Apache Airflow or Cloud Composer.
• Experience working in Agile/Scrum environments with cross-functional teams.
• Strong analytical and problem-solving skills
• Strong collaboration and stakeholder communication skills

Roles & Responsibilities
• Design, develop, and maintain scalable Big Data pipelines using Spark (PySpark/Spark SQL) for batch and real-time processing.
• Build and optimize robust ETL/ELT workflows to ingest, transform, and load data from multiple structured and unstructured sources.
• Develop high-performance SQL queries and implement efficient data models to support analytics and reporting needs.
• rchitect and implement data solutions on Google Cloud Platform (GCP) using services such as BigQuery, Dataproc, Dataflow, Cloud Storage, Pub/Sub, and Composer.
• Monitor, troubleshoot, and resolve production issues in distributed and cloud environments.
• utomate operational processes and enforce coding standards, version control, and documentation practices.
• utomate operational processes and enforce coding standards, version control, and documentation practices.
• Collaborate with cross-functional teams including business stakeholders, data engineers, and product teams
• Drive analytics initiatives to provide actionable insights.
• Participate in Agile ceremonies including sprint planning, design reviews, and retrospectives.
• Continuously improve reporting frameworks and data processes in alignment with Agile delivery practices