Senior Data Engineer

We are seeking a seasoned Senior Data Engineer to join a dynamic team in Washington, DC. The ideal candidate will have a strong background in designing, developing, and maintaining scalable ETL data pipelines that handle both structured and unstructured data formats for batch and streaming processes. In this role, you will utilize advanced analytics tools and methodologies to enhance the organization's analytics platform, work closely with business and technical teams to gather requirements, and design data solutions aligned with organizational objectives. You will also implement complex data operations using Python, Java, Spark, and SQL, optimizing ETL/ELT processes for performance and efficiency. Additionally, you will manage data across various platforms, leveraging cloud services and big data technologies.

Responsibilities:

• Design, develop, and maintain scalable ETL data pipelines for batch and streaming data.
• Handle both structured and unstructured data formats effectively.
• Utilize advanced analytics tools to enhance the organization's analytics platform.
• Collaborate with business and technical teams to gather requirements and design comprehensive data solutions.
• Implement complex data operations using Python, Java, Spark, and SQL.
• Optimize ETL/ELT processes for performance and efficiency across diverse data types.
• Manage data across various platforms, leveraging cloud services and big data technologies.
• Develop and maintain data models that support key business processes and decision-making.
• Provide Level 3 support for data pipelines, ensuring high availability and reliability.
• Source and integrate data from ERP systems and Software as a Service (SaaS) applications.
• Ensure data quality and consistency through rigorous testing and validation processes.
• Monitor and troubleshoot data pipelines to quickly identify and resolve issues.
• Document data workflows, processes, and architecture to ensure knowledge sharing and alignment with industry best practices.
• Collaborate with data scientists and analysts to support advanced analytics initiatives.
• Stay up-to-date with emerging technologies and industry trends to continuously improve data engineering practices.