Data Engineering (Java / Python Tech Lead)

Job Summary:

We are looking for a highly motivated and experienced Implementation. Technical Lead to oversee the end-to-end deployment and operationalization of our private's data platform. The ideal candidate will need to have a strong background in data engineering platform architecture sound technical knowledge on full stack technologies Java Python and will work on diverse projects from building APIs and web applications to data processing and automation around the data platform while ensuring alignment with business and technical requirements. This role requires Java Python programming and related frameworks along with strong data engineering and problem-solving skills mindset.

Mandatory Skills :JAVA, Python, Fast API, Snowflake, Azure, Realtime and batch data processing, SQL, Apache Kafka Spark

Key Responsibilities

Lead the development design development and deployment of data pipelines storage solutions and analytic tools

Ensure scalability reliability and security of the platform to meet current and future demands

Oversee the implementation of data lakes warehouses and ELT or ETL pipelines on azure and snowflake

Develop and Review code and ensure adherence to development best practices and standards

Foster a culture of collaboration innovation and continuous improvement within the team

Drive integration of Realtime and batch data processing capabilities

Ensure compliance with data governance policies security and regulatory requirements

Implement robust data quality lineage and cataloguing processes

Build and integrate APIs and backend systems using frameworks like Django Flask or Fast API

Develop scripts and tools for data analysis transformation and automation

Write unit tests integration tests and conduct debugging to ensure software quality

Work closely with cross functional teams including developer's product managers business analysts and other stakeholders to align the data platform with organizational goals

Translate business requirements into scalable technical solutions

Qualifications

Proven experience in leading the design and development of large-scale data platform

Proven experience as a full stack developer or similar role

Expertise in data platform technologies snowflake Azure SQL Server

Proficiency in data engineering tools and frameworks Apache Kafka Spark

Strong understanding of cloud platforms Azure and containerization Docker Kubernetes

Experience with both batch and Realtime data processing

Deep knowledge of SQL and programming languages JAVA Python

Strong Problem-solving communication and leadership skills

Experience of code deployments using CICD Pipelines

Proficiency with code repos GitHub

Knowledge and experience developing restful APIs

Certifications in Snowflake Azure

Experience in Alternatives Private equity, Private Capital Co-investments debt venture capital hedge and real assets domain.

Working with private market industry solutions Investran Alter domus I Capital Subscribe backstop chronograph

Strong understanding of private market operations investment Lifecyle and structures and fund life cycle

Technology certifications on Azure snowflake Domain certifications on CFA