Sr. Data Engineer

Sr. Data Engineer

Lead activities relating to designing, building, testing, implementing and maintaining solutions, products and processes. Provide IT teams and business personnel with technology solutions by evaluating technology standards, market availability of products, risks and benefits of technology to deliver solutions that meet business/IT needs. Coordinate the tasks of less experienced engineers and collaborate across teams as needed. Independence level: Works independently with minimal guidance. Works closely with scrum masters, project managers, QA, Creative, and UX team members to ensure compliance with project needs. Your role will be centered on the development and improvement of new and existing features. Additionally, you will be reviewing the code of your peers, advocating for standard development methodologies, and participating in an inclusive, energetic and highly cross-functional team. The role encompasses architect, design, implementation, and testing to ensure building products right and to release them with high quality.

Maintain deep knowledge of technology and its application across our business areas or for our customers.

Promote knowledge via seminars, presentations and publications within IT and Constellation.

Participate in technological innovation to drive new business opportunities and solve complex business problems.

Shape and advance core team processes to increase operational efficiency and quality.

Coach and lead other engineers assigned to their product or project team.

Considered highly skilled and proficient in discipline. Conducts complex, important work under minimal supervision and with wide latitude for independent judgment.

PRIMARY DUTIES AND ACCOUNTABILITIES

• Produce high-quality, secure, and maintainable code/configurations
• Learn and understand business processes with limited guidance
• Work collaboratively as a member of the development team to build best-in-class software solutions in an agile environment
• Support and research issues across all application layers and database
• Identify areas to improve and scale our Azure architecture and application design
• Ensure code can be deployed using Azure DevOps
• Design and query database tables, views, functions, stored procedures and batch processes
• Develop, implement, and support interfaces that connect our websites, back-end systems, and various 3rd party cloud solutions, vendors and customers.
• Engage with business partners to gather information, analyze requirements and deliver practical, efficient and cost-effective solutions that satisfy business needs. Write product specifications and design documentation for assigned system components.
• Design, develop and/or review complex code or configurations to ensure solutions meet the requirements of functional and technical specifications. Develop and/or review project technical architecture design and development. Solutions may include but are not limited to the development of various types of applications, web, mobile app, full stack or integrations hosted on premises data centers or in the cloud. Develop automated Unit tests and/or automated UI tests.
• Lead team members and provide oversight for less experienced engineers; Stay on top of the latest technologies and trends. Support development of version control principles (e.g. Git and working within an agile environment)
• Enhance knowledge of and compliance with preferred technologies, methodologies, standards and policies. Maintain technical knowledge and business acumen within own discipline or function. Strong debugging and problem-solving skills; lead peer code reviews.
• Provide IT teams and business personnel with technology solutions by weighing advantages of technology trends, market availability of products, risks and benefits of technology to meet business/IT needs. Participate in IT architecture review and standards setting. Assist in the development of white papers, conducting presentations as needed to explain why a technology is being recommended by IT.

MINIMUM QUALIFICATIONS

• Bachelor's degree (ex: Computer Science or related discipline) and 5 - 8 years of relevant experience (programming technologies) or minimum 9-12 years of relevant combination of education and work experience Knowledge of ETL tools.
• Experience with private and public cloud architectures, pros/cons, and migration considerations
• Minimum of 5 years of RDBMS experience
• Experience with JSON, Parquet, Delta, Avro data structures
• Experience implementing data pipelines using latest technologies and techniques
• Experience with SDLC products (Azure DevOps and GitHub) or similar agile project management tools
• 3-5+ years of hands-on experience in programming languages such as c#, node.js, python, Pyspark, Spark SQL
• At least 3-5 years of consulting or client service delivery experience on Azure
• Experience handling Structured and unstructured datasets
• Strong t-SQL skills with experience in Azure SQL DW
• Experience in Data Modeling and Advanced SQL techniques
• Cloud migration methodologies and processes with tools like Azure Data Factory, Databricks, Fivetran, Event Hub, etc.
• Excellent problem solving, analytical, and critical thinking skills.
• Experience with cloud-native Azure technologies like Azure Functions, Event Grids, Service Bus Queues, Cosmos DB, or related is a plus.
• Experience with designing integration applications, both upstream and downstream applications.
• Experience with git, Azure DevOps, build pipelines, code branching/merging, code management.
• Experience writing and executing stored procedures
• Knowledge of Event driven architecture.

PREFERRED QUALIFICATIONS

• Experience in Medallion Architecture, Data Vault
• Near real time and batch ingestion