Data Engineer - Python

June 18, 2022

Remote, USA

A Nortal  Data Engineer’s role is to lead and implement data engineering projects, support and maintain data pipelines, and provide expertise and best practices regarding data engineering for staff across the company. Typical data engineering projects focus on improving performance and adding features to existing data pipelines. As needed, the  Data Engineer will design and develop new data engineering pipelines as part of the Data Engineering Team. Further, the  Data Engineer will help decide how and implement improvements to pipelines, systems, and infrastructure.  Data Engineers are expected to have an in-depth understanding of data engineering best practices as well as expertise with the tools needed to debug and diagnose issues.  Data Engineers contribute as a team member to testing, QA, and documentation of data pipelines and systems. The Nortal Data Engineering Team is a small agile team of motivated individuals who welcome challenges, adapt quickly, strive to acquire new knowledge, learn new technologies, accept new responsibilities, and work well individually, as a team, and with other teams within the organization. You will work closely with the Data Engineering Lead, product managers, workflow developers and other software engineers to advance projects and meet objectives. We seek an experienced engineer who is interested in pursuing an advanced individual contributor technical career track over a technical management career track. Duties and Responsibilities Assembling large, complex sets of data that meet non-functional and functional business requirementsIdentifying, designing, and implementing internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes  Building required infrastructure for optimal extraction, transformation and loading of data from various data sources using AWS and SQL technologiesBuilding analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition Working with stakeholders including data, design, product and executive teams and assisting them with data-related technical issuesWorking with stakeholders including the Executive, Product, Data and Design teams to support their data infrastructure needs while assisting with data-related technical issues  Data Engineer Expectations Maintain an in-depth understanding of relevant data engineering best practicesDisplay expertise with tools needed to debug and diagnose issuesGiven a medium to large understood problem, can design, implement, and deploy a solutionShow initiative and help when needed without being askedDeliver feedback in a constructive mannerProvide guidance to other engineers and workflow developersWork well with technical leads, incorporating feedback as needed Required Skills AWS: S3 Data Lake, Athena, Redshift, EMR, Glue, ECS, EKSProficiency with Python in a data engineering context. Ex: Pandas, PySparkSource control using GitProficiency with SQLProficiency with workflow orchestration conceptsAdaptable to Windows, Linux, and container-based deployment environmentsWorkflow performance, scaling, and optimization. Big Data solutions in some cases.Security & privacy principals Desired Skill & Qualifications ETL productivity toolsSparkEKSTerraformHelmPython3Nortal US Is An Equal Opportunity Employer And All Qualified Applicants Will Receive Consideration For Employment Without Regard To Race, Color, Religion, Sex, National Origin, Disability Status, Protected Veteran Status, Or Any Other Characteristic Protected By Law.

Built with ❤️ for the ML Community by Dom © 2022 RemoteML