Job Information
Insight Global Lead Data Engineer in Atlanta, Georgia
Job Description
Compensation:
$65/hr to $80/hr. - Exact compensation may vary based on several factors, including skills, experience, and education.
Benefit packages for this role will start on the 31st day of employment and include medical, dental, and vision insurance, as well as HSA, FSA, and DCFSA account options, and 401k retirement account access with employer matching. Employees in this role are also entitled to paid sick leave and/or other paid time off as provided by applicable law.
Job Description:
A Fortune 500 client of Insight Global is looking for a Lead Data Engineer to join their team. This person will be helping develop a new marketing data platform that measures digital marketing effectiveness. The platform is already architected on Azure and they predominantly use scala running on spark. The bulk of this person's role will be hands-on development, but they will also be responsible for interviewing and hiring 3 additional engineers to work alongside them to complete the project.
Job Responsibilities:
Lead end to end design and development of the W+ data solution.
Develop data pipelines to ingest and transform data using Scala/PySpark and SQL on Azure Synapse Spark.
Setup orchestration of the pipelines using Azure Synapse pipeline.
Source and version control code/configuration artifacts using GitHub.
Create GitHub actions to deploy code/configuration artifacts.
Create tables/views (using SQL) in Synapse SQL Serverless pool for consumption.
Investigate and remediate data pipeline errors or performance issues.
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com .
To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/ .
Skills and Requirements
at least 5-7 years' experience in data engineering
In depth understanding of distributed computing (Spark)
Proficient in Scala/PySpark and SQL.
Knowledge of source code repository like GIT/Azure Repos & build tools like Maven.
Experience working with Azure Synapse. PySpark null
We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion,sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military oruniformed service member status, or any other status or characteristic protected by applicable laws, regulations, andordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process, please send a request to HR@insightglobal.com.