Job Description
Key Accountabilities:
- Active contributor to code development in projects and services.
- Manage and scale data pipelines from internal and external data sources to support new product launches and drive data quality across data products.
- Build and own the automation and monitoring frameworks that capture metrics and operational KPIs for data pipeline quality and performance.
- Responsible for adopting best practices around systems integration, security, performance, and data management defined within the organization.
- Empower the business by creating value through the increased adoption of data, data science, and business intelligence landscape.
- Collaborate with internal clients (data science and product teams) to drive solutions and POC discussions.
- Develop and optimize procedures to “productionalize” data engineering pipelines.
- Define and manage SLA’s for data products and processes running in production.
- Support large-scale experimentation was done by data scientists.
- Prototype new approaches and build solutions at scale.
- Research in state-of-the-art methodologies.
- Create documentation for learning and knowledge transfer.
Qualifications
- 5+ years of overall technology experience that includes at least 2+ years of hands-on software development, data engineering.
- 2+ years of experience in SQL optimization and performance tuning, and development experience in programming languages like Python, PySpark, Scala, etc.).
- 1+ years in cloud data engineering experience in Azure. Azure Certification is a plus.
- Experience with version control systems like Github and deployment & CI tools.
- Experience with data modeling, data warehousing, and building high-volume ETL/ELT pipelines.
- Experience with data profiling and data quality tools is a plus.
- Experience in working with large data sets and scaling applications like Kubernetes is a plus.
- Experience with Statistical/ML techniques is a plus.
- Experience with building solutions in the retail or in the supply chain space is a plus
- Understanding metadata management, data lineage, and data glossaries is a plus.
- Working knowledge of agile development, including DevOps and DataOps concepts.
- Familiarity with business intelligence tools (such as PowerBI).
Skills, Abilities, Knowledge:
- Excellent communication skills, both verbal and written, and the ability to influence and demonstrate confidence in communications with senior level management.
- Comfortable with change, especially that which arises through company growth.
- Ability to understand and translate business requirements into data and technical requirements.
- High degree of organization and ability to coordinate effectively with team.
- Positive and flexible attitude and adjust to different needs in an ever-changing environment.
- Foster a team culture of accountability, communication, and self-management.
- Proactively drive impact and engagement while bringing others along.
- Consistently attain/exceed individual and team goals
Differentiating Competencies Required
- Highly influential and having the ability to educate challenging stakeholders on the role of data and its purpose in the business.
- Understands both the engineering and business side of the Data Products released.
- Places the user in the center of decision making.
- Teams up and collaborates for speed, agility, and innovation.
- Experience with and embraces agile methodologies.
- Strong negotiation and decision-making skill.
- Experience managing and working with globally distributed teams.
Ab