Data Engineering Lead

Data Engineering Lead

We are seeking a highly skilled and experienced Data Engineering Tech Lead to join our team in the manufacturing industry. The ideal candidate will have a strong background in data engineering across any cloud platform, with expertise in Azure Synapse, Python, PySpark, IoT, ThingWorx, SQL, Azure SQL, Azure Data Explorer, and data visualization tools like Power BI or Tableau. As a Tech Lead, you will be responsible for architecting, designing, and leading data engineering solutions to support IoT-driven manufacturing processes, predictive analytics, and operational efficiency initiatives. Experience in the manufacturing domain is highly preferred.

Location: Portland, Oregon (Must have)

Responsibilities

• Lead and mentor a team of data engineers, providing technical guidance and enforcing best practices.
• Define and enforce data engineering standards, processes, and architecture principles.
• Design and architect scalable, secure, and high-performance data solutions tailored to manufacturing use cases, such as predictive maintenance, quality monitoring, and production optimization.
• Develop end-to-end data architectures, including ingestion, transformation, storage, and analytics layers, using Azure Synapse, Azure SQL, and Azure Data Explorer.
• Create reusable frameworks and patterns for data pipelines, ensuring modularity and scalability.
• Design solutions for integrating IoT data from platforms like ThingWorx into cloud-based data platforms for real-time analytics and insights.
• Ensure alignment with enterprise architecture standards and data governance policies.
• Build and optimize data pipelines and workflows using PySpark, Python, and Azure Synapse.
• Implement scalable ETL/ELT processes to ingest, transform, and store data from IoT devices, sensors, and manufacturing systems.
• Develop and maintain data models and schemas for analytics and reporting, focusing on manufacturing KPIs such as OEE (Overall Equipment Effectiveness), downtime, and quality metrics.
• Work with IoT platforms, including ThingWorx, to collect and process real-time data from connected manufacturing devices and sensors.
• Architect solutions to integrate IoT data with cloud-based data platforms for advanced analytics and actionable insights.
• Design and develop interactive dashboards and reports using Power BI or Tableau to provide actionable insights to stakeholders.
• Collaborate with business teams to define key metrics and visualizations for monitoring manufacturing performance.
• Optimize dashboards for performance and usability, ensuring they meet business requirements.
• Leverage Azure SQL, Azure Data Explorer, and other Azure services to build and manage data storage and analytics solutions for manufacturing operations.
• Ensure data security, governance, and compliance across cloud environments.
• Communicate technical concepts and solutions effectively to both technical and non-technical stakeholders.

* Act as a Tech Lead in a SAFe Agile environment, participating in Program Increment (PI) planning, sprint planning, and retrospectives.

* Collaborate with Product Owners, Scrum Masters, and other stakeholders to prioritize and deliver features in alignment with business goals.

* Ensure the team adheres to agile principles and practices, delivering incremental value in each sprint.

* Foster a culture of continuous improvement and collaboration within the team.

Must-Have Skills

* Strong experience in data engineering across any cloud platform (Azure, AWS, or GCP).

* Expertise in data architecture and solution design for scalable and secure data platforms.

* Proficiency in Azure Synapse Analytics, Azure SQL, and Azure Data Explorer.

* Strong programming skills in Python and PySpark for data processing and transformation.

* Advanced knowledge of SQL for querying and managing relational databases.

* Experience designing and implementing scalable data pipelines and workflows.

* Knowledge of IoT data processing and integration with platforms like ThingWorx.

* Experience in the manufacturing industry, with a focus on IoT-driven use cases such as predictive maintenance, quality monitoring, and production optimization.

* Proficiency in data visualization tools like Power BI or Tableau for creating dashboards and reports.

* Strong problem-solving skills and ability to work in a fast-paced environment.

* Hands-on experience with SAFe Agile methodologies, including participation in PI planning and sprint execution.

Good-to-Have Skills

* Experience with other Azure services such as Azure Data Factory, Azure Databricks, or Azure Event Hub.

* Familiarity with real-time data processing and streaming technologies.

* Knowledge of data governance, security, and compliance best practices.

* Experience with other IoT platforms or protocols (e.g., MQTT, OPC-UA).

* Familiarity with manufacturing systems such as MES (Manufacturing Execution Systems) or SCADA.

Qualifications

* Bachelor's or Master's degree in Computer Science, Data Engineering, or a related field.

* You will be responsible for analyzing complex data sets, identifying key insights, and developing strategies to optimize industrial processes through the implementation of IoT solutions.

* 10+ years of experience in data engineering, with at least 2 years in a technical leadership role.Experience in Strong SQL and a scripting language such as Python or R

• Proven track record of delivering end-to-end data solutions in a cloud environment.
• Demonstrated experience in data architecture and solution design.
• Prior experience working in the manufacturing industry is highly preferred.

* Experience working with BI tools like Tableau, Power BI

* Strong understanding of data modeling and analytical algorithms

* Experience working in Azure environment primarily Azure Synapse Analytics, ADLS Gen2 preferred

* Ability to cleanse, curate, and manipulate data; as well as visualize and story-telling the results of an analysis

* Clear communication and storytelling via presentations, key performance metrics, reports, and data

* Proven experience and ability to deliver results on multiple projects in a fast-paced agile environment

* Self-starter, highly organized and result oriented; and should be able to work with minimal direction

* Embrace and understand Agile methodology.

* Adjust to new stakeholders when they are identified (and intake/requirement gathering).

* Learn with architect and engineers as we engage in a new technology platform/architecture

* Work across a solutions team that is a mix of teammates from differing functions and located in different places around the world.

* Ensure accountability for teammates and themselves.

* Work with the team to meet goals (team player).

* Communicate early and often when clarity is needed (not make assumptions).

* Be transparent and ask for help when needed.