Senior Data Engineer

Role Overview

Iris Software is one of the fastest-growing IT services companies, helping clients thrive with technology-enabled transformation across various industries. We cover complex, mission-critical applications with the latest technologies such as high-value complex Application & Product Engineering, Data & Analytics, Cloud, DevOps, Data & MLOps, Quality Engineering, and Business Automation.

Key Responsibilities

• Design Scalable Data Engineering Solutions: Design scalable data engineering solutions using PySpark and modern distributed data processing frameworks.
• Define Ingestion, Transformation, and Processing Architectures: Define data ingestion, transformation, and processing architectures aligned with business and analytical objectives.
• Design and Optimize Snowflake or Delta Lake on Databricks Solutions: Design and optimize Snowflake or Delta Lake on Databricks solutions to support enterprise-scale data platforms.
• Lead Implementation of High-Performance Batch and Streaming Data Pipelines: Lead implementation of high-performance batch and streaming data pipelines.
• Design and Optimize Event-Driven Data Architectures: Design and optimize event-driven data architectures using Apache Kafka or Amazon Kinesis.
• Define Data Streaming Standards and Integration Frameworks: Define data streaming standards, integration frameworks, and scalable processing patterns.
• Architect Workflow Orchestration Solutions: Architect workflow orchestration solutions using Apache Airflow or Databricks Workflows.
• Establish Monitoring, Scheduling, and Operational Controls: Establish monitoring, scheduling, and operational controls for reliable pipeline execution.
• Drive Data Quality and Governance Practices: Drive development of business-focused data products by improving data quality, discoverability, usability, documentation, and trusted data consumption across analytical platforms.
• Promote Responsible Use of AI-Assisted Engineering Capabilities: Promote responsible use of AI-assisted engineering capabilities to improve development productivity, testing, documentation, and engineering quality.
• Review Data Pipeline Designs and Implementations: Review data pipeline designs and implementations to ensure adherence to engineering, scalability, and performance standards.
• Troubleshoot Complex Data Processing, Workflow, and Streaming Platform Issues: Troubleshoot complex data processing, workflow, and streaming platform issues through detailed root cause analysis.
• Mentor Team Members on PySpark, Snowflake, Delta Lake, Kafka, Kinesis, Airflow, and Data Engineering Best Practices: Mentor team members on PySpark, Snowflake, Delta Lake, Kafka, Kinesis, Airflow, and data engineering best practices.
• Collaborate with Various Teams and Stakeholders: Collaborate with various teams and stakeholders to support end-to-end data platform delivery.

Behavioral Competencies

• Demonstrates Strong Ownership: Drives data engineering excellence while demonstrating strong ownership.
• Effective Collaboration: Collaborates effectively with various teams and business stakeholders to ensure smooth delivery.
• Quality Focus: Promotes quality-focused engineering through proactive validation, optimization, and continuous improvement.
• Analytical Thinking: Demonstrates strong analytical thinking to evaluate complex data engineering and platform challenges.
• Adaptability: Demonstrates adaptability while managing evolving technologies, data ecosystems, and business requirements.
• Effective Communication: Communicates effectively regarding delivery status, risks, dependencies, and improvement opportunities.
• Attention to Detail: Maintains high attention to detail across data architecture, pipeline design, testing, and implementation activities.

Mandatory Competencies

• Mandatory Skills: Databricks Workflows, PySpark, Apache Kafka, Snowflake

Company Values

At Iris, we believe that people thrive when they have ownership of their career and the right opportunities to shape it. Our Employee Value Proposition is "Build Your Future. Own Your Journey." We foster a culture where your potential is valued, your voice matters, and your work creates real impact. With cutting-edge projects, personalized career development, continuous learning and mentorship, we support you to grow and become your best — both personally and professionally. We take pride in the work we do and the people we support. Join us today and be part of a team that is making a difference.