Senior Data Engineer

About The Role:

We are building a modern data platform that will allow Willow Innovations the ability to make the most optimal business decisions. We are a growing organization and the Senior Data Engineer will have a huge influence in shaping the data strategy. If you feel a deep sense of ownership for your work, are passionate about data, desire to solve problems, are curious and love learning new things, we’d love to talk to you!

Roles & Responsibilities | What You'll Do:

• Own and drive data integration architecture involving streaming, micro-batch and batch
• Work closely with various business groups to collect business requirements
• Design, develop and deploy high performance data pipelines both inbound and outbound
• Build aggregate and enriched data sets needed to effectively analyze / visualize data
• Document, maintain and support data pipelines
• Measure and monitor quality and availability of data
• Implement data policies - retention, privacy, security, and compliance
• Responsible for maintaining the Data Pipeline framework
• Mentor and guide junior data engineers

Experience | What You'll Bring:

Must Have:

• 5+ years of experience in building data lakes and data warehouses
• Experience working with Google Cloud, BigQuery, GCS, Pub Sub, Data Flow etc
• Expert knowledge on SQL and Python programming
• Good understanding of data formats such as JSON, Parquet, ORC, XML, etc
• Experience in tuning queries for performance and scalability
• Experience working with any workflow management tools (Airflow, Oozie, etc)
• Excellent organizational, prioritization and analytical abilities
• Strong communication and interpersonal skills

Nice to Have:

• Experience in Real Time Data Ingestion using Kafka, Spark, Storm etc
• Knowledge of NoSQL
• Experience with Data visualization tools such as Looker, Tableau, PowerBI