Data Engineer

About Private Energy Partners 

Private Energy Partners (PEP) is part of Quinbrook Infrastructure Partners (Quinbrook) group, a specialist investment manager focused exclusively on lower carbon and renewable energy infrastructure investment and operational asset management in the US, UK and Australia. Quinbrook is led and managed by a senior team of power industry professionals who have collectively invested over US$ 8.2 billion in energy infrastructure assets since the early 1990’s, representing over 19.5GW of power supply capacity.  Quinbrook invests across the technology landscape encompassing distributed scale solar PV, onshore wind, battery storage, biomass, fugitive methane recovery, demand response, power-to-x, grid support and flexibility, community energy networks, EV charging and ‘Virtual Power Plants’.  PEP is a cross fund team that provides specialty project and technology advisory services including support for digital activities.

Role Description

We are seeking an experienced and driven Data Engineer to join our growing digital products team based in Los Angeles, CA. In this role, you will design and support data infrastructure that powers our mission to transition to a lower carbon energy future. Working closely with key stakeholders at Quinbrook portfolio companies, you’ll play a crucial role in developing and managing data solutions, primarily leveraging Python, SQL and AWS services, to enable effective decision-making and operational efficiency. This position requires collaboration with the investment and asset management teams across the US, UK, and Australia, as well as engagement with cross-functional teams to meet our data integration, analytics, and reporting needs.

Key Responsibilities:
The ideal candidate should excel at the following activities:

•             Develop and Manage Data Pipelines: Build and maintain ETL (Extract, Transform and Load) processes using AWS Glue and Python scripting for reliable, secure data integration and transformation, with data stored and managed on various AWS services.

•             Create and Maintain Cloud Environments: Design, implement, and maintain a comprehensive cloud environment on AWS, incorporating essential services like for application deployment, networking, and secure data workflows.

•             Infrastructure Automation and Management: Utilize Terraform to manage infrastructure as code (IaC), creating scalable, secure, and reproducible cloud environments on AWS. Manage and configure cloud infrastructure components for seamless identity management, networking, access control, and inter-service communication.

•             Database Management with SQL: Design and optimize SQL queries for efficient data retrieval, transformation, and analysis across relational databases, ensuring high performance in data workflows.

•             Data Security and Compliance: Implement data security protocols and best practices to safeguard data during storage and transfer, adhering to compliance requirements and ensuring data integrity.

•             API Development: Design and develop APIs using FastAPI  or equivalent and Pydantic or equivalent for data validation, supporting secure and efficient data exchanges between systems.

•             CI/CD and Automation: Implement CI/CD workflows with GitHub Actions or Jenkins to streamline deployment processes, ensuring efficient code integration and delivery across the team.

•             Containerization and Deployment: Deploy and manage Docker containers on AWS ECS and ECR, orchestrating data workflows for reliable, efficient processing.

•             Monitoring and Logging: Use AWS CloudWatch and other logging tools to monitor data workflows, identify and resolve issues, and ensure high availability and fault tolerance across distributed systems.

•             Collaborate with Cross-Functional Teams: Engage with other software developers, product managers, analysts, and other stakeholders to design and optimize data architectures that meet business analytics and reporting needs.

Qualifications

•             Bachelor's or master's degree in an engineering discipline

•             3+ years of professional experience in data engineering: Proven expertise in designing, developing, and maintaining data pipelines, with hands-on experience in data integration, ETL processes, and working with cloud-based data infrastructure. Strong ability to translate data requirements into scalable and efficient workflows that support business intelligence and analytics across teams.

•             3+ years of Cloud Computing experience with AWS.

•             2+ year of API development and integration

•             Proficiency in SQL for database management, querying, and optimization to support efficient data workflows and analytics.

•             Strong experience with Terraform for infrastructure as code (IaaC) to manage, scale, and secure cloud environments on AWS.

•             Proficiency with GitHub Actions for implementing CI/CD pipelines and automating deployment workflows.

•             Advanced skills in Python scripting for data transformation, API integration, and automation.

•             Knowledge of data security best practices, with experience implementing secure data handling and compliance protocols.

•             Hands-on experience with Docker containers, deploying and orchestrating them.

•             Strong understanding of CI/CD pipelines in GitHub for efficient code versioning and collaboration.

Preferred Skills:

·       AWS Certifications: Preferred certifications include AWS Certified Solution Architect Associate, Developer Associate, SysOps Administrator Associate, or Data Engineer Associate.

·       Familiarity with Apache Airflow for orchestrating complex data workflows.

·       Experience with FastAPI for developing and maintaining APIs, and Pydantic for data validation within those APIs.

·       Familiarity with the Energy Industry

·       Experience with data monitoring and logging tools, especially AWS CloudWatch.

In this role you will have an outstanding opportunity to add value and take on additional responsibilities commensurate with personal and professional growth over time. PEP offers a long term and rewarding career path for outstanding candidates. We’re looking for someone who is a great fit for our culture, which is hands-on, innovative, fast-paced, disciplined, and humble. This position may require occasional domestic and international travel as part of the job responsibilities.

We encourage you to apply even if you don’t meet all these qualifications and skills, but otherwise believe you would be a good fit.  A career at Private Energy Partners means you’ll have the opportunity to learn and practice new skills, explore interesting fields and do challenging work across our global affiliated portfolio companies. You will be able to make a meaningful impact on accelerating the green energy transition.

Our team is based in Pasadena, CA and candidates interested should be available to work out of the office in a hybrid setting.

Featured Benefits:

·       100% employee covered health insurance including medical insurance, vision insurance, and dental insurance.

·       401(k) with employer match up to 6%

·       Disability and life insurance