Key Responsibility:

• Design & Implement Data Platforms: Design, develop, and maintain robust, scalable data pipelines and ETL processes, with a focus on automation and operational excellence.
• Ensure Data Quality and Governance: Implement automated data validation, quality checks, and monitoring systems to ensure data accuracy, consistency, and reliability.
• Manage CI/CD for Data: Own and optimize the CI/CD pipelines for data engineering workflows, including automated testing and deployment of data transformations and schema changes.
• Architect & Implement IaC: Use Infrastructure as Code (IaC) with Terraform to manage data infrastructure across various cloud platforms (Azure, AWS, GCP).
• Performance & Optimization: Proactively monitor and optimize query performance, data storage, and resource utilization to manage costs and enhance efficiency.
• Collaborate with Stakeholders: Manage communication with technical and business teams to understand requirements, assess technical and business impact, and deliver effective data solutions.
• Strategic Design: Possess the ability to see the big picture in architectural design, conduct thorough risk assessments, and plan for future scalability and growth.

Qualifications:

• Experience: 1-3 years of experience in data engineering, data warehousing, and ETL processes, with a significant portion of that time focused on DataOps or a similar operational role.

• Platform Expertise: Strong experience with data platforms such as Databricks and exposure to multiple cloud environments (Azure, AWS, or GCP).

• Data Processing: Extensive experience with Apache Spark for large-scale data processing.

• Orchestration: Experience working with data orchestration tools like Azure Data Factory (ADF), Apache Airflow, or similar.

• CI/CD & Version Control: knowledge of version control (Git) and experience with CI/CD pipelines (GitLab CI/CD, GitHub Actions).

• IaC: hands-on experience with Terraform.

• Programming: Programming skills in Python and advanced proficiency in SQL

• Soft Skills: Strong stakeholder management, communication, and collaboration skills. The ability to articulate complex technical concepts to non-technical audiences is a must.

• Problem-Solving: Strong problem-solving skills with an ability to analyze technical challenges and their business impact.

• Data Modeling: Experience with data modeling tools and methodologies, specifically with dbt (data build tool).

• AI & ML: Experience with AI-related technologies like Retrieval-Augmented Generation (RAG) and frameworks such as LangChain.

• Data Observability: Hands-on experience with data quality and observability tools such as Great Expectations, Monte Carlo, or Soda Core.

• Data Governance: Familiarity with data governance principles, compliance requirements, and data catalogs (e.g., Unity Catalog).

• Streaming Technologies: Experience with stream processing technologies like Kafka or Flink.

• Containerization: Familiarity with containerization and orchestration tools (e.g., Docker, Kubernetes).

• Open Source: Contributions to open-source projects or relevant certifications.

CP AXTRA | Lotus's
CP AXTRA Public Company Limited.
Nawamin Office: Buengkum, Bangkok 10230, Thailand

“By applying for this position, you consent to the collection, use and disclosure of your personal data to us, our recruitment firms and all relevant third parties for the purpose of processing your application for this job position (or any other suitable positions within Lotus's and its subsidiaries, if any). You understand and acknowledge that your personal data will be processed in accordance with the law and our policy."