Experience:

3 years required

Skills:

Architecture, Automation, Big Data, Power BI, Python, Oracle, Hadoop, Apache, SQL, ETL

Job type:

Full-time

• Design, develop, and maintain robust data pipelines using Apache Airflow, SSIS, PySpark, and Talend to ingest, transform, and load large-scale datasets (300M+ rows) from multiple sources.
• Manage and optimize relational databases including Oracle and SQL Server, ensuring high performance and scalability for analytical workloads.
• Design, build, and maintain cloud data platforms and warehousing solutions using AWS services such as Redshift and S3, alongside ClickHouse for high-performance analyti ...
• Develop and maintain OLAP solutions using SSAS for advanced reporting and analytics, including cube design, partitioning, and aggregation strategies.
• Implement ETL processes for structured and semi-structured data using tools such as Talend and PySpark, ensuring data quality, consistency, and compliance with industry standards.
• Collaborate with BI team, Actuarial Team and business stakeholders to understand requirements and deliver efficient data models and solutions.
• Monitor and improve data pipeline performance, troubleshoot bottlenecks, and implement best practices for parallel processing and resource optimization.
• Ensure data security and compliance with insurance industry regulations and company policies.
• Leverage big data technologies such as Apache Spark for distributed processing and advanced data transformations.
• Automate workflows and scheduling using Airflow DAGs, ensuring reliability and scalability of data operations.
• Document data architecture, processes, and standards, and provide technical guidance to team members.
• Bachelor's degree in Computer Science, Computer Engineering, Information Technology, or a related field.
• 3+ years of experience building and maintaining data pipelines.
• Proficiency in SQL and experience with relational databases (Oracle, SQL Server) including performance tuning and query optimization.
• Hands-on experience with ETL tools such as SSIS, Talend, and PySpark, and data modeling for OLAP solutions using SSAS.
• Knowledge of data pipeline orchestration using Apache Airflow.
• Experience with big data processing frameworks such as Apache Spark for distributed data transformations.
• Hands-on experience with cloud database/data platform services such as AWS Redshift, S3, and ClickHouse.
• Solid understanding of data warehousing concepts, star/snowflake schema, and partitioning strategies for large datasets (300M+ rows).
• Familiarity with insurance industry data or other regulated environments (preferred).
• Proficiency in Python or another scripting language for automation and data manipulation.
• Knowledge of data quality, governance, and security best practices.
• Ability to work with large-scale data and optimize workflows for performance and scalability.
• Preferred Qualifications.
• Experience with Power BI and building efficient data marts.
• Experience with data privacy, governance, and secure data access practices.
• Hands-on with cloud platforms or tools such as AWS (Redshift, S3), ClickHouse, Azure, Databricks, Delta Lake, Snowflake, or Hadoop ecosystem.
• Able to troubleshoot performance issues in both backend pipelines and frontend analytics tools.
• Comfortable working in a fast-paced, high-ownership environment with cross-functional teams.