Senior Data Engineer
Please Note
This is L3 support project
Flexibility to work in rotational shift is mandatory
- CI/CD Pipeline, CSS/SCSS/LESS, PyTest, PostgreSQL / MongoDB / Redis, REST API / GraphQL
- Python Fullstack with Microservices: Software Development Lead
•
We are seeking an experienced Lead Data Engineer to take end-to-end ownership of Enterprise Data Warehouse (EDW) solutions. This role will be responsible for designing, architecting, and optimizing scalable, secure, and high-performance data pipelines using SAP BODS, IBM DataStage, Redwood, SQL, Oracle, and modern data platforms.
The ideal candidate will combine strong hands-on technical expertise with leadership capabilities to drive data engineering best practices and deliver enterprise-grade data solutions aligned with business objectives.
Key Responsibilities
1. EDW Architecture & Data Engineering
- Design and implement scalable data integration and transformation frameworks for the Enterprise Data Warehouse.
- Architect and optimize ETL/ELT pipelines using SAP BODS and IBM DataStage.
- Develop and maintain robust data models (Star/Snowflake schemas, SCD implementation).
- Ensure high availability, reliability, and performance of data pipelines.
- Implement data governance, security, and compliance standards.
2. ETL Development & Optimization
- Develop complex data workflows and transformations.
- Perform performance tuning of ETL jobs and SQL queries.
- Optimize batch processing and scheduling using Redwood.
- Implement error handling, logging, and monitoring mechanisms.
- Manage incremental loads and data reconciliation processes .
3. Database & SQL Development
- Write and optimize complex SQL queries, stored procedures, and functions.
- Work extensively with Oracle databases and related technologies.
- Design and manage indexing strategies and query optimization.
- Support database performance analysis and tuning.
4. Leadership & Team Management
- Lead and mentor a team of data engineers.
- Conduct code reviews and enforce development standards.
- Drive technical best practices and architecture governance.
- Collaborate with cross-functional teams including BI, Analytics, and Application teams.
- Participate in sprint planning, estimation, and technical roadmap discussions.
5. Stakeholder Collaboration
- Work closely with business stakeholders to gather and translate requirements into technical solutions.
- Provide technical guidance and solution recommendations.
- Ensure alignment between enterprise data strategy and business goals.
- Support production issue resolution and root cause analysis.
Required Skills
Technical Expertise
- Strong experience with SAP BODS
- Hands-on experience with IBM DataStage
- Experience with Redwood scheduling
- Advanced SQL development
- Strong expertise in Oracle database
- Deep understanding of Enterprise Data Warehouse architecture
Data Engineering Concepts
- Data modeling (Star/Snowflake schema, SCD Types)
- ETL/ELT best practices
- Data quality frameworks
- Performance tuning and optimization
- Data security and governance standards
Preferred / Good to Have
- Experience with modern cloud data platforms (Azure, AWS, GCP)
- Knowledge of Snowflake, Databricks, or Synapse
- CI/CD implementation for data pipelines
- Exposure to data cataloging and metadata management tools
- Experience in Agile/Scrum methodologies
Qualifications
- Bachelor’s or Master’s degree in Computer Science, Information Systems, or related field.
- 5+ years of experience in Data Engineering.
- 2+ years of experience in leading technical teams.
Key Competencies
- Strong architectural and analytical skills
- Leadership and mentoring capability
- Excellent communication and stakeholder management
- Problem-solving mindset
- Ability to drive enterprise-scale data initiatives
Brillio
IT Consulting · Private · Santa Clara, USA