Design and implement scalable data platforms using Snowflake, Databricks, Delta Lake, and cloud technologies.
Build batch and real-time data pipelines using PySpark, Kafka, and Spark Structured Streaming.
Develop AI-ready data architectures supporting analytics, ML, LLMs, and RAG applications.
Design semantic models, data governance, metadata, and data lineage solutions.
Implement vector databases, embedding pipelines, and retrieval solutions for AI applications.
Build and manage ML/LLMOps pipelines, model deployment, monitoring, and CI/CD.
Ensure data security, RBAC, compliance, and governance across the platform.
Mentor engineering teams and define architecture best practices.
Required Skills
12+ years of experience in Data Engineering/Data Architecture.
Strong experience with **Snowflake, Databricks, PySpark, Kafka, Delta Lake, SQL, and Python**.
Hands-on experience with **AWS (S3, Glue, Redshift, Bedrock, Kinesis)** or Azure.
Experience with **LangChain, LlamaIndex, OpenAI/Bedrock, RAG, Vector Databases (Pinecone, ChromaDB, FAISS, OpenSearch)**.
Good understanding of ML/LLMOps, Data Governance, Data Lineage, and CI/CD.
Excellent communication and stakeholder management skills.
At 3Pillar, we create an environment where people can do their best work while maintaining a healthy work-life balance.
- Flexibility & Well-being – Our remote-first approach gives you the flexibility to work where you perform best, while prioritizing your well-being and personal commitments.
- Global Community – Collaborate with talented colleagues across the globe in a culture built on connection, support, and shared success.
- Your Voice Matters – We foster open communication and multiple feedback channels, ensuring every employee has the opportunity to be heard and make an impact.
- Growth & Development – Gain exposure to diverse clients, industries, and challenges that accelerate learning and career growth.
Our culture is guided by four core values: Collaboration, Outperform, Respect, and Evolve —the principles that shape how we work, grow, and succeed together.