About the Role
As a Principal Engineer, you will be responsible for the architectural design and end-to-end implementation of a high-performance OLAP query engine and semantic layer. The primary goal is to develop a custom JSON-based query language (DSL) and a translation engine that converts these requests into optimized SQL for columnar databases. This role requires a deep understanding of systems programming, compiler theory, and multi-dimensional data modeling to serve high-concurrency read loads in a microservices environment.
What You'll Do
- Architectural Design & Query Execution
- Compiler & DSL development: Design a custom JSON-based OLAP query language. Define eBNF grammars and build a translation layer to convert functional formulas and DSL requests into an optimized execution plan.
- SQL generation & optimization: Implement and oversee the generation of highly optimized SQL tailored for columnar databases including ClickHouse, SingleStore, and DuckDB.
- Semantic layer management: Architect the representation of multi-dimensional data models within columnar structures, ensuring efficient data retrieval and aggregation.
- Custom function logic: Implement logic for custom functions, determining the optimal balance between SQL push-down and application-layer execution
- Caching & Performance Engineering
- Multi-dimensional caching: Envisage and implement a distributed caching strategy for multidimensional data, utilizing shared memory systems like Redis.
- Concurrency & scaling: Build and maintain high-performance microservices using Java (Micronaut or Spring Boot) with Project Loom (Virtual Threads) to handle massive concurrent read loads.
- Low-level optimization: Manage system-level performance including connection pooling, array pooling, and off-heap memory management.
- JVM tuning: Apply expert knowledge of Java Garbage Collectors (including ZGC and Shenandoah) to maintain low-latency response times.
- Engineering Excellence & AI Integration
- AI-augmented development: Leverage AI coding agents and tools to accelerate the software development lifecycle, from prototyping to debugging.
- Technical documentation: Maintain a disciplined approach to software development by documenting architectural decisions, design patterns, and technical specifications.
- Technical leadership: Act as the primary technical authority on query engine internals, providing guidance on complex systems-level challenges.
What We're Looking For
- 15+ years in software engineering with a focus on backend systems and data infrastructure
- Strong theoretical and practical knowledge of compilers, ASTs, eBNF, and query planning
- In-depth understanding of SQL, query optimization, and the internal workings of columnar databases
- Expert proficiency in Java, specifically regarding multi-threading, asynchronous I/O, networking, and memory management
- Proven experience in designing distributed caching layers and managing state across microservices
- Ability to decompose complex data problems into scalable architectural components
- Clear verbal and written communication skills, with a strong tendency to document thoughts and architectural ideas
- A rigorous approach to software craftsmanship, including testing, telemetry, and performance benchmarking
- Minimum of 1 year of experience using AI coding agents to enhance development productivity
- Prior experience developing financial systems such as ERP, EPM, or General Ledger platforms is an added plus
Why this Role
You will own the internals of the query engine at the heart of a next-generation financial planning platform — the layer where compiler theory, columnar database performance, and multi-dimensional semantics converge.
Pay Transparency Statement
At Planful our top priority is to support our employees and ensure each individual has access to equal growth opportunities. Pay decisions take into consideration the individual's skillset, experience, knowledge, qualifications for the role, and specific work location.
The OTE range for this role is $325,000 - $375,000 USD per year, depending on experience and location. As a part of this role, you are also eligible for equity, benefits and wellbeing resources.