Black Canyon Consulting

Data Scientist

Black Canyon Consulting · Bethesda, MD
Bethesda, MD Posted 2026-10-20
Type
Full-time

Overview

We are seeking a Data Scientist to support applied machine learning, natural language processing, AI evaluation, data quality, and analytical product development for a federal permitting modernization initiative.

The Data Scientist will work across structured and unstructured data, including public comments, permitting records, compliance information, documents, and geospatially linked datasets. A key focus of the role will be developing representative evaluation datasets and quantitative methods to measure the performance, reliability, and limitations of AI-enabled and agentic solutions.

This position is remote.

Duties & Responsibilities

  • Develop analytical and machine-learning approaches supporting permitting and natural-resource use cases.
  • Build and maintain labeled datasets, evaluation corpora, and benchmark datasets for AI-enabled services.
  • Define quantitative evaluation methodologies for classification, extraction, retrieval, summarization, entity matching, and other AI/NLP capabilities.
  • Evaluate LLM and agentic workflows , including output quality, retrieval performance, task completion, failure modes, and reliability.
  • Analyze public-comment datasets and develop reusable features and evaluation datasets for comment-processing services.
  • Apply NLP and machine-learning techniques to structured and unstructured data.
  • Evaluate embedding models, semantic retrieval strategies, document-ranking techniques, and AI-generated outputs.
  • Develop statistical approaches for detecting anomalies, reconciliation issues, missing data, and inconsistent records.
  • Partner with Content Pipeline Engineers to measure OCR, extraction, and document-retrieval performance.
  • Partner with Data Engineers to ensure analytical datasets and evaluation pipelines are reproducible and production-ready.
  • Support entity matching and harmonization across data originating in multiple source systems.
  • Design experiments and validation approaches with clearly defined success metrics.
  • Contribute to certified analytical data products, metrics, and governed datasets.
  • Develop reproducible Python analyses, models, evaluation pipelines, and technical documentation.
  • Communicate model performance, limitations, uncertainty, and failure modes to technical and non-technical stakeholders.

Educational Requirements

  • Bachelor's degree in Data Science, Statistics, Computer Science, Mathematics, Engineering, or a related quantitative discipline.

Requirements

  • 5+ years of relevant data science, applied machine learning, or quantitative analytics experience.
  • Strong Python and SQL skills.
  • Experience with statistical analysis, machine learning, NLP, and unstructured text data.
  • Experience designing labeled datasets, benchmark datasets, and train/test evaluation frameworks.
  • Experience evaluating LLM, RAG, or agentic AI solutions , including retrieval quality, output accuracy, and failure modes.
  • Strong understanding of model-evaluation techniques, including precision, recall, F1, ranking metrics, confusion matrices, and related measures.
  • Experience with embeddings, semantic retrieval, or vector-based search.
  • Experience developing reproducible analytical and evaluation workflows.
  • Ability to distinguish data-quality issues from model-performance issues.
  • Strong ability to communicate analytical results, limitations, and uncertainty.

Bonus Skills

  • Experience with agent evaluation frameworks or LLM observability tools.
  • Experience with agentic frameworks such as LangChain, LangGraph, or similar orchestration frameworks .
  • Experience with OpenSearch, vector databases, or enterprise search platforms.
  • Experience with document AI or OCR pipelines.
  • Experience with geospatial analysis.
  • Experience developing entity-resolution, record-linkage, or data-quality models.
  • Experience working in the U.S. Federal Government space .
  • Experience with permitting, regulatory, environmental, energy, or natural-resource datasets.

Benefits and Salary

We attract the best people in the business with our competitive benefits package that includes medical, dental and vision coverage, 401k plan with employer contribution, paid holidays, vacation, and tuition reimbursement.

If you enjoy being a part of a high performing, professional service and technology focused organization, please apply today!

LLMPython
Black Canyon Consulting

Black Canyon Consulting

IT Consulting · Private · Fairfax, USA

Stage & Valuation
Private
Open roles on NewJob
Most hiring in
Engineering (32) · Operations (6) · BizOps (3)
Black Canyon Consulting provides innovative program management, data analytics, and IT support services primarily to Federal R&D programs. The company leverages proprietary technologies and a distributed workforce to deliver custom computer programming and systems design solutions.
IT Consulting Program Management Federal Contracting Data Analytics
E
Senior Network Engineer
Bethesda, MD Hybrid
Engineering
$110K–$160K
E
Python/Django Developer
Bethesda, MD
Engineering
E
Database Architect (Cloud) for SRA
Bethesda, MD
Engineering
See all 50+ roles at Black Canyon Consulting →
8
Lead Data Scientist - Relevancy Sciences
84.51° Cincinnati, OH
Data & ML
$125K–$207K
A
Senior Data Scientist
Asana Vancouver, Canada Hybrid
Data & ML
$176K–$200K
B
Product-Focused Data Scientist
Blue Rose Research Remote Remote
Data & ML
$160K–$210K
C
Data Scientist/Senior Data Scientist
C3.ai Redwood City, CA
Data & ML
$136K–$183K
See all Data & ML roles →

Interested in this role?

Apply directly on the company site — no recruiter middleman, no account required.

Apply now →
Apply on company site