- Location
- San Francisco, CA, United States · Onsite
- Compensation
- $200k – $250k • No equity
- Employment
- Full-time
- Experience level
- MID (2–5 years exp)
- Specialization
- AI Engineer, Machine Learning Engineer, Wellfound
- Source
- Wellfound
Responsibilities
- Build agent harnesses and measure them against each other: memory strategies, context budgeting, tool interface design, and the protocols by which agents hand work to one another
- Debug agent failures at the trajectory level and resolve them in the harness
- Build and maintain the retrieval layer between the agents and the knowledge graph, including query planning, schema-aware retrieval, and reranking
- Implement schema validation, provenance tracking, and rejection handling at the write boundary
- Implement inference methods for incomplete and conflicting evidence, including belief revision, uncertainty propagation, and abstention
- Build extraction pipelines that convert publications, patents, and instrument output into structured records
- Integrate third-party scientific databases and reconcile conflicts between sources
- Construct benchmarks for scientific reasoning tasks, including task selection, labeling strategy, and scoring methodology
- Own the automated evaluation harness used to gate model and prompt changes
- Work directly with pilot users to identify failure modes and translate them into system changes
Required qualifications
- Demonstrated work building multi-step language model agents that have run under real users OR in applying LLMs to perform complex tasks
- Experience designing evaluations for LLM systems, including benchmark construction and model-based grading
- Strong Python, with experience building and operating production backend services
- Experience modeling a complex domain in a typed schema, relational database, or graph database
- Experience with retrieval systems beyond embedding similarity search
- Ability to read primary literature in biology or chemistry, or clear interest in developing it
Preferred qualifications
- Experience with multi-agent orchestration and MCP
- Experience with probabilistic programming, Bayesian inference, or calibration measurement
- Experience with non-monotonic logic, truth maintenance, or other defeasible reasoning systems
- Experience with information extraction from unstructured documents, including table and layout parsing
- Experience with reinforcement learning or finetuning of language models
- Background in cheminformatics, including RDKit, SMILES normalization, or optical chemical structure recognition
- Background in chemical biology, targeted protein degradation, or mass spectrometry proteomics
- Experience with scientific data infrastructure such as ELN, LIMS, or FAIR data standards