Application Architect - DARPA SciFy Project - Penn EngineeringWe are seeking an Application Architect with expertise in NLP and large language models to lead development on the DARPA SciFy project. The University of Pennsylvania leads a multi-university team building AI systems for automated scientific claim assessment and technology forecasting, in partnership with Two Six Technologies. The architect will design and build the core infrastructure for ClaimSpy, our flagship claim feasibility assessment system, extending it from verifiable materials science claims to speculative AI technology forecasting — supporting DARPA's "create and avoid surprise" mission.
This is a 12-month grant funded position with the possibility of an extension dependent on continued funding.Job ResponsibilitiesDesign and build ClaimSpy's agent architecture: multi-LLM pipelines with tool-calling chains for literature retrieval, computational analysis, and claim assessmentDevelop an interactive Claim Card interface for analysts to construct, refine, and explore structured speculative claims, inspect reasoning traces, and review evidence bundlesExtend the Panel of Virtual Experts (PoVE) system to support Tech Council in a Box (TCBX), enabling multi-perspective deliberation over speculative AI forecastsIntegrate Two Six Technologies' speculative claims component into the ClaimSpy pipeline, including ablation studies to assess integration qualityInstrument systems with logging, observability, and error tracing across tool-calling chains to support operational pilot deploymentsCollaborate with researchers on AutoRubric integration for automated LLM-as-judge evaluation of claim assessmentsConform to DARPA's testing and evaluation frameworkCollaborate with postdocs and graduate students to translate research prototypes into production-ready systemsAssist with technical documentation, system testing, and monthly DARPA reportingParticipate in program meetings, technical demonstrations, and pilot exercises with transition partnersQualificationsBachelor's in Computer Science with four to six years of substantial software development experience or an equivalent combination of education and experience.Strong Python skills and hands-on experience with LLM APIs (OpenAI, Anthropic, Google) and agent frameworksExperience building agentic systems with tool use, multi-step reasoning, and orchestrationExperience developing APIs that expose LLM capabilities as services (REST/GraphQL endpoints, streaming, authentication, rate limiting)Experience with API development, system integration, and full-stack web developmentAbility to rapidly prototype and iterate in an agile research environmentStrong communication skills for coordinating across a distributed team and with government stakeholdersPreferredExperience with RAG systems, vector databases, and retrieval pipelinesExperience building interactive data exploration or sensemaking interfacesFamiliarity with scientific literature processing and structured knowledge extractionContainerization and deployment (Docker, Kubernetes)Open source contributionsExperience with evaluation frameworks, benchmarking, or LLM-as-judge pipelinesPrior experience on government-funded research projectsJob Location - City, StatePhiladelphia, PennsylvaniaDepartment / SchoolSchool of Engineering and Applied SciencePay Range$100,000.00 - $150,000.00 Annual RateSalary offers are made based on the candidate's qualifications, experience, skills, and education as they directly relate to the requirements of the position, and in alignment with salary ranges based on external market data for the job's level. Internal organization and peer data at Penn are also considered.Equal Opportunity StatementThe University of Pennsylvania is an equal opportunity employer. Candidates are considered for employment without regard to race, color, sex, sexual orientation, religion, creed, national origin (including shared ancestry or ethnic characteristics), citizenship status, age, disability, veteran status or any class protected under applicable federal, state or local law.Special Requirements Background checks may be required after a conditional job offer is made.
Consideration of the background check will be tailored to the requirements of the job.