Job Title
This role focuses on hands-on optimization of an existing Retrieval Augmented Generation (RAG) application. The core architecture and infrastructure are already in place and largely functional, and you will refine retrieval accuracy, response quality, and overall performance for a customer-facing knowledge application used by a product support organization. This is a technical execution position suited to an Azure AI Engineer, Applied AI Engineer, Machine Learning Engineer, Search/Relevance Engineer, or Software Engineer with direct experience building and tuning RAG solutions in Azure, rather than an AI architect or strategy-focused role.
Responsibilities
- Optimize and enhance an existing Azure AI Foundry RAG implementation to improve reliability and performance.
- Improve retrieval accuracy, ranking, reranking, and context selection to deliver more relevant responses.
- Configure and refine document ingestion pipelines, including chunking strategies, metadata tagging, indexing approaches, and search optimization techniques.
- Evaluate and tune development and staging environments to improve response performance and stability.
- Work with large document sets such as manuals, specifications, and technical documentation to improve retrieval effectiveness for product support use cases.
- Enhance prompt engineering practices and context window strategies to increase first-response accuracy.
- Collaborate with business and technical stakeholders to implement a backlog of planned enhancements and iterative improvements.
- Test and validate retrieval and response quality using real-world customer support scenarios and use cases.
- Continuously monitor, analyze, and refine RAG pipeline components to maintain high-quality search relevance and AI responses.
- Document changes, configurations, and best practices to support maintainability and knowledge sharing within the team.
Skills
- Hands-on experience working with Azure AI Foundry in a production or near-production environment.
- Proven experience building, tuning, or optimizing Retrieval Augmented Generation (RAG) solutions.
- Strong understanding of document ingestion processes, including chunking strategies, metadata tagging, indexing, ranking, and reranking.
- Demonstrated ability to improve search relevance and retrieval accuracy within AI-driven applications.
- Knowledge of vector search, semantic search, and retrieval optimization concepts and techniques.
- Experience testing and validating prompt strategies and AI response quality using realistic use cases.
- Ability to work independently within an established environment and deliver targeted enhancements without extensive supervision.
- Technical experience as an Azure AI Engineer, Applied AI Engineer, Machine Learning Engineer, Search/Relevance Engineer, or Software Engineer with practical experience in RAG solutions.
Additional Skills & Qualifications
- Experience supporting customer service or knowledge management AI solutions, particularly for product support organizations.
- Experience with Azure Cognitive Search or Azure AI Search for advanced search and retrieval scenarios.
- Familiarity with large language model (LLM) evaluation frameworks and methodologies for assessing response quality.
- Experience working with technical documentation, manuals, and product support content in AI and search contexts.
- Strong collaboration skills to work effectively with both technical and non-technical stakeholders.
- Attention to detail and a data-driven approach to measuring and improving retrieval performance
Job Type & Location: This is a Contract position based out of Milwaukee, WI.
Pay and Benefits: The pay range for this position is $65.00 - $85.00/hr. Eligibility requirements apply to some benefits and may depend on your job classification and length of employment. Benefits are subject to change and may be subject to specific elections, plan, or program terms. If eligible, the benefits available for this temporary role may include the following: • Medical, dental & vision • Critical Illness, Accident, and Hospital • 401(k) Retirement Plan – Pre-tax and Roth post-tax contributions available • Life Insurance (Voluntary Life & AD&D for the employee and dependents) • Short and long-term disability • Health Spending Account (HSA) • Transportation benefits • Employee Assistance Program • Time Off/Leave (PTO, Vacation or Sick Leave)
Workplace Type: This is a fully onsite position in Milwaukee,WI.
Application Deadline: This position is anticipated to close on Aug 16, 2026.
About Actalent: Actalent is a global leader in engineering and sciences services and talent solutions. We help visionary companies advance their engineering and science initiatives through access to specialized experts who drive scale, innovation and speed to market. With a network of almost 20,000 consultants and 5,000 clients across the U.S., Canada, Asia and Europe, Actalent serves many of the Fortune 500. We are proud to be an Engineering News-Record (ENR) Top 500 Design Firm for our engineering design services and a ClearlyRated Best of Staffing® winner for both client and talent service.
The company is an equal opportunity employer and will consider all applications without regard to race, sex, age, color, religion, national origin, veteran status, disability, sexual orientation, gender identity, genetic information or any characteristic protected by law.
If you would like to request a reasonable accommodation, such as the modification or adjustment of the job application process or interviewing process due to a disability, please email for other accommodation options.
San Francisco Fair Chance Ordinance: Pursuant to the San Francisco Fair Chance Ordinance, for all positions located in the city and county of San Francisco, we will consider for employment qualified applicants with arrest and conviction records.
Massachusetts Lie Detector: It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.
Use of Artificial Intelligence (AI): We may use Artificial Intelligence (AI) to support parts of our hiring process, including sourcing, screening, and evaluating candidates. AI helps assess applications and qualifications, but final decisions are made by our hiring team. By applying, you acknowledge and agree that your application may be reviewed using AI tools.