Staff AI Engineer - Agent Evaluation Platform (x/f/m)
This listing is for a Staff AI Engineer - Agent Evaluation Platform (x/f/m) role at Doctolib, in Paris, France. It is listed as a full-time, staff/principal-level position.
DGJ summary of this job
Our own short read of this listing, built only from the facts it gives. The employer’s original text follows further down.
This listing is for a Staff AI Engineer - Agent Evaluation Platform (x/f/m) role at Doctolib, in Paris, France. It is listed as a full-time, staff/principal-level position.
Reading the advert, the employer asks for 6+ years of experience and English language skills.
Skills and tools named in the text include Python, machine learning and LLM.
No salary is given in this listing, so check the official pay figures in the DGJ Insight section below where they exist.
Benefits mentioned in the advert: health insurance and paid time off.
The advert does not mention visa sponsorship. If you would move to France for this job, check the official work permit route first (DGJ Insight lists it where we have it).
Dev Global Jobs found this job on Arbeitnow on Nov 21, 2026.
- Employer
- Doctolib
- Location
- Paris, France
- Job type
- Full-time
- Level (from the title)
- staff/principal-level
- Pay
- Not stated by the employer
- Experience asked
- 6+ years
- Languages asked
- English
- Found
- Nov 21, 2026 on Arbeitnow
- Employer text
- 858 words, about 4 min to read
- Apply only on the official page (arbeitnow.fr). Dev Global Jobs is not the employer and never asks for a fee.
- Compare your own experience with the 6+ years the advert asks for, and show it clearly near the top of your CV.
- Mirror the skills the advert names (Python, machine learning and LLM) in your CV where you really have them; applicant tracking systems look for these words.
- Say your level in English plainly (for example B2 or C1).
- Read the employer’s full text below (about 4 min) before you apply.
- Build or check your CV with the free DGJ CV Builder.
DGJ Insight
Our own data about this role and place, from official statistics and our live job index.
Source: OECD Average annual wages, 2025
UN living-cost index for Paris (ICSC post adjustment 49.5), as of 1 Nov 2026.
Source: ICSC post adjustment classifications (icsc.un.org)
- Talent Passport: Qualified employees, researchers, artists, investors; multi-year permit
- Employee (salarié) visa: Employer work authorisation from the labour office
- Temporary worker visa: Contracts under 12 months
About the role
Full text from the employer, as published on Arbeitnow, arranged into sections. Nothing added or removed.
Join a team of passionate and hardworking entrepreneurs to transform healthcare!
Working in the tech team at Doctolib involves building innovative products and features to improve the daily lives of care teams and patients. We work in feature teams in an agile environment, while collaborating with product, design, and business teams.
01What you'll do
We are looking for a Staff AI Software Engineer to join the AI Evaluation team (EvalX) at Doctolib and help every AI product team measure, understand and improve the quality, reliability and safety of their systems.
Our AI products span patient-facing conversational agents, AI-powered phone assistants and clinical assistants supporting healthcare professionals. EvalX builds the platform, tools and methods that let these teams evaluate continuously, and our ambition is to make evaluation a core engineering practice.
In this role, you will work at the intersection of platform engineering, applied AI and product teams: you build shared evaluation capabilities, and you work directly with the teams that use them so they actually get adopted.
In this role, you will:
Design, ship, and operate production-grade evaluation tooling (APIs, services, CI/CD integrations) used by AI product teams across Doctolib
Work hands-on with product teams to assess how they evaluate today, define improvement plans, and help them own their evaluation quality
Design evaluation strategies for agentic systems, combining deterministic checks, LLM-as-a-judge, simulation-based testing, and human review
Build intelligent capabilities for production sampling, dataset representativeness monitoring, distribution-shift detection, and test case generation
Help teams define meaningful quality metrics, representative datasets, and reliable quality gates throughout the development lifecycle
Turn recurring product needs into reusable, self-service platform capabilities, and measure success through adoption, team autonomy, and evaluation coverage
Contribute to cross-functional initiatives that shape Doctolib's AI platform and best practices for building and evaluating agentic systems
02Who you are
Before you read on, if you don't have the exact profile described below, but you feel this job description matches your skill set, we still encourage you to apply! You could be our next team mate if you:
Have 6+ years of experience as a software engineer, with significant experience using Python; or as a machine learning engineer
Have built or operated evaluation for LLM-powered or agentic systems: datasets, LLM-as-a-judge, human review, production-driven evaluation
Understand and promote software development best practices
Are product-oriented and enjoy working directly with other teams, challenging assumptions, and turning complex needs into concrete solutions
Can influence engineering practices and drive change without hierarchical authority: your job is not done until your solution is adopted
Are comfortable speaking and writing in English in your day-to-day with the team
Now, it would be fantastic if you:
Have experience with applied ML techniques such as embeddings, clustering, statistical sampling, and distribution monitoring
Have experience building developer platforms or internal tooling, or in a Forward Deployed Engineering role
Have experience in the medical domain or another regulated environment
03What we offer
Free comprehensive health insurance (basic package) for you and your children
25 days of paid vacation per year, plus up to 14 days of RTT
Free mental health and coaching services through our partner Moka.care
Work from abroad for up to 10 days per year thanks to our flexibility days policy
Lunch vouchers (Swile card) worth €8.50 per working day, with €4.50 covered by Doctolib
A subsidy from the work council to refund part of the membership to a sport club or a creative class
50% reimbursement of your public transport subscription
Enrollment in Doctolib's long-term employee value sharing plan called DoctoGrowth
ParentCare Program: Enjoy full salary coverage (100%) during your first month of birth leave, and 75% during the second, covered by Doctolib
For caregivers and workers with disabilities, a package including an adaptation of the remote policy, extra days off for medical reasons, and psychological support
04The interview process
A first interview with a member of the Talent Acquisition team
A coding interview
An agentics system design
A behavioural interview
Reference check
05Job details
Permanent position
Full Time
Workplace: Levallois
Start date : As soon as possible
If you would like to find out more about tech life at Doctolib, feel free to read our latest Medium blog articles!
At Doctolib, we believe in improving access to healthcare for everyone - regardless of where you come from, what you look like. This translates into our recruitment process: Doctolib is an equal opportunity employer. We don't just accept diversity at Doctolib, we respect and celebrate it!
The more diverse ideas are heard, the more our product will truly improve healthcare for all. You are welcome to apply to Doctolib, regardless of your gender, religion, age, sexual orientation, ethnicity, disability, or place of origin. If you have a disability, let us know if there's any way we can make the interview process smoother for you!
All the information transmitted via this form is processed by Doctolib for the purpose of managing applications. For more information on how Doctolib processes your application data, click here.
If you wish to exercise your rights or if you have any questions about the processing of your data, you can write to us at ----- Find more English Speaking Jobs in France on Arbeitnow
Ask CSI about this job
Answers link to jobs and tools on Dev Global Jobs.
Review Dev Global Jobs
Used this site to find a job? Tell other job seekers what worked and what did not.