Member of Technical Staff (Web Access Engineer, Crawler)
Perplexity is hiring for the position of Member of Technical Staff (Web Access Engineer, Crawler) in London, United Kingdom. The listing describes it as a full-time, staff/principal-level position.
DGJ summary of this job
Our own short read of this listing, built only from the facts it gives. The employer’s original text follows further down.
Perplexity is hiring for the position of Member of Technical Staff (Web Access Engineer, Crawler) in London, United Kingdom. The listing describes it as a full-time, staff/principal-level position.
From the advert, the main requirements are 5+ years of experience.
The advert mentions Python, JavaScript, Rust and C++ among the skills and tools for this role.
The employer does not state a salary in this listing; DGJ Insight below shows an official pay benchmark for the place where one exists.
Visa sponsorship is not mentioned. Applicants from outside United Kingdom should check the official work permit route before applying.
Dev Global Jobs found this job on Arbeitnow on Nov 14, 2026.
- Employer
- Perplexity
- Location
- London, United Kingdom
- Job type
- Full-time
- Level (from the title)
- staff/principal-level
- Pay
- Not stated by the employer
- Experience asked
- 5+ years
- Found
- Nov 14, 2026 on Arbeitnow
- Employer text
- 346 words, about 2 min to read
- Apply only on the official page (arbeitnow.co.uk). Dev Global Jobs is not the employer and never asks for a fee.
- Compare your own experience with the 5+ years the advert asks for, and show it clearly near the top of your CV.
- Mirror the skills the advert names (Python, JavaScript, Rust and C++) in your CV where you really have them; applicant tracking systems look for these words.
- Read the employer’s full text below (about 2 min) before you apply.
- Build or check your CV with the free DGJ CV Builder.
DGJ Insight
Our own data about this role and place, from official statistics and our live job index.
Source: OECD Average annual wages, 2025
UN living-cost index for United Kingdom (ICSC post adjustment 76.4), as of 1 Oct 2026.
Source: ICSC post adjustment classifications (icsc.un.org)
- Skilled Worker visa: Job offer from a licensed sponsor at the required skill and salary level; English requirement
- Health and Care Worker visa: Eligible health and social care roles with a licensed sponsor
- Global Talent visa: Endorsement as a leader or potential leader in a listed field
Official routes: UK Visas and Immigration (GOV.UK)
About the role
Full text from the employer, as published on Arbeitnow, arranged into sections. Nothing added or removed.
Perplexity is hiring a Web Access Engineer for the Crawler team to build reliable fetching and rendering systems for AI-powered search. You'll work on freely available public web content, within site permissions and approved access policies, and develop the infrastructure that supplies content to downstream indexing and retrieval systems.
Your focus will be diagnosing network errors, page-loading issues, and incomplete rendering. You'll trace problems from network protocols through client-side execution, distinguish technical failures from intentional access restrictions, and turn findings into production fixes that improve content quality and efficiency at scale.
01Responsibilities
Own fetching and rendering improvements end to end, from investigation through implementation and production rollout, partnering with crawler engineers and SREs.
Investigate fetch and rendering failures across DNS, TLS, HTTP, caching, redirects, and client-side rendering, and classify root causes.
Improve page loading, completion detection, and content capture to preserve relevant content and handle dynamic pages reliably.
Ensure the crawler respects site-imposed access restrictions and rate limits, identifies itself clearly, and retries only transient technical errors.
Build tools to reproduce failures, compare crawler output against expected page content, and turn findings into regression tests.
Define metrics for fetching reliability, content completeness, and error classification, and use them to prioritize improvements.
02Qualifications
5+ years of software engineering experience in web infrastructure, browsers, CDNs, or crawling, with strong programming skills in at least one of Go, Python, Rust, or C++.
Deep understanding of HTTP/1.1, HTTP/2, and HTTP/3, including status semantics, caching, conditional requests, content encoding, and redirects; strong knowledge of TLS, DNS, cookies, and sessions.
Understanding of CDN caching and content delivery, HTTP error handling, retry backoff, and server-defined rate limits.
Knowledge of page lifecycle, network idle, service workers, client-side rendering, and shadow DOM; experience debugging JavaScript and fluency with Chrome DevTools, CDP, HAR files, curl, and packet captures.
Comfort working independently when the path forward is not fully defined.
03Nice to have
Experience developing CDN, caching, or web performance systems.
Experience operating crawlers, headless browsers, or distributed content-acquisition systems at scale.
Contributions to a browser engine or JavaScript runtime.
Find Jobs in United Kingdom on Arbeitnow
Ask CSI about this job
Answers link to jobs and tools on Dev Global Jobs.
Review Dev Global Jobs
Used this site to find a job? Tell other job seekers what worked and what did not.