agentnative

The last engineering roles standing before humans largely disappear from software development.

Agent engineer ads require agent experience in 49% of roles. Another 30% list it only as a preference. We read the requirement lines of 242 agent engineer roles on our board and counted each skill once. A skill was required when any hard requirement line named it, and preferred only when nothing but the preferred lines did. Python, TypeScript and backend engineering behave like filters whenever they appear. Building agents splits: a filter for some teams and a tiebreaker for many others. We read that split as a sign of how young the work is. Our reading is that engineers with long agent experience in production are still scarce, so a team that makes it a hard line risks a small pool, while many teams hire strong engineers who can learn the loop and keep agent work as what moves one of them ahead. LLM evals sit the same way, required in 36% of these roles and preferred only in 19%. So one application has two jobs. In our reading, the requirements decide whether the person screening keeps reading, and the preferences decide who stands out once several applicants clear the bar. An application that answers only the requirements reads like the rest of the pile, and one that opens with a clever side project while skipping a stated requirement risks never being read to the end. Hiring teams face the same choice from the other side. Moving agent experience into the requirements screens out the backend engineer who could learn the loop on the job, and leaving it among the preferences works best when the ad says what would count as evidence. Three things the ads on our board say to look for and show: - Check whether the ad names agents among its requirements or its preferences, because that decides what leads your application. - Answer every stated requirement first, in the ad's own words, with one line of evidence for each. - Show one small agent you built for the preference, such as a loop of about 100 lines with stop rules, a dollar cap and a log of every request, which Chapter 2 of Harness Engineering walks through for free. A requirement line is the ad's wording on paper, so these shares count what teams ask for in writing and say nothing about whom they end up hiring. We would write every agent application as two answers, one to the filter and one to the preference, because the ads on our board use agent experience as both and a single answer leaves one of them unmet.

Same model, strongest harness. Same model, weakest harness. On Terminal-Bench 2.0, 18.4 points separate the two scores. The free Chapter 1 of our book Harness Engineering, The Harness Is What You Ship, gathers one model's published results on that benchmark across the agent harnesses built around it. The model stays the same throughout. What changes is the tools it can call, what goes into its context and the checks on its output, and those choices move the score on their own. The chapter is careful about how far that goes. Counted on verified results alone, the spread shrinks to 4.9 points; the benchmark's own authors write that choosing the model usually matters more, and METR found that product harnesses did not beat simple scaffolds. Even on that cautious reading, a team holding its model fixed still has points to win or lose in the code around it. We read this as the reason agent teams hire the way they do. Our reading is that a team building on a model it did not train owns the harness and little else: which tools the agent gets and how those tools fail, what the window holds, when a run gets checked and when it stops. In our view, that is the code its engineers write, review and debug week after week. The chapter's sharpest example is how quickly that work dates. It records that Anthropic cut Claude Code's system prompt by more than 80% for Opus 5. In our reading, instructions written around one model's weaknesses had become clutter for the next, so every component in a harness carries an assumption someone has to revisit. That gives an engineer moving into agent work something specific to build and show. Hold one model fixed, change one thing around it, a tool description, a retrieval step or a verification pass, and report the score before and after on the same tasks. A project like that answers the question we expect an agent interview to ask sooner or later: what did you change, and how do you know it helped. In a hiring manager's seat, we would test the same skill directly: hand the candidate a fixed model, a failing agent and its traces, and ask what they would change around the model first. Our view is that an ad listing model names and frameworks draws people who have used them, while an ad naming tool design, context and checks draws the people who build them. We think candidates should prepare for the harness and teams should hire for it, because on any given model it is the part of the score a team controls.

20% of inference roles on our board involve Triton. Across all AI roles, 3% do. That gap is one of the clearest signs in the roles we track that inference engineering has become a job of its own. Of the 97 inference roles open on our board, 66% involve GPU optimization, a skill most AI ads leave alone. The work these teams describe sits beneath the model API: how requests are batched, how memory is spent on a GPU, how fast a kernel runs. We read the reason as cost. A team serving open models pays for every token in GPU hours, so a kernel that wastes memory bandwidth turns into a bigger bill and a slower product. Our book GPU Architecture and Programming argues that most performance problems in deep learning are memory problems in disguise, because the chip does arithmetic far faster than it can fetch the numbers. An engineer who can tell whether a workload is bound by compute or by memory is the one who can make it cheaper. The ordinary platform work stays in these ads too. Inference roles ask for LLM app infrastructure about as often as AI roles in general, so in our reading the kernel skills arrive on top of it. Say you are three years into backend work and have built products on hosted model APIs. That experience counts, and the ads we read ask for more beneath it. What sets an application apart is one piece of work below the API: a fused kernel written in Triton, a profile showing where a serving path stalled on memory, a throughput figure before and after a change. In the interview, expect to be asked why a model server runs out of memory before it runs out of compute, and answer with numbers. Teams hiring for inference face a thin pool for the same reason. In our reading, many engineers who have written kernels come from graphics, high-performance computing or compiler work, and an ad that asks for years of LLM experience first screens them out. Naming Triton or kernel work in the requirements tells those engineers the role is for them. The inference group is small, 97 roles, and an ad that names Triton says nothing about how deep the kernel work goes. Even so, these ads point the same way, below the API, where general AI ads rarely go. If we were aiming at inference work, we would put one measured kernel or serving change at the top of the CV, because it is the evidence these ads ask for and, in our reading, few applicants can show.

If you have spent years shipping backend or infrastructure systems, AI platform roles are asking for what you already have. On our board, 2 of 53 AI platform roles that state a minimum accept 2 years or fewer. Across all the AI roles we track that state one, 11% do. Platform ads open to juniors far less often than AI ads as a whole, and 79% of the platform ads that state a minimum set the bar at 5 years or more. We read the reason in the job itself. In our view, a platform team runs the shared systems that model teams build on, so a mistake there reaches many people at once. A team hiring for that seat, as we read it, wants someone who has already carried a pager for a system people relied on, migrated it without losing data and paid for a bad capacity guess. Those lessons come from time in production, and an ad can only ask for them as years. We read this as good news for engineers who assume the way into AI work runs through a research credential. The years these ads count are years of experience, and a backend or infrastructure engineer has been collecting them all along. Our view is that outside research roles, a record of systems that stayed up tells an AI hiring team more than a degree does. What that engineer needs is translation. Say you have spent most of a decade on payments infrastructure: the queue you scaled, the deploy pipeline you hardened and the cost review you led all have counterparts in platform work. In our reading, an application that leaves those links for the reader to draw is weaker than one that draws them, so put the platform vocabulary next to the systems you ran. Hiring managers face the same gap from the other side. In our reading, an AI platform ad that leads with model names draws candidates who know the models and quietly screens out the infrastructure engineers the job needs. Leading with the systems the role will own, and stating the years plainly, keeps the strongest pool reading to the end. The sample is small: 53 of 76 AI platform roles on our board state a minimum, so the ones that stay silent may be more flexible than this suggests. A backend engineer with years of production behind them should apply to AI platform roles now, because the requirement that filters most applicants out is the one they already meet.

Embedding similarity cannot see the word not. An eval built on it can pass an answer that says the opposite of the reference. Say a support agent is asked whether a customer qualifies for a refund. The reference answer reads "you are eligible" and the agent replies "you are not eligible". The two sentences share their topic and nearly every word, so an embedding model can place them close together and cosine similarity rates them as a near match. In our reading, a threshold set to accept paraphrases would let that reply through, and the eval would report a pass on an answer that gives the customer the wrong decision. Chapter 2 of Evaluating Agentic AI, free to read, works through this case alongside token F1, edit distance, BLEU, ROUGE and BERTScore. The book's position is that these reference metrics are fast, deterministic and need no LLM, which makes them good at catching an output that drifted from a known-good answer after a prompt change. It is equally plain that similarity stands in for quality and can be fooled. Our reading is that a check this cheap is easy to stop questioning, which is how a blind spot can sit inside an eval without anyone noticing. We read this as the kind of knowledge that separates evaluation candidates. Plenty of engineers can wire up a similarity score. Fewer can explain why their eval passed a wrong answer, and that explanation is what an interviewer is probing when the conversation turns to how you know your agent works. A hiring manager gets a cheap screen from the same case: describe the refund reply and ask what the metric would make of it. Three things the chapter's evidence says to check, ask or show: - Check whether your metric can tell a negated answer from its reference before you trust any threshold on it. - Ask which human labels the team's LLM judge was checked against, and whether those examples were held out. - Show one case from your own work where a cheap metric passed a wrong answer and say what caught it instead. Negation is one blind spot among several, and the book still recommends these metrics as a first filter before paying for an LLM judge, so the lesson is to put a second check on top of them and keep them running. The same discipline applies one level up. An LLM judge is a classifier with its own blind spots, and we would trust one only after it agrees with human labels on examples it has never seen.

Customer-facing AI work means a move to San Francisco, people say. On our board, most forward deployed roles are posted elsewhere. Of the 418 forward deployed roles we track, 23% are posted in the SF Bay Area. New York follows with 84 roles, close to the Bay's 98, and London is the next hub after them. We read the belief as a carry-over from model building, where the work sits with the training team. A forward deployed engineer works inside a customer's systems, so in our reading the role follows the customers an employer sells to, and those customers are spread across many cities. Place still matters for this job. Only 22 of the forward deployed roles we track are remote with no city, so nearly every one expects you to live near an office. The useful question for a search is which of the hubs is in reach. Say you have shipped backend work for years in New York and want a customer-facing AI role. The roles are already posted in your city, in numbers close to the Bay's. A move west would add a relocation and weeks to the search for a pool that is only a little larger. Filter by metro first, and plan a move only if the companies you want post nowhere near you. Teams hiring for the role face the same map from the other side. An ad that names a single Bay Area office competes in the hub where every AI employer recruits. Naming a second metro where the team already has people widens the pool, and stating the days on site answers the question a candidate will ask on the first call. A role posted in several cities counts in each, so the city totals overlap where a company lists both New York and the Bay. We think a forward deployed search should start from where these roles sit. With New York close behind the Bay and few roles open from anywhere, the city an engineer lives in now is often already a hiring hub for this job.

In a case round, the answer that scores starts with the goal, the budget and the success measure, said before any design. Our Forward Deployed Engineering Interview book grades answers on a four-part shape: a claim, the number behind it, the trade-off and the next step. Each part leans on the framing. The number has to measure something the customer named, and a trade-off only means something against a budget the candidate has stated. A candidate who opens on architecture has nothing to hang either on. The book also tracks which rounds now let candidates use AI, from loops that ban it to loops that require it. Our reading is that this raises the value of the opening. An assistant can sketch components in seconds, but it cannot tell the interviewer which customer is waiting, what the team can spend or what done looks like. Where tools are allowed in the room, the opening minutes are where a candidate's own judgement is easiest to see. So the step worth rehearsing is the one before the first box. Three things to say out loud: - Restate the goal in the customer's words and name who is waiting on the result. - State the budget you are designing to: latency, cost per request, people and weeks. - Name the number that would show the customer it worked, before you choose a model. A hiring team gains from scoring that opening as its own step. Two interviewers who grade the frame separately from the design have something concrete to agree on, and the book draws on Sackett's 2022 validity estimates for structured interviews, where that kind of shared rubric is the point. In our view, a loop that grades only the final design, in a round where an assistant can draw one, learns little about the engineer it would send to a customer. The book builds its picture of the loop from published pages and candidate reports, so each company runs its own version of this round. We would rehearse the opening of a case round more than any diagram. In a room where tools can draw the boxes, the goal, the budget and the measure are what show how a candidate would start work at a real customer.

Each model launch looks like a win for the people who train models. On our board, the hiring lands on the people who serve them. The belief has a strong form: a launch is the work of a research team, so if launches drive hiring, the people to hire are the ones who can train the next model. Our reading is that a launch is where the work starts for every team that wants to use the model. Someone has to fit it on the hardware, keep it fast under load and know what each request costs. That is the work ML systems roles describe. Of the 99 ML systems roles we read this week, 65% involve inference, and Kubernetes appears in 54% of them. Ray, a tool for spreading work across a cluster, turns up in 17% of these roles against 4% of all AI roles. We read this as a consequence of who ships what: most teams run models someone else trained, so their engineering effort goes into serving, scaling and operating them. Apple's ads show the same pull at a company most engineers would recognise. A Sr. AI Inference Platform Engineer role posted on 29 October 2026 covers benchmarking, capacity projections and inference platform analysis, with a posted base of $175,000 to $308,500. A Machine Learning Research Engineer ad asks for on-device optimisation such as quantisation, beside multimodal and vision-language models. This week's launches lean the same way, towards small multimodal models built to run on a phone or a laptop. If you want one of these roles, build the evidence a serving team would check. Take an open model, quantise it, serve it behind an API, load test it, and write down the latency, the throughput and the cost per request. Then explain the trade-off you chose, the accuracy you gave up against the memory you saved, because that is the judgement an ad asking for benchmarking and capacity projections describes. A team hiring for this work can widen its pool by naming what the job does. Distributed systems appears in 94% of ML systems roles, and backend engineers who have run services under load already bring it. In our reading they pick up the model side faster than researchers pick up operations, and an ad that leads with training experience screens them out and leaves the serving problem open for longer. The ML systems group is 99 roles, a small sample, and a role usually names several domains, so the shares overlap. Our view is that every launch adds serving work at the teams that adopt the model, and the engineer who can say what a model costs to run is the one those teams hire next.

Reading every ML systems ad line by line, we found distributed systems screens candidates in and inference sets them apart. A job site shows a title and a list of skills. We read the requirement lines of 99 ML systems roles on our board, each from the company's own careers page, and sorted every skill by where it sits. A skill counts as required when any hard requirement names it, and as preferred only when nothing but the nice-to-have lines do. Distributed systems is required in 63% of those roles, more than any other skill we counted, while machine learning is required in 39%. Inference runs the other way. It is required in 14% and listed only as preferred in 28%, the one skill we counted that sits in the bonus list more often than the required one. We read the split as a description of the job. Our reading is that ML systems work runs on many machines at once, and a team will not hand that part to someone still learning it, so the ads filter for it. Serving a model fast is a narrower skill, and in our reading many teams expect to teach it to someone who already understands the cluster underneath. Say you have spent years running services across many machines and have never served a model. The bar these ads filter on is one you already clear. Read each ad as two lists. The required lines decide whether your CV gets passed on, so answer each with a system you ran and the scale it ran at. The preferred lines decide who gets remembered, and inference is the line these ads most often leave open. One serving project answers it: an open model behind an endpoint, load-tested, with the latency and throughput you measured before and after a change such as batching. An application that answers only the first list competes with every other infrastructure engineer who clears it. The same split hands hiring teams a wide pool, and a decision. If the role needs someone serving models in the first week, inference belongs in the required lines. Left as a preference, it invites candidates who will need months to learn it, and a loop that never tests serving will not find out which of them already can. These are 99 roles from the companies Agent Native follows, a small sample, and a preferred line can weigh more in the interview than its place in the ad suggests. An infrastructure engineer eyeing ML systems work is closer than the title implies, because the filter is a skill they already have, and one measured serving project covers the preference that sets candidates apart.

If you want an evaluation role, bring a failure you found in real traces and the measure you built from it. Evaluation has its own job titles now, and the ads say plainly what the work is. Glean's AI Engineer, Data and Evals posting on Greenhouse gives the role evalsets, agentic LLM judges, benchmarks and the infrastructure to run them. HP's Agentic AI Systems Evaluation Engineer posting asks the engineer to connect user stories and product goals to measurable outcomes. Apple's Machine Learning Engineer, Evaluation posting in Responsible AI and Safety describes turning policy and risk assessments into measurable criteria. The products differ, and the task is the same. Someone states a goal in words, and the engineer decides what number would show it is met. A goal does not say which failures to count. The traces do. Evaluating Agentic AI makes the point in its Chapter 1: deterministic checks are free, fast and never hallucinate, but they catch only what you anticipated and encoded. In our reading, you anticipate well only after reading real conversations and sorting what went wrong in them. An engineer who writes metrics first measures what was easy to imagine, and the suite stays green while users hit the failure nobody wrote down. So the skill to show is error analysis. A portfolio eval that starts from failing traces, groups them into failure types and only then writes a check for each, a rule where a pattern is enough and a judge where it is not, answers the question these ads ask. In the interview, the strongest reply to "how would you measure this goal" opens with what you would read first. Glean's posting lists a posted base range of $200,000 - $280,000, without equity or bonus. Our reading is that at that price a team wants more than a dashboard of scores; it wants the failures behind them. Hiring teams can test for this in a single round. Hand a candidate real traces and one product goal, and ask which criteria they would write first and why. A candidate who reads before proposing a metric is doing the job as these ads describe it. These are individual postings our research found, so they show what the job asks without saying how common it is on our board. Error analysis is the evaluation skill these employers need most and candidates practise least, which makes a set of read traces and the criteria they produced the strongest thing an evaluation applicant can bring.

A research engineer title signals top pay, people say. On our board, its posted base spans $250k to $440k in the middle half alone. The title does lead. Across 67 posted US ranges from 23 companies, research engineer roles post a median base of $370k, against $260k for the AI roles we track. Yet the upper tail of research engineer ranges reaches $630k, and the bottom of the middle half sits below the median for AI roles as a whole. We read that spread as a sign that one title covers several jobs and several levels. In our reading, the title names a kind of work, and the price follows the team and the seniority behind it. Two offers can share every word of a title and still be priced for different scope. The belief survives because people repeat the median, and a median describes the middle of a group. The team making a given offer may sit anywhere in it. In a group this small, a few teams posting several high bands can lift the figure everyone quotes, which is why we treat posted pay as the opening of a conversation. So an engineer weighing a research engineer offer should price the team before the title. Ask which group the role sits in, what level the offer is pitched at, and what that team's own posted range is. Then ask how equity is valued and what the bonus looks like, because posted base leaves both out, and either can change what an offer is worth. A hiring manager gains from the same precision. An ad that names the level and the team's own range tells the right candidates they fit. An ad that leans on the title alone invites every applicant to anchor on the highest figure they have heard, and the first pay conversation starts with a gap to close. These are 67 ranges from 23 companies Agent Native follows, read from their own careers pages, so one employer's bands can shift the whole group. We would negotiate a research engineer offer from the team's range and the level on the table, because the title by itself covers offers that sit a long way apart.

Research engineer roles on our board go by 48 different titles and 63% of them name no level.

Software engineer interviews until 2026: - Solve 2 LeetCode mediums in 45 minutes. - Design a system that handles 10 million users. - Explain how you'd optimize a SQL query. - Implement an LRU cache from scratch. - Analyze time and space complexity. Software engineer interviews in 2026: Canva expects candidates to use AI tools in technical interviews, Google tells candidates they may not use them, and Anthropic sits between the two. Its technical interviews run in Colab and CodeSignal live coding tools, and candidates may look things up. Three well-known employers have written three different rules for the same kind of round, and a candidate who prepares for one rule and meets another walks in with the wrong habits. The split is a difference in what each company wants the round to measure. Canva expects machine learning, backend and frontend candidates to use AI tools, which treats the assistant as part of the job under test. Google's published guidance keeps the assistant out, so the round measures what a candidate can reason through alone. Anthropic's rule sits closer to everyday work: a reference to hand, the judgement still the candidate's. Where the tools are allowed, we expect the bar to move a lot higher. Once an assistant can draft the function, a passing test says less about the candidate, so what we would listen for is how they checked the output: the edge case the tests skipped, the line they rewrote because it handled an error badly, the reason they trusted the rest. Say that reasoning aloud as you work. Hiring teams that allow assistants should grade it openly, or the round risks rewarding speed at accepting suggestions.

38% of ML systems roles are remote, against 26% of all the AI roles we track. ML systems work runs on clusters that sit in a data centre whatever the team's address, so an engineer in another city reaches them the same way as one at the next desk. Forward deployed roles are remote in 23% of cases, a job built around customers keeps people near them. Agent engineer roles are remote least often of all, at 20%. That makes ML systems a real route into AI for an infrastructure engineer who wants to stay remote: running fleets, scheduling jobs, keeping a cluster healthy at night. A team hiring ML systems engineers is competing with peers where remote work is common. An office-only ad in this family asks candidates to give up something many of their alternatives offer, so the job has to make up for it. A team that is remote within limits should name the countries in the ad and spare both sides a call that ends in its first minute. We think ML systems is the clearest remote path into AI work on our board for engineers who already run infrastructure, as long as they check the country line before they weigh the salary.

Code changes at Meta grew 220% in a year, while features reaching users grew 36%. By Meta's own figures show that major technical and security incidents grew 40% over the same year, faster than the features users could see. Meta also puts the growth in engineer time spent firefighting at 70%. Agents speed up the writing of a change far more than the deciding, integrating and proving that turn it into something a user can rely on. The counted growth sat in changes to internal platforms and infrastructure, and we think more change there means more surface to maintain and more places to break, while the road to a shipped feature kept its old pace. Meta's leadership noticed the shortfall from the top. Speaking to staff at a town hall, Mark Zuckerberg described the progress of agent development in recent months as something that "hasn't really accelerated in the way that we expected." A team that tracked merged pull requests and closed tickets would have read that year as a triumph. The bill lands elsewhere, in our view: on the on-call rotation that absorbs the incidents, and on the engineers whose week fills with firefighting instead of the next feature. A dashboard built on output alone hides that bill from the people who approve the next quarter's plans. The check is cheap to try. Put features that reached users and incidents on the same page as merged changes, and review the three together. If change volume climbs while the other two stay flat or get worse, your agents are producing work your team then has to pay for. Chapter 1 of Good Engineering sets Meta's figures beside other telemetry on output and outcomes, and it is free to read; Chapters 1 and 2 are open to everyone. We would stop reporting merged changes as productivity in any team running coding agents, because that is the one number agents raise whether or not users gain anything.

If you build search agents, show where the retrieved passage goes before you show how well you search. We call this the evidence-delivery gap, i.e., why search agents keep searching after they have found the answer.

1,679 open AI roles on our board this week. 83% of them say nothing about visa sponsorship. Read together, the ads from 221 companies describe a narrow door. 46% of the roles are posted in the SF Bay Area. 69% of the ads that state a minimum ask for 5 years or more. Forward deployed engineering, at 26% of AI roles, is the largest family we can name. In our reading, the role these ads add up to is senior, close to San Francisco and often near the customer, and its ad says little about the terms a candidate needs to know first. We read the shape as a matter of what the work demands. A team that puts engineers in front of customers wants people who can ship without someone senior beside them, and that pushes the years asked up. Forward deployed titles and a Bay Area office fit the same picture: the work sits where the product team and its customers meet. The silences say something about how these ads get written. Only 17% of AI roles say anything about sponsorship, and only 5% refuse outright. 55% post a pay range, with a median posted US base of $262k, and 24% do not say whether the job is remote, hybrid or on-site. Our view is that a team racing to fill seats writes the requirements and leaves the terms for the call. Say you are a backend engineer on a visa, living outside California. Most of these ads will not tell you whether you can apply. Raise sponsorship and the pay range in the first call, before any take-home, and ask where the team works day to day. Search past the obvious titles as well: Member of Technical Staff postings hold AI work that a search for "AI engineer" misses. The team writing the ad pays for each missing line. An engineer who needs sponsorship and meets silence can move to the next ad, and a candidate who cannot place the pay spends a call finding out. One sentence on visas and one posted range keep both in the pool. Early-career engineers face the hardest version: 11% of the ads that state a minimum accept 2 years or fewer. Applied AI roles are more open at 26%, though that rests on a small group of ads. And an ad's wording can be narrower than a company's policy, so some of the silence on visas may hide a yes. On our board, the openings cluster in a few cities and in work that meets customers. We would apply where the roles are and ask early about what the ad left out; the teams that write those answers into the ad reach more of the engineers they want.

In the Bay Area, 39% of AI roles on our board are on-site. In Seattle, 14% are. Across the 764 Bay Area roles we track, on-site is the most common arrangement, and it runs above the 32% we count across every AI role on our board. Seattle leans the other way: 43% of its roles are hybrid, more than in any other metro we follow.

If you are weighing a Bay Area offer against a New York, you typically learn little from the city and a great deal from the company, the scope and the title. Staff ranges spread widest in both metros, so that is where the employer matters most and where a quick comparison misleads most. You should choose between these two cities on the role, the level and the equity, because posted base gives neither metro a lead worth relocating for.

More transparency always provides higher quality pool of candidates.