This is the first post here, so a word on what the blog is for: notes from building Job Puffin, what we find out rating LinkedIn postings against CVs, and what changes in the product. The guides are for the questions a job seeker searches; this is for the rest. It will not be often.
The first thing worth writing down is the one that shaped the product most: a rating of a job against a person cannot be an average, and it took three tries to stop making it one.
The 39
The first version did the obvious thing. The model read the posting against the CV and what the search was after, scored seven criteria out of a hundred, and the rating was the weighted mean. It looked right on most postings and wrong on exactly the ones that mattered.
A search for contract work was shown a permanent role. The terms criterion came back low, as it should, and the other six came back middling, as they would for a job that was otherwise a reasonable fit. The weighted mean was 39. A 39 reads as "maybe, if nothing better turns up". The right answer was "no, this is a permanent job and you are looking for a contract", and no amount of weighting gets a mean to say that. One number in a mean can lower the result; it cannot veto it.
Telling the model to sink it
The second try kept the mean and asked the model to do the vetoing: when the posting broke one of the search's musts, push every criterion the must touched down into the bottom band. This failed in two ways. It still did not sink the job far enough, because a must rarely touches more than a criterion or two, and the rest of the mean held it up. And it bent the notes. The skills note started talking about the contract. A criterion that is supposed to say how well the person's stack fits the posting was being used to carry a verdict about the terms, and once that happens the notes stop being readable as reasons.
A gate, kept apart from the score
What works is to stop asking one number to do two jobs. A must is a hard limit in the search's own words: contract only, London or remote, a day rate, sponsorship. The model goes through each must and says whether the posting breaks it, meets it or says nothing about it, with the posting's words as the evidence. A broken must is a dealbreaker, shown with the quote, and it is kept out of the score. The score goes on measuring how well the job fits, criterion by criterion, as if the must were not there.
That sounds like it gives up information, and it is the opposite. The permanent role now shows as a dealbreaker with "this is a permanent, full-time position" beside it, and a score that says how good a fit it would have been otherwise. The inbox sorts the dealbreakers to the bottom. The user sees both facts and neither one is hidden inside the other.
The third status matters as much as the other two. A posting that says nothing about pay has not broken a must about pay; it has left a question. Treating silence as a no would throw out most good postings, since most postings are silent on something. So an unstated must stays a question, and the criterion it belongs to is scored on what the posting does say.
Ask for everything, keep what you need
A smaller lesson from the same change. The first version of the gate asked the model to list only the musts the posting broke. It listed the met ones too, each with an explanation that ended "so this must is met, not broken". Models complete lists. Asking instead for the status of every must, broken or met or unstated, and keeping the broken ones in code, fixed it at once, and the met and unstated ones turned out to be worth keeping anyway.
Reason first, number second
Each criterion's note is written before its score, in that order in the answer, so that the score follows from a reason you can read. Done the other way round the note becomes a justification of a number already chosen, and it reads like one. This is also why the notes are shown beside every score in the app rather than behind a click: a score with no reason is an opinion, and the point was never to have an opinion about your job search.
The overall is arithmetic
The model is not asked for an overall rating. Job Puffin works it out as the weighted mean of the seven scores, the same way for every posting, so that two jobs' numbers mean the same thing and can be compared. Once the musts were out of it, the mean was the right tool for what remained: fit is a matter of degree, and a degree is what a mean measures.
What was left out
There is no culture-fit criterion, on purpose. A posting almost never shows any, and a guessed score with a weight attached is noise that moves the mean. The seven that are there are the ones a posting can be read for: the level, the direction, the skills it screens on, the evidence in the CV, the domain, the terms, and how competitive an application would be.
How this looks on a real posting is on the About page, and the method for doing it by hand, musts first and then the fit, is the guide to judging a posting.
