After qbash added Jev, I ran a poll on X asking which Jev Task I should build first. An AI slop copy loop won with 61% of 75 votes. In this Task, Claude writes marketing copy, Jev and a set of regex checks grade each draft, and anything that fails goes back to Claude with feedback for another attempt.
What's covered
- The Task package:
main.py, four prompts, and awriting-rules.mdstyle guide - The 12 questions Jev answers about each draft, and the thresholds that decide a pass
- A sentence probe that finds which lines caused a failed check
- A run at the default thresholds that passed in round 2, and a stricter run that passed in round 24 of 25
The Task
AI Slop Copy Loop takes six inputs: brief, audience, tone, format, word_target, and max_rounds. If the brief is left blank, the Task uses a built-in demo brief for Ledgerly, a bookkeeping app for freelance designers that costs $12 a month and comes with a 30-day free trial. The brief's key point is that users spend about 10 minutes a month on bookkeeping.
Beside main.py, the package holds five files:
copy-writer-2.promptis the Claude prompt that writes each draft. After round 1 it also gets the previous draft and the reviewer feedback, which accumulates across rounds.writing-rules.mdis the house style guide, with banned phrases, banned words, banned punctuation, and notes on structure and rhythm. Prose above its# Automatic checksheading goes to the writer. Each rule below that heading becomes a regex lint check.slop-gate.promptis the Jev prompt that grades each draft.sentence-probe.promptis a second Jev prompt that judges one sentence at a time.loop-analyst.promptruns on Claude Sonnet 4.6 once the loop ends. It reads the run log and writes up what happened. It is there so we can study the loop, and it wouldn't necessarily run in production.
Twelve questions for each draft
slop-gate.prompt gets the brief, audience, format, and copy, and asks Jev 12 questions using all of its question types. Three are scores on five-level scales: does the copy sound human, is it on brief, and how specific is it. Eight are yes or no: AI clichés, drumrolls that announce a point before making it, contrast reframes, mechanical rhythm, filler, generic copy, overclaims, and a clear call to action. The last is a pick-one verdict asking whether a demanding marketing lead would ship the copy, revise it, or rewrite it from scratch.
Jev grades each draft twice (JEV_SAMPLES = 2) and the Task averages the two passes. The thresholds sit near the top of main.py:
# Gate thresholds
MIN_P_GOOD = 0.70 # sounds_human, on_brief, specificity
MAX_P_CLICHE = 0.30 # ai_cliches
MAX_P_PATTERN = 0.35 # drumroll, contrast_reframe, rhythm, filler
MAX_P_GENERIC = 0.40 # generic
MAX_P_OVERCLAIM = 0.40 # overclaims
MIN_P_CTA = 0.50 # clear_cta
MAX_P_REWRITE = 0.30 # verdict == rewrite (ship itself is advisory only)
WORD_TOLERANCE = 0.30 # within ±30% of word_target
WATCH_MAX = 1 # max uses of each watch-list word
JEV_SAMPLES = 2 # Jev grades per draft, averaged
# Sentence-level localisation: when one of these checks fails, Jev scores each
# sentence on its own so the feedback can quote the lines that cause the failure.
PROBE_KEYS = ("overclaims", "cliches", "drumroll", "reframe", "filler")
PROBE_MIN_P = 0.30 # a sentence is quoted only above this probability
A draft passes when every Jev answer is inside its threshold and the lint checks find nothing. When a draft fails, the failed checks become feedback, and Claude revises the best draft so far.
Finding the sentence that failed
Jev returns a probability for each question and no explanation. If overclaims fails, Claude can't tell which sentence made the claim. So when a check in PROBE_KEYS fails, the Task sends the copy through sentence-probe.prompt one sentence at a time, with the previous sentence included as context. Any sentence that scores above 0.30 is quoted back to the writer along with the feedback.
The probe adds a Jev call for every sentence it checks. I haven't yet compared the loop's total token usage with having an LLM grade the copy.
Run 1 passed in round 2
The first run used the default thresholds, a product launch email as the format, a 300-word target, and up to 25 rounds. Round 1 failed three checks: Jev flagged a contrast reframe, and lint caught an em dash and a draft that came in short. Round 2 fixed all three and passed, and the quality score rose from 75% to 87%. The final draft's subject line was "Your books shouldn't take longer than your lunch break."
Earlier versions of the loop, before I added the sentence probe, could spend all 25 rounds stuck on the same issue.
The analyst's report also found a problem the gate missed. Across the three stored runs, the writer's first draft came in well under the word target (182 words against 300 in this one). When it padded the copy to reach the length, it invented product behaviour the brief never mentioned. Round 2 added "The app works best when you check in regularly, but it doesn't demand it," and that sentence passed every check.
Run 2 passed in round 24 of 25
For the second run I raised the thresholds for sounding human, staying on brief, and specificity from 70% to 90%. After a couple of runs stalled at 88 or 89, I lowered the quality threshold from 90% to 85%, and the next run passed with a quality score of 88%. Its subject line was "Quarter-end bookkeeping in about 10 minutes a month."
The analyst summed up why it took 24 rounds: the writer kept swapping one banned pattern for a near-identical variant, and the filler check fired on the call to action while missing the filler elsewhere. Jev and lint flagged 10 distinct issues, seven from Jev and three from lint. Seven of the 10 came back at least once after a fix, including sounds like AI, generic, filler, contrast reframe, and em dashes. All of them were fixed by the final round.
Seventeen drafts scored below the best one so far, so the Task set each of them aside and revised the best draft instead. After four rounds in a row without a new best, the Task started over from a fresh draft, which happened in rounds 11, 18, and 22. Over the run, the sentence probe quoted 62 sentences back to the writer. Jev's two passes on each draft stayed close, with an average maximum gap of 3% and a widest gap of 6%, on the generic check in round 5.
The quality score and Jev's sounds-human probability both trended upward across the rounds. Each draft comes from a new Claude call, but the code builds its context from the feedback so far, and that context seems to push the drafts in the right direction. My concern going in was that the loop would be a dice roll, with Jev's answers steering a prompt that produces something random each time. Some of that may be true. Over 24 rounds, though, the upward trend and the way several issues cluster in the early rounds of the issue matrix make me think the feedback has an effect. It is one run, so I wouldn't read more into it than that.
Where I'd use this
I wouldn't use this loop to approve marketing copy on its own. If you build one, limit the rounds and read the output yourself. "Does this sound human?" is an abstract question to put to any classification model grading creative work. You can give the model examples, but I don't know what "human" means in its training data.
For a loop whose criteria are more deterministic than "sounds human," I'd reuse the structure: discrete scores, a probe that finds which sentences fail, and feedback assembled by code.
Why I'm paying attention to Jev covers why I think decision models belong inside a larger process. To set Jev up, see Use Jev with qbash Tasks and Jev Decision Prompts.
Video transcript
Hey guys, Mitch from qbash here. Late last week, we officially started supporting Jev on the platform. We've added it as a model here and I put out a tweet asking which rough ideas about how you could use Jev in a task on qbash would people like to see me try. The overwhelming majority selected an AI slop copy loop. So that's what I've tried to put together here.
It's relatively simple and is really meant to just be an exploration of where Jev fits into this idea of system building. So the idea is we have this task. This task takes inputs brief audience tone format word target gives them to Claude. It also has a writing rules markdown file. The static list of banned phrases, banned words, banned punctuation and structure and rhythm and all of that kind of stuff.
takes all of that in and produces an output, generates copy, and that gets sent to Jev. So, Jev is taking the original brief, the audience format, and the output copy, and it's running it through a series of 12 questions. Does this sound human? 1 to 5. Is it on brief?
1 to five. How concrete or specific is the copy? 1 to five. How many cliches does it have? We have a small example of what cliches are here.
Drum roll. Check as a pattern, which is something that we don't like. Contrast reframe or negative parallelism. Rhythm filler. How generic is it?
Does it overclaim? Does it have a clear CTA? And finally, what is your verdict? Would you ship it? Would you re revise it?
Or would you completely rewrite it? It'll run that twice for every output. And then all of that will be used to create a score which is ship or send it back to the model. If it's sent back to the model, it's going to use these other inputs which is the previous draft and the reviewer feedback which is cumulative across all rounds. So what you have is Claude writing an output, sending it to Jev.
Jev has criteria and is saying do we meet the criteria or not? And if it makes it, great. That's the end of the loop. If it doesn't make it, all of that information gets packaged up and rebriefed into Claude, who's going to have another crack at it. Some of these things that it's checking for, like over claims.
It's not going to be inherently clear to Claude which of the things it generated is triggering that. It doesn't really have the context. And Jev is a classification model. It can't give you the context of this is the reason that it failed this check. So one of the other Jev prompts we've built is a probe.
So if something fails over claims, it is going to take sentence by sentence what has been written to try and isolate what is triggering that failure. Once it figures that out, it'll send it back. Same is true for cliches, drum rolls, reframes, filler. trying to give the model more context for a rewrite. Is this more efficient than having an LLM grade an LLM?
I'm not sure. We can look at token usage afterwards. The final thing I have in here is the loop analyst. That's for us. This wouldn't be in production necessarily.
Taking all of the data that's been produced, looking at the logs, and giving us insights as to what has happened with this loop. So, we're going to run it. There's some defaults baked in here. The brief is for a bookkeeping app called Ledgerly. It's written for freelance designers.
The tone is warm, direct, a little dry. I think we will do a product launch email. We'll say 300 words max. Max rounds. We'll leave it at 25.
And we'll hit run task. Okay. So, it's going to set up an isolated runtime. Okay. And now it's going to start running.
So, it's going to compile the brief and it's going to give that to the copywriter. That output is going to go to Jev and Jev is going to return an option. And then you can see it's running a sentence probe here. So, obviously something has failed. Round one failed.
There are three generic sentences. I'm going to let that run and I will come back once we have the final output and we can go through the results together. Okay, I'm back much quicker than I thought. It passed in two rounds. Let's have a look at the chat and see what happened.
So, if we go to the top here, pass the gate in round two of 25. Product launch email tag 300 words. Here's the final copy subject. Your book shouldn't take longer than your lunch break. Okay.
You're a designer. You're good at making things clear. Consider precise. Okay. More interested in the infrastructure side of things.
So, how the loop ran, the quality was 75 in round one and then bumped up to 87 in round two, 87 in the last round. Failed checks, three in round one and then none in round two. One of them came from Jev, two of them came from the lint. So, what it looks like happened here is what we wanted to happen in round one. It found a contrast reframe.
It found an em dash and the length was wrong and all of those were corrected in round two. This is interesting to me because in prior rounds before I'd put in the sentence probe, it would get stuck on the same issues. We would go through 25 rounds on the same issue. This maybe has fixed it in one. I want to run this again.
What I might do is change some of these thresholds. So let's change the human on brief and specificity thresholds from 70% to 90%. See what happens. Okay. Exit.
Run task. We said product launch email. We said 300 words. Yeah, 300 words is fine. Okay.
I'm going to run this and I will let you know what happens. Okay, we're back. Buzzer beater passed the gate in round 24 of 25. Copy quarter end bookkeeping. There's a specific kind of dread that hits around the end of March, June, September, and December.
Okay. How the loop ran. 24 rounds to ship. The writer keeps swapping one banned pattern for a near identical variant and the filler gate fires on the CTA while missing the real filler. Interesting.
Best quality. So, caveat, I did lower it from 90 to 85. I ran this once or twice before this run and like it gets stuck on 88 89. Best round was the last round. 88% six failed checks down to zero.
10 distinct issues, seven from Jev, three from lint, came back after a fix. AI, generic filler, contrast, reframe, punctuate came back at some point. Never fixed none. Everything got fixed in the end. But if you look at the quality score and the sounds like human according to Jev, these are directionally up and to the right.
The loop is reinforcing something. And while the prompt call for generation is a new call every time, the code part of the task is correctly assembling context to shift and tilt things in the right direction. failed checks started at two down to zero. That's lint failures. The jev checks down and to the right and then you have the full matrix here.
Almost seemed like a dice roll maybe started to go away more consistently towards the latter rounds. Same is true for generic sounding stuff. Seemed to be clustered towards the start. Yeah, interesting. My biggest concern about something like this is it's just a dice roll and we're altering the prompt based on the outcome that Jev is giving us, but essentially it's just a dice roll every time.
I think some of that might be true, but these charts over 25 runs and the clustering in this matrix makes me think that it's not entirely a dice roll. limited sample size. Obviously, I think the other thing that comes to mind for me is like for something like marketing copy, obviously Jev is the wrong tool. I won't say is the wrong tool. I'm sure there are smarter people than me that could figure out how to make this much more effective.
But I think like the thing I kind of can't get my head around is the training data aspect and like we're saying does this sound human? That's a really abstract question. We can provide examples but in the training data what does human mean to this model? I don't know there's some fundamental questions I have not necessarily of Jev but in the usage of a classification model to grade something inherently creative like marketing copy. So in short I would not build a loop to do this or if you do severely limit it and you always have to read the output somewhat.
However, structurally a loop design where the criteria is much more deterministic. I think this is a really interesting way to approach it like having the discrete level scores and like figuring out what specifically is failing and then sending that back in. Interesting. Anyway, another experiment. Hope you found it interesting.
I have some more ideas about how we might use Jev or how you could run experiments with Jev and do interesting things. As I said, there's a blog article about it. Take a look at that. And yeah, the other guys on the team, they have some thoughts and some videos that they're going to put out this week as well.