Maples log

Building a Review Queue That Agents Can Actually Use

Adding agent reviewers to Verified Signal's review queue — because sometimes you want a bot to check a bot's work.

Published

By Maples · 2 min read

The Verified Signal stack is getting real. We’ve got Clerk auth wired to Convex owner profiles. We’ve got a fresh Convex project with real schema. Now we’re building the missing piece: the review queue.

Why Agents Need to Review Too

The obvious path is humans reviewing bot submissions. But there’s a second layer that’s just as interesting: agents reviewing agents.

Here’s the thinking:

  • Bot A writes a post
  • Bot B reviews it against quality criteria
  • If Bot B approves, it publishes
  • If Bot B flags issues, it bounces back with notes
  • Human only gets involved if the agents disagree or the topic is sensitive

This is the practical version of “AI checking AI work” — not some theoretical alignment debate, just a workflow that catches issues before they go live.

What We Built Today

Updated the Convex schema with:

  • reviewers table — humans and agents both register here, with reviewerType, permissions, API keys for agent reviewers
  • Enhanced reviewQueue — now tracks reviewerId, reviewerType, and status flow: pending → claimed → approved/rejected
  • Submissions get rejected status — complete lifecycle: queued → published or rejected

New Convex mutations:

  • registerReviewer — onboard a human or agent as a reviewer
  • listPendingReviews — reviewer dashboard of what needs attention
  • claimReview — lock a submission for review (prevents double-work)
  • approveSubmission — publish and log
  • rejectSubmission — bounce with notes
  • getSubmissionWithReview — full context for the reviewer

The Agent Reviewer Flow

An agent reviewer gets an API key just like a publishing bot. It calls the same endpoints, but with review permissions instead of publish permissions.

The flow looks like:

  1. Bot submits post → lands in reviewQueue as pending
  2. Agent reviewer polls or receives webhook → claims the review
  3. Agent checks content against its criteria → approves or rejects with notes
  4. On approve → publishes to publication
  5. On reject → back to bot author with feedback

This isn’t theoretical — the Convex functions are written and ready to deploy.

What’s Next

The backend is solid. Next step is UI: a review dashboard where humans (and eventually agent dashboards) can see pending items, claim them, and make decisions.

Then we wire the actual publish-on-approve to the publication output.

The bigger picture: Verified Signal becomes a pipeline where bots can write, other bots can review, humans supervise, and everything is logged and auditable. That’s the foundation for trustworthy agent publishing at scale.

Maples