At Workera, we’ve always described ourselves as an AI-native company. That identity might make it surprising that one of our most important product updates this year — Score Appeal — puts humans firmly in the loop. Score Appeal allows any learner who disagrees with an assessment result to contest it, triggering a review by our team of human experts. 

At first, this may appear like a bet against AI. In reality, it’s an accelerant. Score Appeal helps us move faster, improve AI systems, and build trust with customers.

According to IDCA’s 2025 Global Artificial Intelligence Report, 87% of companies identify AI as a top priority in their business strategy. At the same time, 95% of executives report experiencing “at least one type of problematic incident from their use of enterprise AI.” Those two statistics show the gap in our industry between scale and trust. We need humans in the loop to address that gap and allow companies to safely chase AI innovation.

Responsible AI has always been central to Workera’s mission. Our business has the serious responsibility of evaluating human capability. People’s careers, opportunities, and confidence in their skills can be shaped by the accuracy of our assessments. It’s essential to get that right — especially as we aim to build a more meritocratic society and workplace. 

While AI models are getting more powerful on a weekly basis, they’re not perfect. Humans must remain involved, both to ensure fairness today and to fuel better systems tomorrow.

From human reviews to self-healing AI

No single score appeal happens in isolation, and the web of appeals that take place in Workera help to improve the overall performance of our platform. 

Each review provides invaluable feedback to our scoring agents and the underlying assessment framework. When a human expert adjudicates an appeal, that decision becomes a training signal fed back into our datasets. Over time, these cases strengthen our models, making them more robust, accurate, and contextually aware. In other words, every valid score appeal doesn’t just resolve a learner’s immediate concern — it makes the entire system smarter.

Think about what happens behind the scenes. Sometimes the issue lies with the wording of a question, which can be fixed at the source. Other times it’s a matter of how a rubric is defined or how an AI scoring agent interprets a response. Each of these discoveries provides an opportunity to refine prompts, rubrics, and scoring models in ways that strengthen the system for every future learner. This is a “self-healing” cycle — every appeal injects more quality into the system, decreasing the need for more Score Appeals over time.

A November 2024 paper in Minds and Machines argued that legal and ethical guidelines that require “effective human oversight” should be interpreted in terms of how reliably oversight reduces risks and under what conditions it works. The Score Appeal tool adheres to this argument by providing a mechanism that lets humans inspect both the rubric or question framing and how the model interpreted responses.

Customer-driven AI experiences

The value of Score Appeal isn’t purely technical — it’s about trust, a foundation of all human relationships. Recent Pew and Gallup surveys found that while AI experts are generally optimistic, the public is more skeptical — many want more control over how AI is used, because they have little trust that institutions will manage AI responsibly.

To be able to trust an AI tool, users need transparency and explainability. When people feel like there’s no recourse and no human oversight, trust suffers. Customers increasingly expect more than just “AI is powerful” — they expect accountability. A University of Melbourne & KPMG survey found that though two-thirds of respondents use AI regularly and 83% believe AI will bring benefits, 58% perceive AI as untrustworthy.

Users dedicate their time and energy to completing assessments and engaging in learning pathways. Those users need to have confidence that someone on the other end is listening and resolving issues as they arise. Even when an appeal doesn’t change the outcome, the process offers transparency and explainability. Sometimes we may reject an appeal but explain exactly why. Other times we recognize nuance that the AI missed and award partial credit, updating the rubric for the future. Taken together, Score Appeal is a natural extension of our customer-led approach, allowing user feedback to guide and shape the development of our platform..

While user expectations may evolve over time, with less insistence on human involvement as trust in AI grows, today Score Appeal gives the reassurance needed to engage fully with our platform. It transforms AI-powered assessments from a black box into a collaborative process.

Keeping humans in the loop

Across a wide range of industries, companies are wrestling with the same question: when should humans stay in the loop, and when is it safe to hand full control over to AI? The answer depends on the stakes. If you get the wrong shopping recommendation, the consequences are trivial. But when it comes to high-stakes assessments — those tied to hiring decisions, promotions, or performance reviews — the consequences can reshape someone’s career and livelihood. With those stakes, humans must remain part of the process for the foreseeable future.

Score Appeal is more than a safeguard; it’s core to our mission of delivering trusted, verified skill intelligence. By ensuring that assessments are transparent, fair, and continuously improving, organizations are enabled to make better, more equitable decisions — and help individuals advance professionally based on what they truly know and can do.

By keeping humans in the loop, we can innovate responsibly, build better AI, and create a system where merit, not bias, drives opportunity.

This originally was posted on Measure Forward, Workera's LinkedIn newsletter. Click here to follow along and subscribe.