Are likes or dislikes on AI products useful? True feedback design should be able to answer "What's wrong and why?"

"This answer is not good" might be due to incorrect facts, being too long, having an inappropriate style, being from an unreliable source, or not following instructions. There was only one click. The team knew the user was dissatisfied, but they didn't know whether to fix the model, Prompt, search, interface or product rules. The feedback needs to be specific enough and at the same time, it should not turn users into free annotators.

01 First, clarify what problem the feedback aims to solve

Feedback may be used to personalize the current user, monitor quality, train models, optimize prompts, diagnose searches, or modify UX. Different purposes require different questions.

If the team adds controls merely because "all AI products should have likes/dislikes", the data is often not used in the end.

02 Simple likes and dislikes are suitable for low-cost signals, but the information volume is limited

Likes and dislikes have fast clicks and high coverage, making them suitable for establishing an overall trend. But looking at it alone cannot explain the reason for the failure.

Microsoft HAX Guideline 15 advocates granular feedback, that is, allowing users to express more specific preferences. You can provide optional reasons after clicking and clicking, instead of popping up a long questionnaire at the very beginning.

03 The reason option should correspond to the actions that the team can take

For example, "factual error, failure to follow requirements, overly lengthy, problematic source, incorrect format." Each category should be able to route to different troubleshooting.

Don't provide a bunch of vague words other than "others", such as "unprofessional", "unintelligent", "dissatisfied", as these are hard to be transformed into improvements.

04 It's not necessary to actively ask for feedback every time you answer

If there is a prominent rating below each message, users will ignore it. It's even more likely to be disturbed if an investigation pops up after the task is completed.

You can make more proactive requests for new features, low-confidence outputs, or sampled sessions, while maintaining low-interference entry points for the rest. HAX also suggests providing granular feedback in regular interactions instead of continuous interruptions.

05 What value will be generated by telling users feedback

The Feedback + Control guide of Google PAIR emphasizes that users should understand the value of feedback and when it has an impact. If the user corrects their preference, it can be explained that "this will affect subsequent recommendations." If it is only used for product improvement, it should also be clear.

Don't imply that a single click will immediately "train the model", unless the system does.

06 Explicit feedback and implicit behavior should not be confused

Users may copy answers either because they are good or because they want to criticize them. Re-generation might be due to the desire to explore, rather than necessarily a failure. Behavioral signals need to be interpreted with caution.

An explicit "fact error" is more reliable than simply clicking Regenerate, but has a lower coverage. The evaluation system usually requires a combination of multiple signals.

07 The feedback data itself may also contain sensitive content

Users may paste customer information, internal documents or personal data in the "Supplementary Instructions". The collection and storage of feedback should enter the same data governance process.

Enterprise products also need to specify whether the feedback will be manually viewed, whether it will be used for model training, and whether the relevant data usage can be turned off.

08 The true closed loop is to connect feedback to evaluation, repair and revalidation

Weekly statistics of the click rate do not equal improvement. The team needs to sample and review, attribute to the model/retrieval /Prompt/UX, form a test set, and then verify whether the fix reduces similar errors.

Good Feedback UX is ultimately not about collecting more clicks, but about building a system where user issues can truly be absorbed by the product.

09 The feedback entry should be connected to error recovery instead of merely sending data to the background

After the user clicks "Factual Error", if the product can immediately provide "View source/Re-search/submit Correct answer", the feedback will also help the user complete the current task.

This immediate value will increase the probability that users are willing to give feedback, which is more useful than simply saying "Thank you for your feedback".

10 Different tasks require different evaluation dimensions

Image generation may focus on composition, text accuracy, and style. The code focuses on runability, security, and compliance with requirements. Enterprise Q&A focuses on facts, sources and authorities.

It is difficult to apply a uniform set of "Helpful/Not helpful" to all products. The evaluation model should be inferred from the definition of task quality.

11 The feedback data needs to be connected to the offline evaluation set

High-quality user error correction can be filtered into the test set, and each subsequent model or Prompt update will automatically revert.

In this way, user feedback will not only be operational reports but also become part of the product quality engineering.

Frequently Asked Questions

Does an AI product have to have some likes and dislikes?

Not necessarily, but low-cost feedback entry points are usually valuable; The prerequisite is that the team knows how to use the data.

Should users be forced to write the reason after clicking?

It is not recommended to enforce. It can provide quick reasons and allow for optional supplementation, reducing feedback costs.

Can regeneration be regarded as negative feedback?

It can only serve as a weak signal. Users may just be exploring different answers and cannot be directly equated with "dissatisfaction".

Will feedback automatically train the model?

It depends on the product realization. The interface should not give users training expectations that do not match reality.

How to determine if feedback design is effective?

See if the feedback can be attributed, formed into executable improvements, and verified in subsequent evaluations whether the problems have decreased.