How Our Fact-Checking System Works

Our website employs advanced language models to assess the factuality of statements. Here's an overview of the process:

Input

Users submit questions for evaluation.

Language Processing

We use the current best open-source AI model to analyze the statement's structure and content.

Optimizing Language Models

Language models excel at small tasks but struggle with larger ones. They perform best in interactive scenarios. We break down the fact-checking process into smaller, manageable task.

This list is a simplifed version of the series of prompts we sent to the model:

  1. Frame the sides of the debate about the fact in question.
  2. Create search terms for both sides of the debate.
  3. Extract the best arguments from search results.
  4. Evaluate and rate arguments for relevance and factuality.
  5. Select the best arguments from each side, ensuring balance.
  6. Determine factuality based on selected arguments.
  7. Provide a detailed explanation of the conclusion.

Why This Improves Fact-Checking

  • Removes inherent bias from internet-trained models
  • Ensures balanced consideration of arguments
  • Leverages language models' strength in small, focused tasks
  • Provides more objective results from non-objective models

Models are trained from the internet, and as a result, they tend to think what the internet thinks. The internet is not objective or balanced, and there is generally more content written about the popular consensus, which results in the models being trained to have a bias that aligns with the consensus.

This is why first refining the arguments from both sides and then prompting the model to make its decision based strictly on the arguments provided helps remove the bias that is trained into the model. Thus, allowing us to get an objective response from a non-objective model.

Also, by controlling the amount and arguments that the language model uses to make its determination, we take advantage of the fact that language models perform better at small tasks. We give the model a small amount of arguments to use to make its decision, and we make sure that we use only the best arguments from each side and an equal amount of arguments from each side to avoid any bias that may occur from giving it too much data for one side of the argument. We believe this is how language models get trained with inherent biases in the first place, as the internet is not balanced or objective.

A More Comprehensive View Of The Process

  1. Question Analysis: The process begins with a user-submitted question. The system analyzes the question to understand its context and determine the two sides of the debate it presents.

  2. Research Phase:
    • The system generates relevant search terms for both sides of the debate.
    • It then performs web searches using these terms to gather information from various sources.
    • The collected information is processed to extract key talking points related to the question.

  3. Evaluation of Talking Points:
    • Each talking point is categorized as supporting either the "pro" or "con" side of the debate.
    • The system then evaluates each point for its relevance to the question and its factual accuracy, assigning ratings on a scale of 1-10 for both criteria.
  4. Content Creation:
    • Using the analyzed information, the system generates a comprehensive article addressing the question.
    • The article includes an introduction, a structured outline, and detailed sections covering various aspects of the topic.
    • The content is written to be objective, considering both sides of the debate.
  5. Visual Enhancement:
    • The system generates prompts for creating relevant images, including a cover image and additional images for key sections of the article.
    • These prompts are used to generate images using AI-powered image generation tools.
  6. Final Article Assembly:
    • The system combines the written content with the generated images.
    • It creates a title, chooses an appropriate category for the article, and optimizes the content for search engines.
  7. Quality Control:
    • Throughout the process, the system performs multiple attempts and checks to ensure the quality and accuracy of the generated content.
    • If any part of the process fails, the system logs the error and attempts to recover or use default values where necessary.

The end result is a comprehensive, balanced, and visually appealing article that thoroughly explores the question from multiple angles, providing users with an in-depth analysis of the topic.

This automated process allows for rapid creation of informative content on a wide range of topics, leveraging artificial intelligence to gather, analyze, and present information in a user-friendly format.

Learn More: Version History