Media
1 item
Description
Score the sources behind AI responses. Verity audits the sources behind AI-generated responses so you know whether to trust them — before you cite them, share them, or act on them. Large language models hallucinate. They cite real URLs that link to pages whose content contradicts the claim entirely. They attribute statistics to papers that never made those assertions. They produce confident, well-structured answers backed by sources that don't hold up. Existing tools tell you whether a link resolves. Verity reads the source and determines whether it actually supports what the AI said. ——— HOW IT WORKS ——— Verity runs a 7-stage verification pipeline in the background as you read a ChatGPT response. By the time you finish reading, every cited source has been independently scored. 1. URL Liveness Check — A dead link scores 0 immediately, no exceptions. 2. Content Extraction — Verity fetches the full source body via HTTP GET. For JavaScript-rendered pages, it falls back to a headless browser. Paywalled or bot-blocked pages are explicitly flagged as "content inaccessible — metadata only" rather than silently passed or failed. 3. Signal Extraction — Five signals are extracted in parallel: • Domain credibility (academic journal, official body, established news outlet, etc.) • Publication recency (missing dates are flagged) • Author presence (named authors score higher than corporate bylines) • Relevance to your query • Claim alignment — whether the specific claim made by the AI is actually supported by what the source says 4. Composite Scoring — Signals are combined using a weighted formula. Claim alignment carries the highest weight. A confirmed DOI adds a credibility bonus. 5. Verdict Mapping — Each source receives one of four trust levels: • Reliable (75+): Real, credible, current, and supports the claim • Treat with Caution (50–74): One or more signals are weak or absent • Exercise Skepticism (<50): Multiple signals are poor • Unverified (0): Dead link or content could not be retrieved 6. Local LLM Scoring — A locally hosted Ollama model evaluates each source against the original claim and generates a plain-English verdict and one-sentence implication. All inference runs on your machine. No query content leaves your device. 7. Further Reading — Verity identifies the overarching topic and suggests three pre-verified authoritative sources for further reading, constructed from deterministic search URLs rather than direct model output — eliminating hallucination risk in the recommendations themselves. ——— WHAT MAKES VERITY DIFFERENT ——— Most citation tools confirm that a source exists. Verity determines whether the source contains the information the AI claimed it does. That distinction — between a metadata badge and genuine claim-level verification — is the entire point. ——— HONEST LIMITATIONS ——— Verity is in active development and is genuinely in beta. Getting reliable content from every website on the internet is a hard engineering problem. Cloudflare, paywalls, dynamic rendering, and aggressive bot detection mean some sources will return incomplete results. When that happens, Verity tells you explicitly rather than hiding the failure. Feedback and bug reports are welcomed — every failure mode is a data point. ——— PRIVACY ——— All scoring and inference runs locally via o4-mini. No user data, query content, or browsing activity is transmitted to external servers or sold to third parties. ——— BUILT AT ——— First Place — EWB Annual Hackathon, University of Toronto. Recognized for innovation in AI accountability and citation integrity.