Ed is a PR professional who writes tech cynic newsletter Where’s Your Ed At. He has been a prominent voice in AI backlash in the wake of LLMs
Ed has been critical of AI AI Metrics or more specifically how they are used by large companies to make their systems seem impressive.
There’s an honor system, and honor systems often don’t work when there’s billions of dollars on the line, and there’s no real consequences for actually cheating.
AI metrics did not used to operate on an honour system but in fact we relied upon peer review to make sure that models performed as claimed