UpTrajectory Review

A Hacker News contributor has built a public tracker that catalogs which AI benchmark challenges have actually been solved versus which remain open, with the explicit aim of monitoring 'goalpost shifting' as capabilities advance. The post itself is minimal—just a URL and engagement metrics—but the linked project addresses a growing frustration in both research and business communities: the lack of clear, persistent records of what AI systems can and cannot do at any given moment.

For small-business operators navigating AI adoption decisions, this tracker serves a practical purpose beyond academic interest. When vendors claim their tools can 'handle customer service' or 'automate content creation,' business owners need reference points to evaluate whether those claims represent solved problems or aspirational marketing. A public record of demonstrated capabilities—separated from promotional rhetoric—provides grounding for procurement decisions and helps operators avoid overpaying for AI features that remain unreliable or experimental.

The 115 comments suggest significant community engagement, likely reflecting ongoing debates about benchmark validity and the tendency to redefine success thresholds once AI systems approach them. This tracker implicitly challenges the AI industry's incentive structure: companies benefit from ambiguity about capabilities, while buyers and regulators need precision. The project's value lies in forcing specificity—either a challenge has been demonstrably solved under defined conditions, or it hasn't.

Second-order effects could be substantial. If widely adopted, such tracking might pressure AI developers to be more conservative in capability claims, potentially slowing hype cycles but improving trust. Conversely, it could accelerate competitive pressure as gaps become publicly visible. For businesses, clearer capability maps enable better workforce planning: knowing which tasks AI genuinely handles well versus where human oversight remains essential affects hiring, training, and process design decisions.

Watch whether this tracker gains institutional support from research organizations or standards bodies, and whether major AI labs engage with or dismiss its methodology. Business operators should bookmark the resource and check it before major AI investments, particularly when evaluating vendor claims about 'autonomous' capabilities. The tracker won't eliminate uncertainty, but it provides a community-maintained reality check against which to measure promises.

Takeaway: Before accepting AI vendor claims, check public capability trackers to verify whether promised features represent solved problems or marketing aspirations.