archethect 🏴

I solved Auditing Skills Benchmarking for us with 🤍opensource and only for fun and better audits 🤍autoresearch loop 🤍deterministic contests hosting it at forefy.com/benchmarks - benchmark per category (best at logic problems? best at math problems? best poc generator?) - deterministic score (not your typical AI testing AI) - opensource versioned benchmarks, you can create a benchmark or submit benchmark execution results at a cost of several tokens (run locally via your agent) - cross-model scoring, cross-env differentiation sums - autoresearch incentive to run - if you run it locally you are not only contributing to the truth of the benchmark, but also IMPROVE LOCALLY the version of your skill, just for yourself (share if you want but don't have to) - safe self-benchmarking is possible via commit-pinned, audited skills (you're running audited skills in the benchmark loop) - don't want to run? just enjoy knowing the best skills out there per your use-case - clear benchmark winner leaderboards - yeah, we will also benchmark all-in-one audit skills (anything from the auditor skill registry) benchmarks repo: github.com/forefy/benchmarks How to contribute / benefit: - watch for upcoming benchmarks this week (comment below for skills you want to see benchmarked✨✨) - improve the accuracy of published benchmarks by logging in and running your own (only need a few tokens and a agentic CLI) - if you're confused on how to contribute DM me - I heard @archethect is cooking some serious stuff !! and has helped me set this weekend project in motion 🔥 🔁 Repost to have more people contributing and improve benchmark truthfullness !!!