{ "Ad-Hoc Evaluation of Agents": { "companies": { "major": [ { "name": "Braintrust", "rationale": "

Builds a production evaluation platform with tracing, datasets, scorers, and agent-specific task evaluation. It is one of the leading commercial tools for testing whether agents work before shipping or selecting them.

", "url": "https://www.braintrust.dev/" }, { "name": "LangChain / LangSmith", "rationale": "

LangSmith provides tracing, trajectory evaluation, datasets, and regression testing for agents and LLM applications. Its ecosystem reach makes it central to practical agent eval workflows.

", "url": "https://www.langchain.com/langsmith" }, { "name": "Patronus AI", "rationale": "

Provides evaluators, tracing, monitoring, and agent failure analysis for production AI systems. It is important for independent automated assessment of agent behavior and safety.

", "url": "https://www.patronus.ai/" }, { "name": "Arize AI / Phoenix", "rationale": "

Phoenix is a widely used open-source observability and evaluation platform for LLM and agent applications. Its tracing and eval stack supports evidence-based agent comparison and debugging.

", "url": "https://phoenix.arize.com/" }, { "name": "Olas", "rationale": "

Runs Mech Marketplace, where autonomous agents can offer off-chain services and earn rewards. It is a concrete decentralized agent-service marketplace where capability and trust signals matter.

", "url": "https://olas.network/mech-marketplace" }, { "name": "Virtuals Protocol", "rationale": "

Its Agent Commerce Protocol includes escrow, agreement verification, evaluation phases, evaluator agents, and reputation. This is directly aligned with agent-to-agent commerce and pre-commitment trust.

", "url": "https://whitepaper.virtuals.io/about-virtuals/agent-commerce-protocol" } ] }, "nonprofits": { "major": [ { "name": "Princeton Language and Intelligence", "rationale": "

Home to major agent evaluation work including SWE-bench and related Princeton agent research. It is one of the most central academic hubs for measuring real-world agent capability.

", "url": "https://pli.princeton.edu/" }, { "name": "METR", "rationale": "

Specializes in empirical evaluations of frontier AI capabilities, autonomy, and task horizons. While safety-oriented, its methodology is highly relevant to estimating whether agents can complete tasks before deployment.

", "url": "https://metr.org/" }, { "name": "THUDM / Tsinghua Knowledge Engineering Group", "rationale": "

Maintains AgentBench, an influential benchmark suite for evaluating LLMs as agents across interactive environments. It is important for cross-domain agent capability measurement.

", "url": "https://github.com/THUDM/AgentBench" } ] }, "papers": { "major": [ { "name": "AgentSelect: Benchmark for Narrative Query-to-Agent Recommendation", "rationale": "

Reframes agent choice as query-to-agent recommendation over capability profiles, with large-scale interaction data and marketplace transfer. It is the most direct research artifact for choosing unknown agents before hiring them.

", "url": "https://arxiv.org/abs/2603.03761" }, { "name": "τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains", "rationale": "

Evaluates tool-using conversational agents in realistic enterprise domains with policies, APIs, users, and database-state checks. It gives buyers a task-grounded signal of whether an agent can actually complete service work.

", "url": "https://arxiv.org/abs/2406.12045" }, { "name": "SWE-bench: Can Language Models Resolve Real-World GitHub Issues?", "rationale": "

Uses real GitHub issues and tests to evaluate software engineering agents by execution outcome. It became a canonical capability signal for selecting coding agents and agent scaffolds.

", "url": "https://arxiv.org/abs/2310.06770" }, { "name": "AgentBench: Evaluating LLMs as Agents", "rationale": "

An early multi-environment benchmark for evaluating LLMs as interactive agents across OS, database, web, game, and reasoning tasks. It helped establish cross-domain agent capability measurement.

", "url": "https://arxiv.org/abs/2308.03688" }, { "name": "GAIA: a benchmark for General AI Assistants", "rationale": "

Benchmarks general assistants on real-world questions requiring tool use, web search, file handling, and multi-step reasoning. It is widely used as a coarse capability signal for general-purpose agents.

", "url": "https://arxiv.org/abs/2311.12983" }, { "name": "WebArena: A Realistic Web Environment for Building Autonomous Agents", "rationale": "

Provides sandboxed, functional web applications and programmatic task checks for autonomous web agents. It matters for evaluating whether an agent can execute browser-based work rather than merely answer questions.

", "url": "https://arxiv.org/abs/2307.13854" }, { "name": "When Agent Markets Arrive", "rationale": "

Introduces Diagon, a programmable agent market with job posting, bidding, execution, payment, and reputation. It directly studies how market design and selection rules shape agent economies.

", "url": "https://arxiv.org/abs/2604.06688" } ] }, "people": { "major": [ { "name": "Shunyu Yao", "rationale": "

Key contributor to ReAct, SWE-bench, and τ-bench, spanning reasoning-action methods and practical agent evaluation. His work is central to measuring whether agents can execute real tasks.

", "url": "https://ysymyth.github.io/" }, { "name": "Karthik Narasimhan", "rationale": "

Princeton and Sierra researcher behind major agent benchmarks including SWE-bench and τ-bench. His group has shaped how tool-using and software agents are evaluated.

", "url": "https://www.cs.princeton.edu/~karthikn/" }, { "name": "Carlos E. Jimenez", "rationale": "

Lead author of SWE-bench and related software-agent evaluation work. SWE-bench is one of the clearest examples of verifying task capability before trusting an agent.

", "url": "https://www.carlosejimenez.com/" }, { "name": "Clémentine Fourrier", "rationale": "

Coauthor of GAIA and a visible contributor to open evaluation practice. GAIA is a widely used signal for general assistant capability in agent selection contexts.

", "url": "https://huggingface.co/clefourrier" }, { "name": "Graham Neubig", "rationale": "

Senior contributor to WebArena and broader agent evaluation and software-agent work. His projects provide realistic environments for testing agents before deployment or selection.

", "url": "https://www.phontron.com/" } ] }, "tags": [ "multi-agent systems", "oversight", "infrastructure" ] }, "Agent Heterogeneity": { "companies": { "major": [ { "name": "Microsoft AutoGen", "rationale": "

AutoGen is one of the canonical open-source frameworks for multi-agent LLM applications. It is central because agents can be customized across LLMs, tools, humans, and conversation patterns.

", "url": "https://github.com/microsoft/autogen" }, { "name": "Together AI", "rationale": "

Together AI introduced and operationalized the Mixture-of-Agents reference implementation using multiple open models. Its inference platform also supports the open-model diversity needed for heterogeneous agent stacks.

", "url": "https://www.together.ai/" }, { "name": "CrewAI", "rationale": "

CrewAI is a widely used framework for assembling role-specialized agent crews. It is relevant because its abstractions encourage differentiated agents and can be paired with multiple model providers.

", "url": "https://www.crewai.com/" } ] }, "nonprofits": { "major": [] }, "papers": { "major": [ { "name": "Mixture-of-Agents Enhances Large Language Model Capabilities", "rationale": "

Introduces a layered Mixture-of-Agents architecture that combines outputs from multiple LLM agents. It is a core reference for using model diversity at inference time instead of relying on one frontier model.

", "url": "https://arxiv.org/abs/2406.04692" }, { "name": "X-MAS: Towards Building Multi-Agent Systems with Heterogeneous LLMs", "rationale": "

Directly studies heterogeneous LLM-driven multi-agent systems and introduces X-MAS-Bench for choosing models by function and domain. It is one of the most on-point papers for replacing homogeneous agent teams with deliberately diverse ones.

", "url": "https://arxiv.org/abs/2505.16997" }, { "name": "Position: Stop Overvaluing Multi-Agent Debate: We Must Rethink Evaluation and Embrace Model Heterogeneity", "rationale": "

Systematically evaluates multi-agent debate and argues that model heterogeneity is a key design principle. It is important because it reframes debate gains as fragile unless agent diversity is made explicit.

", "url": "https://openreview.net/forum?id=tMJvb9JDsd" }, { "name": "LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion", "rationale": "

Builds an ensemble system that ranks and fuses outputs from multiple LLMs. It is a foundational precursor to heterogeneous agent tooling because it operationalizes complementary model strengths.

", "url": "https://aclanthology.org/2023.acl-long.792/" }, { "name": "AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation", "rationale": "

Introduces a widely used framework for composing conversable agents with configurable LLMs, tools, humans, and interaction patterns. It matters as practical infrastructure for building non-identical agent teams.

", "url": "https://arxiv.org/abs/2308.08155" }, { "name": "RMoA: Optimizing Mixture-of-Agents through Diversity Maximization and Residual Compensation", "rationale": "

Improves Mixture-of-Agents with explicit diversity selection and residual aggregation. It is important for turning diversity from an informal ensemble idea into a selection objective.

", "url": "https://aclanthology.org/2025.findings-acl.342/" }, { "name": "Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?", "rationale": "

Challenges naive model mixing by showing that multiple samples from a strong single model can outperform mixed-model MoA in many settings. It is a necessary counterweight for the field because it clarifies when heterogeneity helps versus when quality dilution dominates.

", "url": "https://arxiv.org/abs/2502.00674" } ] }, "people": { "major": [ { "name": "Qingyun Wu", "rationale": "

Co-creator of AutoGen and founder of AG2ai, with a research focus on AI agents, multi-agent systems, and automated machine learning. He is central to practical tooling for configurable agent teams.

", "url": "https://qingyun.me/" }, { "name": "Chi Wang", "rationale": "

Founder and key technical lead behind AutoGen, one of the most influential open-source multi-agent frameworks. His work is important for making agent composition, model choice, and tool integration programmable.

", "url": "https://openreview.net/profile?id=~Chi_Wang3" }, { "name": "Rui Ye", "rationale": "

Lead author of X-MAS and MASLab, both directly concerned with building and evaluating LLM-based multi-agent systems. His work is especially relevant to heterogeneous model assignment across agent functions.

", "url": "https://rui-ye.github.io/" }, { "name": "Yilun Du", "rationale": "

Lead author of the foundational multiagent debate paper. His work established a key baseline for later research asking whether debate agents should be genuinely diverse rather than copies.

", "url": "https://yilundu.github.io/" }, { "name": "Bill Yuchen Lin", "rationale": "

Co-author of LLM-Blender and related work on model ranking, fusion, evaluation, and agents. His work is important for combining outputs from different models into stronger systems.

", "url": "https://yuchenlin.xyz/" }, { "name": "James Zou", "rationale": "

Senior co-author of Mixture-of-Agents and a Stanford researcher on reliable AI and autonomous AI scientist systems. His group connects model ensembles, agent collaboration, and robustness in applied settings.

", "url": "https://profiles.stanford.edu/james-zou" } ] }, "tags": [ "multi-agent systems", "alignment" ] }, "Agent Routing": { "companies": { "major": [ { "name": "Martian", "rationale": "

Commercializes model routing and co-released RouterBench. It is one of the clearest companies built around prompt-level selection among heterogeneous LLM providers.

", "url": "https://withmartian.com/" }, { "name": "OpenRouter", "rationale": "

Runs a large unified model gateway with provider routing, fallback, and automatic routing features. Its scale makes it important infrastructure for applications that need to reach many model providers through one interface.

", "url": "https://openrouter.ai/" }, { "name": "Not Diamond", "rationale": "

Offers pretrained and custom LLM routers that optimize quality, cost, and latency. It is a focused company in the routing layer rather than a generic agent framework.

", "url": "https://www.notdiamond.ai/" }, { "name": "Unify", "rationale": "

Built benchmark-driven dynamic LLM routing that selects models and providers by quality, speed, and cost preferences. It is a notable commercial attempt to make model choice automatic per prompt.

", "url": "https://unify.ai/" }, { "name": "Fetch.ai Agentverse", "rationale": "

Provides an agent discovery and marketplace platform in the Fetch.ai ecosystem. It is directly relevant to agent routing because agents need to be discoverable, comparable, and invocable by users or other agents.

", "url": "https://www.fetch.ai/agentverse" }, { "name": "Katanemo", "rationale": "

Builds Arch Gateway and Arch-Router for AI-native gateways and preference-aligned LLM routing. Its work connects agent gateways, policy, and model selection.

", "url": "https://katanemo.com/" } ] }, "nonprofits": { "major": [ { "name": "AGNTCY", "rationale": "

Linux Foundation project building open infrastructure for the Internet of Agents, including discovery, identity, messaging, and observability. Its Agent Directory Service is especially central to open agent routing.

", "url": "https://agntcy.org/" }, { "name": "LMSYS Org", "rationale": "

Open systems organization behind RouteLLM and major preference-data resources such as Chatbot Arena. It is important to routing because learned routers often depend on comparative model outcome data.

", "url": "https://lmsys.org/" } ] }, "papers": { "major": [ { "name": "The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver", "rationale": "

Foundational paper for allocating tasks among autonomous agents through calls for proposals, bids, and awards. It remains the classic precedent for routing work to capable agents in a distributed economy.

", "url": "https://doi.org/10.1109/TC.1980.1675516" }, { "name": "FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance", "rationale": "

Introduces LLM cascades that learn which models to call for different queries. It established cost and quality aware routing as a practical alternative to always using the largest model.

", "url": "https://arxiv.org/abs/2305.05176" }, { "name": "RouterBench: A Benchmark for Multi-LLM Routing System", "rationale": "

Provides a benchmark and large dataset of model outcomes for evaluating LLM routers. It helped make routing a measurable systems problem rather than an ad hoc product feature.

", "url": "https://arxiv.org/abs/2403.12031" }, { "name": "Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing", "rationale": "

Proposes routing between smaller and larger models based on predicted query difficulty and a tunable quality target. It is an important early formulation of quality constrained routing.

", "url": "https://arxiv.org/abs/2404.14618" }, { "name": "RouteLLM: Learning to Route LLMs with Preference Data", "rationale": "

Trains router models from human preference data to choose between strong and weak LLMs. Its open framework is widely referenced for production style LLM routing.

", "url": "https://arxiv.org/abs/2406.18665" }, { "name": "MasRouter: Learning to Route LLMs for Multi-Agent Systems", "rationale": "

Defines multi-agent system routing, including collaboration mode, role allocation, and model assignment. It is one of the most direct research contributions to routing sets of agents rather than single models.

", "url": "https://arxiv.org/abs/2502.11133" }, { "name": "The AGNTCY Agent Directory Service: Architecture and Implementation", "rationale": "

Specifies a distributed directory for agent capabilities, metadata, provenance, and discovery. It is central infrastructure for open agent routing because routers need reliable ways to find and verify candidate agents.

", "url": "https://arxiv.org/abs/2509.18787" } ] }, "people": { "major": [ { "name": "Reid G. Smith", "rationale": "

Author of the Contract Net Protocol, the classic task-allocation mechanism for distributed problem solvers. His work is the historical root of many agent negotiation and routing ideas.

", "url": "https://reidgsmith.com/" }, { "name": "Lingjiao Chen", "rationale": "

Lead author of FrugalGPT, a foundational LLM cascade and routing paper. Her work helped define cost-aware model selection for heterogeneous AI providers.

", "url": "https://lingjiaochen.com/" }, { "name": "Matei Zaharia", "rationale": "

Coauthor of FrugalGPT and a leading systems researcher. His work connects routing to scalable, cost-efficient AI serving.

", "url": "https://cs.stanford.edu/~matei/" }, { "name": "James Zou", "rationale": "

Coauthor of FrugalGPT and a prominent researcher in trustworthy and efficient AI systems. His contributions helped legitimize routing as a practical deployment strategy.

", "url": "https://www.james-zou.com/" }, { "name": "Wei-Lin Chiang", "rationale": "

Coauthor of RouteLLM and a key LMSYS researcher. His work on model evaluation and preference data is directly tied to learning routers from real model comparisons.

", "url": "https://infwinston.github.io/" }, { "name": "Joseph E. Gonzalez", "rationale": "

Coauthor of RouteLLM and a major figure in ML systems. He is included for work on scalable serving and routing frameworks for LLM deployment.

", "url": "https://people.eecs.berkeley.edu/~jegonzal/" }, { "name": "Ion Stoica", "rationale": "

Coauthor of RouteLLM and a leading distributed systems researcher. His involvement anchors LLM routing in broader systems and serving infrastructure.

", "url": "https://people.eecs.berkeley.edu/~istoica/" }, { "name": "Kurt Keutzer", "rationale": "

Coauthor of RouterBench and a senior Berkeley systems researcher. His lab's work helped establish standardized evaluation for multi-LLM routing.

", "url": "https://people.eecs.berkeley.edu/~keutzer/" } ] }, "tags": [ "multi-agent systems", "infrastructure", "economics" ] }, "Agentic Security": { "companies": { "major": [ { "name": "Invariant Labs", "rationale": "

Builds security tooling for AI agents and has produced core research such as formal security guarantees and AgentDojo contributions. It is one of the most directly focused companies in agentic security.

", "url": "https://invariantlabs.ai/" }, { "name": "Lakera", "rationale": "

Provides Lakera Guard and related AI security tools for prompt injection, data leakage, and agent behavior defense. Its Gandalf and attack-data efforts have made it a prominent commercial actor in LLM security.

", "url": "https://www.lakera.ai/" }, { "name": "Straiker", "rationale": "

Offers runtime guardrails and red teaming for agentic applications, including protection against tool misuse, data leakage, and agent manipulation. Its positioning is specifically centered on securing production agents.

", "url": "https://www.straiker.ai/" }, { "name": "Zenity", "rationale": "

Focuses on agent-centric security and governance across enterprise copilots and agent platforms. It emphasizes runtime monitoring of agent execution paths, tool calls, memory access, and prompt injection attempts.

", "url": "https://zenity.io/" }, { "name": "WitnessAI", "rationale": "

Provides enterprise AI security and governance with controls for data flows and agent actions. It is notable for positioning agent security as a core enterprise security category.

", "url": "https://witness.ai/" }, { "name": "Prompt Security", "rationale": "

Builds runtime GenAI and agent security controls, including prompt injection prevention, data leakage controls, and MCP gateway security. Its acquisition by SentinelOne reflects the category's commercial importance.

", "url": "https://prompt.security/" } ] }, "nonprofits": { "major": [ { "name": "OWASP GenAI Security Project / Agentic Security Initiative", "rationale": "

Maintains community guidance for generative and agentic AI security, including the OWASP Top 10 for Agentic Applications. It is the most visible open standards effort for agent-specific risks and mitigations.

", "url": "https://genai.owasp.org/initiatives/agentic-security-initiative/" }, { "name": "Cloud Security Alliance MAESTRO", "rationale": "

Provides a multi-agent threat modeling framework for agentic AI systems. CSA's related agentic control-plane and IAM guidance make it a central standards-oriented effort for enterprise agent security.

", "url": "https://labs.cloudsecurityalliance.org/maestro/" }, { "name": "MITRE ATLAS", "rationale": "

Maintains a living knowledge base of adversary tactics and techniques for AI-enabled systems, including LLM prompt injection and AI agent tool data poisoning. It is important for mapping agent attacks into a threat-informed defense model.

", "url": "https://atlas.mitre.org/" } ] }, "papers": { "major": [ { "name": "Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection", "rationale": "

Introduced and systematized indirect prompt injection against LLM-integrated applications, including realistic attacks through retrieved web and tool content. It is a foundational threat model for agentic security because agents act on untrusted external data.

", "url": "https://arxiv.org/abs/2302.12173" }, { "name": "AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents", "rationale": "

Defines an extensible benchmark for tool-using agents operating over untrusted data, with realistic tasks and security test cases. It became a central evaluation environment for agent prompt injection defenses.

", "url": "https://arxiv.org/abs/2406.13352" }, { "name": "InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents", "rationale": "

Introduces a benchmark of more than a thousand indirect prompt injection cases across user and attacker tools. It helped establish that tool-integrated agents face concrete data exfiltration and harmful-action risks.

", "url": "https://arxiv.org/abs/2403.02691" }, { "name": "Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents", "rationale": "

Formalizes attack and defense evaluation for LLM agents across many scenarios, tools, attacks, defenses, and model backbones. It is one of the broadest early benchmark suites for agent security.

", "url": "https://arxiv.org/abs/2410.02644" }, { "name": "AI Agents with Formal Security Guarantees", "rationale": "

Proposes adding a formal security analyzer around agents, enforcing explicit policy rules before actions execute. It is important because it shifts defenses from prompt-level detection to hard constraints on agent behavior.

", "url": "https://invariantlabs.ai/theme/research/ai_agents_with_formal_security.pdf" }, { "name": "Defeating Prompt Injections by Design", "rationale": "

Introduces CaMeL, a system defense that separates trusted control flow from untrusted data flow and uses capabilities to prevent unauthorized leakage. It is a leading design-oriented defense for secure LLM agents.

", "url": "https://arxiv.org/abs/2503.18813" }, { "name": "Progent: Programmable Privilege Control for LLM Agents", "rationale": "

Introduces a domain-specific language and runtime mechanism for least-privilege control over agent tool calls. It is central to the access-control view of agentic security.

", "url": "https://arxiv.org/abs/2504.11703" }, { "name": "A Framework for Formalizing LLM Agent Security", "rationale": "

Defines contextual security properties for LLM agents, including task alignment, action alignment, source authorization, and data isolation. It provides a formal vocabulary for comparing attacks and defenses.

", "url": "https://arxiv.org/abs/2603.19469" } ] }, "people": { "major": [ { "name": "Florian Tramèr", "rationale": "

Coauthor of AgentDojo and CaMeL, with a broader research program in machine learning security. His work anchors both benchmark design and system-level defenses for agent prompt injection.

", "url": "https://floriantramer.com/" }, { "name": "Dawn Song", "rationale": "

Leads a major AI security research group with repeated contributions to LLM agent security, including Progent, GuardAgent, and formal agent-security frameworks. Her group has shaped both attack taxonomies and defense mechanisms.

", "url": "https://people.eecs.berkeley.edu/~dawnsong/" }, { "name": "Daniel Kang", "rationale": "

Senior author on InjecAgent, one of the key benchmarks for indirect prompt injection in tool-integrated agents. His lab has helped quantify the practical vulnerability of deployed agent designs.

", "url": "https://kang.cs.illinois.edu/" }, { "name": "Edoardo Debenedetti", "rationale": "

Lead author on AgentDojo and CaMeL, two central works in evaluating and defending tool-using LLM agents. His work directly targets prompt injection robustness in agentic systems.

", "url": "https://agentdojo.spylab.ai/" }, { "name": "Kai Greshake", "rationale": "

Lead author of the foundational indirect prompt injection paper, which framed untrusted retrieved data as an attack channel for LLM-integrated applications. This threat model is core to agentic security.

", "url": "https://arxiv.org/abs/2302.12173" }, { "name": "Martin Vechev", "rationale": "

Research leader behind formal methods for safe and secure AI systems, including Invariant's work on formally constrained agents. His group connects program analysis and policy enforcement to agent security.

", "url": "https://www.sri.inf.ethz.ch/people/martin" }, { "name": "Chaowei Xiao", "rationale": "

Frequent contributor to agent prompt-injection attacks and defenses, including AutoHijacker, WASP-related work, AgentDyn, FATH, and system-level defense papers. He is central to the adversarial evaluation side of the field.

", "url": "https://xiaocw11.github.io/" } ] }, "tags": [ "safety", "multi-agent systems", "oversight" ] }, "Anomaly & Drift Detection in (Multi-)Agentic AI": { "companies": { "major": [ { "name": "Arize AI", "rationale": "

Arize and Phoenix are widely used AI observability and evaluation platforms with tracing, evals, root-cause workflows, and drift-oriented monitoring. They are central tooling for turning agent traces into anomaly signals.

", "url": "https://arize.com/" }, { "name": "LangChain / LangSmith", "rationale": "

LangSmith is tightly connected to LangChain and LangGraph agent development, with tracing, evaluation, and monitoring for tool calls and decision paths. Its importance comes from the adoption of the LangChain agent ecosystem.

", "url": "https://www.langchain.com/langsmith" }, { "name": "Langfuse", "rationale": "

Open-source LLM engineering platform for tracing, evals, prompt management, datasets, and agent graphs. It is a major practical substrate for detecting regressions in production agent workflows.

", "url": "https://langfuse.com/" }, { "name": "WhyLabs", "rationale": "

AI observability platform focused on monitoring degradation, data quality, schema changes, drift, and LLM guardrails. It is especially strong on the classic model-monitoring and drift side of the area.

", "url": "https://whylabs.ai/" }, { "name": "Fiddler AI", "rationale": "

Enterprise AI observability platform for ML models, LLM applications, and emerging multi-agent systems. Its relevance is production monitoring, drift detection, explainability, and security workflows.

", "url": "https://www.fiddler.ai/" }, { "name": "Galileo", "rationale": "

GenAI evaluation and observability platform where offline evals become production guardrails. It is relevant because anomaly detection for agents often depends on continuous eval signals, not only infrastructure telemetry.

", "url": "https://www.galileo.ai/" }, { "name": "AgentOps", "rationale": "

Purpose-built observability platform for AI agents, with session traces, replay, cost, latency, and framework integrations. It is one of the more agent-specific companies in the space.

", "url": "https://www.agentops.ai/" }, { "name": "Datadog LLM Observability", "rationale": "

Datadog brings LLM and agent monitoring into a mainstream observability platform with OpenTelemetry support. It matters for teams that need agent anomalies correlated with infrastructure, service, and security telemetry.

", "url": "https://www.datadoghq.com/product/llm-observability/" } ] }, "nonprofits": { "major": [ { "name": "Cloud Native Computing Foundation / OpenTelemetry", "rationale": "

OpenTelemetry is the core vendor-neutral standard for traces, metrics, and logs, including emerging GenAI and agent semantic conventions. Interoperable telemetry is a prerequisite for decentralized agent health monitoring.

", "url": "https://www.cncf.io/projects/opentelemetry/" }, { "name": "MITRE ATLAS", "rationale": "

Living knowledge base of adversary tactics and techniques against AI-enabled systems. It helps connect anomaly detection in agentic systems to concrete threat models and detection opportunities.

", "url": "https://atlas.mitre.org/" } ] }, "papers": { "major": [ { "name": "Anomaly Detection: A Survey", "rationale": "

Canonical survey and taxonomy for anomaly types, data assumptions, and detection methods. It is the broad base for modeling normal behavior so meaningful deviations stand out.

", "url": "https://doi.org/10.1145/1541880.1541882" }, { "name": "A Survey on Concept Drift Adaptation", "rationale": "

Organizes drift detection, adaptation, and evaluation for nonstationary learning systems. It anchors the slow policy and distribution drift side of this area.

", "url": "https://doi.org/10.1145/2523813" }, { "name": "Learning with Drift Detection", "rationale": "

Introduces DDM, monitoring online error to flag warning and drift levels. It remains a standard baseline for detecting when a learned model stops matching its environment.

", "url": "https://doi.org/10.1007/978-3-540-28645-5_29" }, { "name": "Learning from Time-Changing Data with Adaptive Windowing", "rationale": "

Introduces ADWIN, adaptive windows with statistical guarantees for changing streams. It matters because agent fleets need online baselines that expand in stable periods and shrink around change.

", "url": "https://doi.org/10.1137/1.9781611972771.42" }, { "name": "Decentralized Anomaly Detection in Cooperative Multi-Agent Reinforcement Learning", "rationale": "

Formulates anomaly detection inside cooperative MARL without relying on a central monitor. It is one of the clearest papers matching self-policing multi-agent health.

", "url": "https://doi.org/10.24963/ijcai.2023/19" }, { "name": "AgentOps: Enabling Observability of LLM Agents", "rationale": "

Defines telemetry and observability artifacts for LLM agents across their lifecycle. It gives the trace substrate needed before anomalies or drift can be detected.

", "url": "https://arxiv.org/abs/2411.05285" }, { "name": "SentinelAgent: Graph-based Anomaly Detection in Multi-Agent Systems", "rationale": "

Models LLM multi-agent executions as dynamic graphs and detects node, edge, and path anomalies. It is a central recent attempt to catch systemic agent failures and covert risk paths.

", "url": "https://arxiv.org/abs/2505.24201" }, { "name": "Detecting Silent Failures in Multi-Agentic AI Trajectories", "rationale": "

Creates benchmark datasets for silent failures in multi-agent trajectories, including drift, cycles, tool failures, and missing details. It is directly about detecting failures that ordinary error logs miss.

", "url": "https://arxiv.org/abs/2511.04032" } ] }, "people": { "major": [ { "name": "João Gama", "rationale": "

Foundational concept-drift researcher and coauthor of DDM and the ACM Computing Surveys drift adaptation survey. His work defines much of the online drift detection vocabulary.

", "url": "https://www.dcc.fc.up.pt/~jgama/" }, { "name": "Albert Bifet", "rationale": "

Coauthor of ADWIN and a central figure in data-stream mining and adaptive learning. His work is important for online baselines in nonstationary agent environments.

", "url": "https://albertbifet.com/" }, { "name": "Varun Chandola", "rationale": "

Lead author of the influential ACM anomaly detection survey. His work helped standardize how anomaly detection problems are framed across domains.

", "url": "https://cse.buffalo.edu/~chandola/about.html" }, { "name": "Indrė Žliobaitė", "rationale": "

Major researcher on learning from evolving data and coauthor of the concept drift adaptation survey. Her work is central to evaluating drift under changing data histories.

", "url": "https://www.zliobaite.com/" }, { "name": "György Dán", "rationale": "

Researches secure and dependable networked systems and multi-agent learning. He is directly relevant through work on decentralized anomaly detection in cooperative MARL.

", "url": "https://people.kth.se/~gyuri/" }, { "name": "Qingyun Wu", "rationale": "

Works on LLM multi-agent systems, AutoGen-style agent frameworks, AgentOps, and automated failure attribution. This makes her central to the agentic observability side of the area.

", "url": "https://qingyun-wu.github.io/" }, { "name": "Kun Sun", "rationale": "

Systems security researcher working on LLM agent risks and multi-agent anomaly detection. He is a key author behind SentinelAgent-style graph monitoring for agent systems.

", "url": "https://cs.gmu.edu/~ksun3/" } ] }, "tags": [ "safety", "oversight", "multi-agent systems" ] }, "Automated Mechanism Design": { "companies": { "major": [ { "name": "Salesforce AI Research, The AI Economist", "rationale": "

Research project that uses reinforcement learning to automate economic policy and incentive design. It is one of the most visible company-backed efforts in AI-driven mechanism and policy design.

", "url": "https://www.salesforceairesearch.com/projects/the-ai-economist" }, { "name": "Google Research", "rationale": "

Google researchers have been central to differentiable economics, including the key deep optimal auctions work. The lab is also active in newer mechanism design problems involving AI agents and LLMs.

", "url": "https://research.google/pubs/optimal-auctions-through-deep-learning/" } ] }, "nonprofits": { "major": [ { "name": "ACM SIGecom", "rationale": "

The central professional community for economics and computation, including ACM EC and SIGecom Exchanges. It is the main venue ecosystem for AMD papers, surveys, and researchers.

", "url": "https://www.sigecom.org/" } ] }, "papers": { "major": [ { "name": "Complexity of Mechanism Design", "rationale": "

Foundational paper by Conitzer and Sandholm that introduced automated mechanism design as computing a mechanism for the specific instance at hand. It established key complexity barriers and helped define the optimization view of mechanism design.

", "url": "https://arxiv.org/abs/cs/0205075" }, { "name": "Computationally Feasible Automated Mechanism Design: General Approach and Case Studies", "rationale": "

Introduces a scalable parametric approach to AMD, replacing exhaustive search over mechanisms with search over carefully chosen mechanism families. It is a central bridge from the original LP framing toward practical automated design.

", "url": "https://ojs.aaai.org/index.php/AAAI/article/view/7708" }, { "name": "Automated Design of Revenue-Maximizing Combinatorial Auctions", "rationale": "

Landmark work on automated design for high-revenue combinatorial auctions using parameterized VCG-based and affine maximizer auction families. It made AMD concrete for a core unsolved multi-item auction problem.

", "url": "https://doi.org/10.1287/opre.2015.1398" }, { "name": "Sample Complexity of Automated Mechanism Design", "rationale": "

Gives the first sample-complexity analysis for standard deterministic combinatorial auction classes used in AMD. It put data-driven mechanism search on firmer learning-theoretic foundations.

", "url": "https://papers.nips.cc/paper_files/paper/2016/hash/c667d53acd899a97a85de0c201ba99be-Abstract.html" }, { "name": "Optimal Auctions through Deep Learning: Advances in Differentiable Economics", "rationale": "

Introduced the deep learning approach to optimal auction design, commonly associated with RegretNet and differentiable economics. It launched the modern neural AMD line by optimizing allocation and payment rules from samples.

", "url": "https://proceedings.mlr.press/v97/duetting19a.html" }, { "name": "Automated Mechanism Design: A Survey", "rationale": "

Recent survey that organizes AMD into optimization, learning-theory, and differentiable-economics threads. It is the clearest current map of the field and its open problems.

", "url": "https://www.sigecom.org/exchanges/volume_22/2/CURRY.pdf" } ] }, "people": { "major": [ { "name": "Vincent Conitzer", "rationale": "

Co-originator of automated mechanism design with Tuomas Sandholm. His work spans the foundational complexity framing plus later robust, dynamic, and interpretable AMD variants.

", "url": "https://www.cs.cmu.edu/~conitzer/" }, { "name": "Tuomas Sandholm", "rationale": "

Co-originator of AMD and a central figure in automated auction and marketplace design. His work on revenue-maximizing combinatorial auctions and sample complexity shaped the field's practical and theoretical agenda.

", "url": "https://www.cs.cmu.edu/~sandholm/" }, { "name": "David C. Parkes", "rationale": "

Leading researcher in computational mechanism design, differentiable economics, and AI-based economic design. He is central to RegretNet-style neural auction design, the AI Economist, and current AMD surveys.

", "url": "https://parkes.seas.harvard.edu/" }, { "name": "Maria-Florina Balcan", "rationale": "

Key contributor to the learning-theoretic foundations of mechanism design and AMD. Her work with Sandholm and Vitercik on sample complexity is central for data-driven automated design.

", "url": "https://www.cs.cmu.edu/~ninamf/" }, { "name": "Paul Dütting", "rationale": "

Lead contributor to the deep learning approach to optimal auctions and differentiable economics. His work helped make neural networks a mainstream tool for automated mechanism design.

", "url": "https://paulduetting.com/" } ] }, "tags": [ "economics", "multi-agent systems" ] }, "Automated Protocol Design": { "companies": { "major": [ { "name": "Fetch.ai", "rationale": "

Builds a decentralized agent ecosystem with uAgents, Agentverse, agent discovery, and communication protocols. It is one of the most directly relevant companies for open agent-to-agent interaction infrastructure.

", "url": "https://fetch.ai/" }, { "name": "Olas (Valory)", "rationale": "

Develops the Olas protocol and Open Autonomy stack for decentralized autonomous agent services. Its focus on multiagent services, registries, and coordination makes it highly relevant to protocolized agent interaction.

", "url": "https://olas.network/" } ] }, "nonprofits": { "major": [] }, "papers": { "major": [ { "name": "Protocol Synthesis with Dialogue Structure Theory", "rationale": "

Early direct work on agents synthesizing and communicating interaction protocols during participation. It is highly aligned with on-the-fly protocol generation rather than fixed standards.

", "url": "https://doi.org/10.1007/11794578_13" }, { "name": "Muon: Designing Multiagent Communication Protocols from Interaction Scenarios", "rationale": "

Introduces a commitment-based method for deriving multiagent communication protocols from example interaction scenarios. It is a core paper on semi-automated protocol design for decentralized agents.

", "url": "https://doi.org/10.1007/s10458-014-9264-2" }, { "name": "Tosca: Operationalizing Commitments Over Information Protocols", "rationale": "

Presents Tosca, a technique for automatically synthesizing information protocols from commitment specifications. It connects high-level social meaning to decentralized message protocols.

", "url": "https://www.ijcai.org/proceedings/2017/37" }, { "name": "Clouseau: Generating Communication Protocols from Commitments", "rationale": "

Directly addresses automatic generation of communication protocols from commitment-based interaction specifications. It proves correctness, safety, and liveness for decentralized enactment.

", "url": "https://doi.org/10.1609/aaai.v34i05.6215" }, { "name": "Complexity of Mechanism Design", "rationale": "

Foundational automated mechanism design paper proposing mechanisms computed for the setting at hand. It grounds the mechanism-design side of automated protocol generation.

", "url": "https://arxiv.org/abs/cs/0205075" }, { "name": "An Algorithm for Automatically Designing Deterministic Mechanisms without Payments", "rationale": "

Presents one of the first algorithms specifically for automated mechanism design. It matters because it turns game-rule construction into an algorithmic search problem.

", "url": "https://scholars.duke.edu/publication/766201" }, { "name": "Learning to Communicate with Deep Multi-Agent Reinforcement Learning", "rationale": "

Introduces RIAL and DIAL for end-to-end learning of communication protocols among cooperative agents. It is a landmark in emergent agent communication.

", "url": "https://papers.nips.cc/paper/6042-learning-to-communicate-with-deep-multi-agent-reinforcement-learning" }, { "name": "Learning Multiagent Communication with Backpropagation", "rationale": "

Introduces CommNet, where agents learn continuous communication alongside policy learning. It is central to neural methods for learned coordination protocols.

", "url": "https://papers.nips.cc/paper/6398-learning-multiagent-communication-with-backpropagation" } ] }, "people": { "major": [ { "name": "Munindar P. Singh", "rationale": "

Field-shaping researcher on social commitments, interaction-oriented programming, BSPL, and generated information protocols. His work anchors the commitment-based branch of automated protocol design.

", "url": "https://www.csc2.ncsu.edu/faculty/mpsingh/" }, { "name": "Amit K. Chopra", "rationale": "

Central contributor to commitment alignment, Tosca, and Clouseau. His work is directly about compiling high-level interaction meanings into decentralized protocols.

", "url": "https://www.lancaster.ac.uk/scc/about-us/people/amit-chopra" }, { "name": "Vincent Conitzer", "rationale": "

Co-founder of automated mechanism design as a computational AI research area. His work is foundational for automatically generating rules of interaction among strategic agents.

", "url": "https://www.cs.cmu.edu/~conitzer/" }, { "name": "Tuomas Sandholm", "rationale": "

Co-founder of automated mechanism design and a leading researcher in computational markets and auctions. His work links agent interaction rules to algorithmic optimization.

", "url": "https://www.cs.cmu.edu/~sandholm/" }, { "name": "Jakob Foerster", "rationale": "

Lead author of DIAL and RIAL, landmark methods for agents learning communication protocols. His broader MARL work is central to learned coordination among agents.

", "url": "https://foersterlab.com/" }, { "name": "Sainbayar Sukhbaatar", "rationale": "

Lead author of CommNet, an influential neural architecture for learned multiagent communication. His work is a core reference for differentiable protocol emergence.

", "url": "https://openreview.net/profile?id=~Sainbayar_Sukhbaatar1" }, { "name": "Igor Mordatch", "rationale": "

Lead author on grounded compositional language emergence in multiagent populations. His work is important for agents inventing communication systems from task pressure.

", "url": "https://mordatch.github.io/" }, { "name": "Shimon Whiteson", "rationale": "

Coauthor of DIAL and a leading MARL researcher. His work is relevant to decentralized execution, learned communication, and cooperative agent coordination.

", "url": "https://whiteson.org/" } ] }, "tags": [ "multi-agent systems", "infrastructure", "distributed systems" ] }, "Capability Delegation & Revocation": { "companies": { "major": [ { "name": "Agoric", "rationale": "

Builds a smart-contract platform around object-capability security, Hardened JavaScript, and least-authority patterns. It is one of the main commercial homes for modern ocap systems work.

", "url": "https://agoric.com/" }, { "name": "Storacha", "rationale": "

Operates UCAN-based decentralized storage infrastructure and maintains w3up and ucanto tooling. It is a leading deployed example of capability delegation for services, devices, and user agents.

", "url": "https://storacha.network/" }, { "name": "Digital Bazaar", "rationale": "

Develops linked-data security, DID, VC, and ZCAP-LD software. It is one of the core companies behind web standards work that expresses delegated authorization as capabilities.

", "url": "https://digitalbazaar.com/" }, { "name": "SpruceID", "rationale": "

Important for wallet-based authorization, Sign-In with Ethereum, and ReCaps. Its work brings resource-scoped capabilities into Ethereum and decentralized identity workflows.

", "url": "https://www.spruceid.com/" }, { "name": "Fission Codes", "rationale": "

Historically central to UCAN, WNFS, and local-first authorization tooling. Although closed, it was a major origin point for user-controlled capability delegation in decentralized apps.

", "url": "https://fission.codes/" } ] }, "nonprofits": { "major": [ { "name": "UCAN Working Group", "rationale": "

Maintains the core UCAN specifications for delegation, invocation, and revocation. It is the most directly relevant standards community for user-controlled capability chains.

", "url": "https://ucan.xyz/" }, { "name": "W3C Credentials Community Group", "rationale": "

Incubates decentralized identity and credential work, including ZCAP-LD. It is central for standardizing linked-data authorization capabilities on the web.

", "url": "https://www.w3.org/community/credentials/" }, { "name": "Internet Engineering Task Force", "rationale": "

Published SPKI and many related security delegation and token standards. Its role is foundational for interoperable authorization, certificates, and revocation mechanisms.

", "url": "https://www.ietf.org/" }, { "name": "Spritely Institute", "rationale": "

Develops Goblins and advances OCapN for secure distributed object-capability programming. It is a central nonprofit actor for peer-to-peer capability-based agent interaction.

", "url": "https://spritely.institute/" } ] }, "papers": { "major": [ { "name": "Programming Semantics for Multiprogrammed Computations", "rationale": "

Foundational capability-system paper that introduced unforgeable references for protected sharing of computing objects. It is the starting point for later work on delegated authority and least privilege.

", "url": "https://doi.org/10.1145/365230.365252" }, { "name": "Capability Myths Demolished", "rationale": "

Clarifies why capabilities are not equivalent to ACLs and directly addresses the myth that capability access cannot be revoked. It is central to the modern object-capability security argument.

", "url": "https://papers.agoric.com/papers/capability-myths-demolished/abstract/" }, { "name": "Robust Composition: Towards a Unified Approach to Access Control and Concurrency Control", "rationale": "

Mark Miller's dissertation gives a full object-capability framework for secure cooperation among mutually suspicious components. It connects delegation, confinement, revocation patterns, and distributed object interactions.

", "url": "https://www.erights.org/talks/thesis/" }, { "name": "SPKI Certificate Theory (RFC 2693)", "rationale": "

Defines authorization certificates that bind explicit permissions to keys and names, with delegation and validity mechanisms. It is a key ancestor of certificate-capability systems such as UCAN and ZCAP-LD.

", "url": "https://www.rfc-editor.org/rfc/rfc2693" }, { "name": "Macaroons: Cookies with Contextual Caveats for Decentralized Authorization in the Cloud", "rationale": "

Introduces attenuable bearer credentials with contextual caveats for decentralized delegation. It made practical, web-deployable scoped delegation a mainstream design pattern.

", "url": "https://research.google/pubs/macaroons-cookies-with-contextual-caveats-for-decentralized-authorization-in-the-cloud/" }, { "name": "UCAN Delegation Specification", "rationale": "

Specifies cryptographically verifiable chains of attenuated authority between principals. It is one of the most direct modern specifications for auditable agent-to-agent capability delegation.

", "url": "https://ucan.xyz/delegation/" }, { "name": "UCAN Revocation Specification", "rationale": "

Defines how UCAN delegations can be invalidated and how revocation authority can itself be delegated. It directly targets revocable proof chains for local-first and decentralized systems.

", "url": "https://ucan.xyz/revocation/" }, { "name": "Authorization Capabilities for Linked Data (ZCAP-LD)", "rationale": "

Specifies linked-data object capabilities with delegation chains, caveats, invocations, and revocation hooks. It is the main W3C CCG style approach to web-native capability delegation.

", "url": "https://w3c-ccg.github.io/zcap-spec/" } ] }, "people": { "major": [ { "name": "Mark S. Miller", "rationale": "

Core object-capability theorist behind E, CapTP, Capability Myths Demolished, and Robust Composition. His work is foundational for least-authority delegation and revocation by indirection.

", "url": "https://www.erights.org/" }, { "name": "Carl M. Ellison", "rationale": "

Principal SPKI contributor whose work centered authorization certificates rather than identity certificates. SPKI remains a key ancestor for chained capability-style delegation.

", "url": "https://theworld.com/~cme/html/spki.html" }, { "name": "Butler W. Lampson", "rationale": "

Foundational access-control researcher and coauthor of SDSI and SPKI-related work. His ideas on protection, naming, and authority influenced later capability delegation systems.

", "url": "https://www.microsoft.com/en-us/research/people/blampson/" }, { "name": "Ronald L. Rivest", "rationale": "

Co-creator of SDSI and coauthor of SPKI Certificate Theory. His work helped establish public-key-based local naming and authorization certificates as a basis for delegation.

", "url": "https://people.csail.mit.edu/rivest/" }, { "name": "Brooklyn Zelenka", "rationale": "

Key UCAN originator and local-first authorization researcher. Her work connects capability chains, DIDs, CRDTs, and revocation for user-controlled systems.

", "url": "https://notes.brooklynzelenka.com/" }, { "name": "Manu Sporny", "rationale": "

Major W3C identity standards contributor and central figure in the ZCAP-LD ecosystem. His work connects linked data, verifiable credentials, DIDs, and authorization capabilities.

", "url": "https://manu.sporny.org/" }, { "name": "Dave Longley", "rationale": "

Digital Bazaar technologist behind linked data security libraries and ZCAP-LD implementation work. He is important to practical capability delegation in the W3C CCG stack.

", "url": "https://digitalbazaar.com/" } ] }, "tags": [ "infrastructure", "safety", "distributed systems" ] }, "Collective Oversight": { "companies": { "major": [ { "name": "Kleros", "rationale": "

Kleros is the leading deployed Web3 protocol for decentralized arbitration by crowdsourced jurors. It is the clearest company-scale example of collective adjudication with penalties and incentives.

", "url": "https://kleros.io/" }, { "name": "X Community Notes", "rationale": "

Community Notes is the most prominent live system for crowd-written and crowd-rated public corrections. Its bridging algorithm makes cross-perspective agreement central to oversight.

", "url": "https://communitynotes.x.com/" }, { "name": "Reddit", "rationale": "

Reddit is one of the largest examples of distributed community moderation, with local subreddits setting and enforcing norms through volunteer moderators and user reports. It is a canonical platform case for community-level governance.

", "url": "https://www.reddit.com/" }, { "name": "Stack Exchange", "rationale": "

Stack Exchange uses reputation, voting, flags, review queues, close votes, and elected moderators to distribute quality control. Its long-running peer moderation system is a major model for community enforcement.

", "url": "https://stackexchange.com/" } ] }, "nonprofits": { "major": [ { "name": "Wikimedia Foundation", "rationale": "

Wikimedia supports Wikipedia, one of the largest and most mature systems of volunteer rule enforcement, dispute resolution, and peer governance. It is a central real-world example of collective oversight at scale.

", "url": "https://wikimediafoundation.org/" }, { "name": "Metagov", "rationale": "

Metagov is a research and practitioner community focused on digital self-governance. It has directly supported Modular Politics, PolicyKit-related infrastructure, and governance experiments for online communities.

", "url": "https://metagov.org/" }, { "name": "Citizens and Technology Lab", "rationale": "

CAT Lab works with online communities to study and test interventions for healthier digital spaces. Its community-engaged research makes it a key nonprofit lab for evidence-based collective moderation.

", "url": "https://citizensandtech.org/" } ] }, "papers": { "major": [ { "name": "Collective Action and the Evolution of Social Norms", "rationale": "

Ostrom gives the foundational account of how communities solve collective action problems through reciprocity, monitoring, and graduated sanctions. It is a core theoretical basis for crowd-enforced norms without a central referee.

", "url": "https://doi.org/10.1257/jep.14.3.137" }, { "name": "Incentivising Monitoring in Open Normative Systems", "rationale": "

This paper directly models norm monitoring in open systems such as Wikipedia, where violations are matters of judgment rather than crisp rule matching. Its peer-prediction and scrip mechanisms are highly aligned with collective oversight.

", "url": "https://doi.org/10.1609/aaai.v31i1.10610" }, { "name": "Digital Juries: A Civics-Oriented Approach to Platform Governance", "rationale": "

Introduces digital juries as a participatory way to adjudicate content moderation cases. It is one of the clearest HCI proposals for replacing platform-only moderation with peer judgment.

", "url": "https://doi.org/10.1145/3313831.3376293" }, { "name": "Can Online Juries Make Consistent, Repeatable Decisions?", "rationale": "

Tests whether online juries can produce stable judgments on similar cases. It matters because consistency is a central objection to decentralized adjudication.

", "url": "https://doi.org/10.1145/3411764.3445433" }, { "name": "PolicyKit: Building Governance in Online Communities", "rationale": "

Presents infrastructure for communities to encode and execute governance procedures, including random juries, deliberation, and policy change. It turns collective oversight from a social ideal into programmable community process.

", "url": "https://arxiv.org/abs/2008.04236" }, { "name": "Kleros Long Paper v2.0.2", "rationale": "

Specifies Kleros as a blockchain-based decentralized decision protocol using crowdsourced jurors and game-theoretic incentives. It is the canonical technical document for decentralized justice in Web3.

", "url": "https://kleros.io/yellowpaper.pdf" }, { "name": "Birdwatch: Crowd Wisdom and Bridging Algorithms can Inform Understanding and Reduce the Spread of Misinformation", "rationale": "

Describes the core bridging-based approach behind Twitter/X Community Notes. It is a major deployed example of crowdsourced judgment that requires cross-perspective agreement rather than simple majority rule.

", "url": "https://arxiv.org/abs/2210.15723" }, { "name": "Decentralized Justice: A Comparative Analysis of Blockchain Online Dispute Resolution Projects", "rationale": "

Surveys blockchain dispute-resolution systems and frames decentralized justice as an industry and research area. It contextualizes Kleros-like juror systems as alternatives to conventional online dispute resolution.

", "url": "https://doi.org/10.3389/fbloc.2021.564551" } ] }, "people": { "major": [ { "name": "Elinor Ostrom", "rationale": "

Ostrom’s work on commons governance, monitoring, and graduated sanctions is foundational for collective enforcement without centralized command. Her design principles strongly inform online self-governance research.

", "url": "https://www.nobelprize.org/prizes/economic-sciences/2009/ostrom/biographical/" }, { "name": "Amy X. Zhang", "rationale": "

Zhang is a central researcher on democratic online governance, digital juries, PolicyKit, and community self-governance tools. Her work directly targets distributed alternatives to top-down platform moderation.

", "url": "https://homes.cs.washington.edu/~axz/" }, { "name": "Michael S. Bernstein", "rationale": "

Bernstein coauthored PolicyKit and major online jury research. His social computing work is central to making crowd governance and deliberative moderation operational.

", "url": "https://hci.stanford.edu/msb/" }, { "name": "Federico Ast", "rationale": "

Ast is a cofounder of Kleros and a leading advocate for decentralized justice. His work links crowdsourced juries, dispute resolution, blockchain incentives, and institutional design.

", "url": "https://federicoast.com/" }, { "name": "Clément Lesaege", "rationale": "

Lesaege is a Kleros cofounder and coauthor of its long paper. He is important for the mechanism and cryptoeconomic design of decentralized juror systems.

", "url": "https://github.com/clesaege" }, { "name": "Nathan Schneider", "rationale": "

Schneider is a leading scholar of democratic online governance, Metagov, Modular Politics, and governable spaces. His work frames online communities as polities capable of self-rule.

", "url": "https://nathanschneider.info/" } ] }, "tags": [ "oversight", "law & governance" ] }, "Collusion & Cartel Detection": { "companies": { "major": [ { "name": "RealPage", "rationale": "

Central real-world case for alleged algorithmic rent coordination through revenue-management software. Its systems are a key reference point for third-party pricing algorithm collusion.

", "url": "https://www.realpage.com/" }, { "name": "Agri Stats", "rationale": "

Important example of a data intermediary accused of facilitating coordinated pricing and output behavior through competitively sensitive benchmarking. It is a core hub-and-spoke information-sharing case.

", "url": "https://www.agristats.com/" }, { "name": "Kalibrate", "rationale": "

Provider of fuel-pricing software relevant to algorithmic gasoline pricing and alleged coordination concerns. It maps closely onto high-frequency retail-price monitoring use cases.

", "url": "https://kalibrate.com/kalibrate-fuel-pricing-software/" }, { "name": "Cendyn / Rainmaker", "rationale": "

Hotel revenue-management provider linked to litigation and scrutiny over algorithmic room-rate coordination. It is a notable travel and hospitality analogue to rent-pricing software cases.

", "url": "https://www.cendyn.com/" } ] }, "nonprofits": { "major": [ { "name": "OECD Competition Division", "rationale": "

Produces major policy work on algorithms, collusion, bid-rigging, and cartel detection tools. It is one of the central international standard-setters for competition enforcement practice.

", "url": "https://www.oecd.org/competition/" }, { "name": "International Competition Network", "rationale": "

Global network of competition authorities with cartel working groups and enforcement-practice guidance. It is important for spreading detection methods across agencies.

", "url": "https://www.internationalcompetitionnetwork.org/" }, { "name": "American Antitrust Institute", "rationale": "

US antitrust nonprofit active on cartel enforcement, algorithmic pricing, and competition policy. It helps translate research and litigation developments into policy recommendations.

", "url": "https://www.antitrustinstitute.org/" }, { "name": "Flashbots Collective", "rationale": "

Research and governance community focused on MEV, block building, and transaction-ordering markets. It is central for decentralized routing and rent-extraction concerns.

", "url": "https://collective.flashbots.net/" } ] }, "papers": { "major": [ { "name": "Artificial Intelligence, Algorithmic Pricing, and Collusion", "rationale": "

Foundational experimental economics paper showing that Q-learning pricing algorithms can learn supracompetitive prices without communication. It anchors modern work on autonomous agent collusion.

", "url": "https://doi.org/10.1257/aer.20190623" }, { "name": "Detecting Cartels", "rationale": "

Canonical synthesis of behavioral and structural cartel screens. It lays out the market markers that make collusion detectable from prices, quantities, bids, and conduct.

", "url": "https://www.econ2.jhu.edu/REPEC/papers/wp526harrington.pdf" }, { "name": "A Variance Screen for Collusion", "rationale": "

Introduces the influential low-variance price screen for detecting collusive periods. It is a central empirical template for monitoring markets for price-fixing signals.

", "url": "https://doi.org/10.1016/j.ijindorg.2005.10.003" }, { "name": "Detection of Bid Rigging in Procurement Auctions", "rationale": "

Classic econometric study distinguishing cartel bids from competitive bids in procurement auctions. It remains a core reference for detecting coordinated bidding behavior.

", "url": "https://doi.org/10.1086/261885" }, { "name": "Deciding Between Competition and Collusion", "rationale": "

Develops structural tests to decide whether procurement bids are more consistent with competition or collusion. It is foundational for model-based bid-rigging detection.

", "url": "https://doi.org/10.1162/003465303772815871" }, { "name": "Algorithmic Pricing and Competition: Empirical Evidence from the German Retail Gasoline Market", "rationale": "

Provides leading real-world empirical evidence that adoption of algorithmic pricing can raise margins in oligopolistic gasoline markets. It connects cartel detection to high-frequency algorithmic pricing data.

", "url": "https://doi.org/10.1086/726906" }, { "name": "Frontiers: Algorithmic Collusion: Supra-competitive Prices via Independent Algorithms", "rationale": "

Shows how independent machine-learning pricing systems can produce supracompetitive outcomes through correlated experiments. It broadens collusion concerns beyond explicit communication or shared software.

", "url": "https://doi.org/10.1287/mksc.2020.1276" }, { "name": "Algorithmic Collusion by Large Language Models", "rationale": "

Tests LLM-based pricing agents and finds autonomous supracompetitive pricing in oligopoly and auction settings. It is directly relevant to detecting collusion among AI agents.

", "url": "https://arxiv.org/abs/2404.00806" } ] }, "people": { "major": [ { "name": "Joseph E. Harrington Jr.", "rationale": "

Leading economist on cartels, collusion dynamics, cartel detection, and algorithmic pricing law. His work defines many of the behavioral markers used to detect collusion.

", "url": "https://bepp.wharton.upenn.edu/profile/harrij/" }, { "name": "Rosa M. Abrantes-Metz", "rationale": "

Key developer and advocate of empirical screens for cartel detection, including the variance screen. Her work links econometrics, antitrust compliance, and proactive monitoring.

", "url": "https://www.thinkbrg.com/people/rosa-abrantes-metz/" }, { "name": "Emilio Calvano", "rationale": "

Coauthor of the foundational AER paper on AI pricing algorithms learning to collude. His research is central to the economics of algorithmic market coordination.

", "url": "https://economia.uniroma2.it/faculty/674/calvano-emilio" }, { "name": "Giacomo Calzolari", "rationale": "

Major researcher on AI, markets, and competition policy. He coauthored core work on autonomous algorithmic collusion and policy responses.

", "url": "https://www.eui.eu/people?id=giacomo-calzolari" }, { "name": "Ariel Ezrachi", "rationale": "

Field-shaping legal scholar on algorithmic collusion and digital competition. His work with Maurice Stucke helped frame algorithmic tacit collusion for regulators.

", "url": "https://www.law.ox.ac.uk/people/ariel-ezrachi" }, { "name": "Maurice E. Stucke", "rationale": "

Influential antitrust scholar on algorithm-driven markets, collusion, and competition policy. His work with Ariel Ezrachi is central to legal debates on algorithmic pricing.

", "url": "https://law.utk.edu/people/maurice-stucke/" }, { "name": "Robert H. Porter", "rationale": "

Coauthor of the classic bid-rigging detection paper in procurement auctions. His work is foundational for econometric identification of collusive bidding.

", "url": "https://economics.northwestern.edu/people/directory/robert-porter.html" }, { "name": "Patrick Bajari", "rationale": "

Coauthor of major structural methods for distinguishing competition from collusion in procurement auctions. His auction and antitrust work is central to model-based cartel detection.

", "url": "https://patbajari.ai/" } ] }, "tags": [ "economics", "safety", "oversight", "multi-agent systems" ] }, "Compute & Inference Markets": { "companies": { "major": [ { "name": "Overclock Labs / Akash Network", "rationale": "

Akash is a leading decentralized cloud marketplace where providers bid to supply CPU and GPU resources. It is one of the most directly relevant live networks for permissionless AI compute acquisition.

", "url": "https://akash.network/" }, { "name": "Vast.ai", "rationale": "

Vast.ai is a large two-sided GPU rental marketplace connecting independent hosts with AI users. It is critical to the practical market layer for distributed, non-hyperscaler GPU access.

", "url": "https://vast.ai/" }, { "name": "Gensyn", "rationale": "

Gensyn builds a decentralized protocol for verifiable ML training, inference, evaluation, and payments. It is one of the highest-signal companies in trust-minimized AI compute markets.

", "url": "https://www.gensyn.ai/" }, { "name": "OTOY / Render Network", "rationale": "

OTOY originated Render Network, a decentralized GPU network with real creative workloads and expanding AI compute relevance. It is a major example of idle GPU supply becoming a marketplace.

", "url": "https://home.otoy.com/" }, { "name": "io.net", "rationale": "

io.net aggregates decentralized GPU resources into clusters for AI and ML workloads. It is a major DePIN compute marketplace focused on GPU liquidity.

", "url": "https://io.net/" }, { "name": "Aethir", "rationale": "

Aethir operates a distributed GPU cloud aimed at AI, gaming, and virtualized compute. It is important for enterprise-grade decentralized GPU supply.

", "url": "https://aethir.com/" }, { "name": "Golem Network", "rationale": "

Golem is one of the earliest decentralized compute marketplaces, with requestors and providers exchanging compute through token payments. Its historical role and continuing AI-facing work make it important.

", "url": "https://www.golem.network/" }, { "name": "Ritual", "rationale": "

Ritual builds infrastructure for decentralized AI execution, including inference pathways for onchain applications. It is more specialized than GPU marketplaces but central to verified AI compute flows.

", "url": "https://ritual.net/" } ] }, "nonprofits": { "major": [ { "name": "Render Network Foundation", "rationale": "

Governance organization for Render Network, one of the most established decentralized GPU networks. It is central to protocol governance, grants, and ecosystem coordination.

", "url": "https://renderfoundation.com/" }, { "name": "OpenTensor Foundation", "rationale": "

Foundation associated with Bittensor's open-source protocol and ecosystem tooling. It is highly relevant because Bittensor is one of the core decentralized AI market protocols.

", "url": "https://bittensor.com/" }, { "name": "Golem Foundation", "rationale": "

Supports research and development around open, decentralized, user-controlled internet infrastructure. It is important because Golem is a foundational decentralized compute network.

", "url": "https://golem.foundation/" } ] }, "papers": { "major": [ { "name": "BitTensor: A Peer-to-Peer Intelligence Market", "rationale": "

Introduces a token-incentivized peer-to-peer market where model-serving nodes are evaluated and rewarded. It is foundational for Bittensor-style intelligence and inference markets.

", "url": "https://arxiv.org/abs/2003.03917" }, { "name": "Petals: Collaborative Inference and Fine-tuning of Large Models", "rationale": "

Shows how large language model inference and fine-tuning can run collaboratively across independently operated machines. It is one of the clearest technical precedents for fluid inference across non-hyperscale substrates.

", "url": "https://arxiv.org/abs/2209.01188" }, { "name": "SkyPilot: An Intercloud Broker for Sky Computing", "rationale": "

Defines and implements an intercloud broker that places jobs across multiple clouds by price and availability. It is central to the non-crypto side of avoiding single-provider compute lock-in.

", "url": "https://www.usenix.org/conference/nsdi23/presentation/yang-zongheng" }, { "name": "Gensyn Protocol Litepaper", "rationale": "

Describes a trustless protocol for deep learning computation with solvers, verifiers, and market payments. It is a core design document for verifiable decentralized ML compute markets.

", "url": "https://docs.gensyn.ai/litepaper" }, { "name": "Akash Network: Decentralized Cloud Infrastructure Marketplace", "rationale": "

Specifies Akash's decentralized cloud exchange and reverse-auction marketplace for compute. It is a major protocol blueprint for permissionless cloud and GPU procurement.

", "url": "https://akash.network/whitepaper/" }, { "name": "The Golem Project Whitepaper", "rationale": "

Early whitepaper for a global market in unused computing power. It helped establish the decentralized compute marketplace category before the current AI GPU wave.

", "url": "https://www.golem.network/crowdfunding/Golemwhitepaper.pdf" }, { "name": "Render Network Whitepaper", "rationale": "

Explains Render's decentralized GPU supply network, originally for rendering and later relevant to AI workloads. It is important for market-based use of idle GPU capacity.

", "url": "https://renderfoundation.com/whitepaper" }, { "name": "A Scalable Verification Solution for Blockchains", "rationale": "

Introduces TrueBit's verification-game approach for outsourcing computation with economic security. It is a foundational verification primitive for trust-minimized compute markets.

", "url": "https://arxiv.org/abs/1908.04756" } ] }, "people": { "major": [ { "name": "Greg Osuri", "rationale": "

Founder of Overclock Labs and a key builder behind Akash Network. He is central to the decentralized cloud marketplace side of this area.

", "url": "https://akash.network/blog/the-economics-of-akash-network-and-token/" }, { "name": "Jake Cannell", "rationale": "

Co-founder and CEO of Vast.ai, one of the most important independent GPU marketplace platforms. Vast.ai is a major non-blockchain example of smoothing access to distributed GPU supply.

", "url": "https://vast.ai/about" }, { "name": "Jules Urbach", "rationale": "

Founder of OTOY and Render Network. He has been one of the most visible builders of decentralized GPU rendering and compute supply networks.

", "url": "https://home.otoy.com/team/jules-urbach/" }, { "name": "Max Ryabinin", "rationale": "

Lead researcher across Learning@home, Hivemind, and Petals. His work is foundational for collaborative training and inference over independently operated machines.

", "url": "https://arxiv.org/abs/2209.01188" }, { "name": "Ion Stoica", "rationale": "

Co-director-level figure in Berkeley systems research and coauthor on SkyPilot. His work on sky computing directly targets cloud lock-in and portable compute acquisition.

", "url": "https://people.eecs.berkeley.edu/~istoica/" }, { "name": "Jason Teutsch", "rationale": "

Founder of TrueBit and a leading figure in verifiable outsourced computation. His work underpins trust mechanisms needed for permissionless compute markets.

", "url": "https://people.cs.uchicago.edu/~teutsch/" }, { "name": "Jacob Steeves", "rationale": "

Co-founder of Bittensor and author on its peer-to-peer intelligence market paper. He is central to token-incentivized AI service markets.

", "url": "https://bittensor.com/academia" }, { "name": "Harry Grieve", "rationale": "

Co-founder of Gensyn, a major decentralized ML compute protocol. He is important for the verifiable training and compute-market branch of the field.

", "url": "https://www.gensyn.ai/" } ] }, "tags": [ "compute", "economics", "infrastructure" ] }, "Cooperative Agents": { "companies": { "major": [ { "name": "Google DeepMind", "rationale": "

Produced foundational Cooperative AI work including sequential social dilemmas and Melting Pot. It remains central to research on how cooperation emerges among learning agents.

", "url": "https://deepmind.google/" }, { "name": "Google Cloud", "rationale": "

Originated the Agent2Agent protocol for interoperability across agents from different vendors and frameworks. This directly supports cross-owner task delegation and coordination.

", "url": "https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/" }, { "name": "Microsoft", "rationale": "

Created AutoGen, one of the most influential LLM multi-agent frameworks, and supports open agent interoperability work. Its tools mainstream multi-agent conversation and enterprise collaboration patterns.

", "url": "https://www.microsoft.com/en-us/research/project/autogen/" }, { "name": "LangChain", "rationale": "

LangGraph is a leading framework for stateful, multi-agent LLM workflows. It is widely used to engineer supervision, handoffs, memory, and task decomposition among agents.

", "url": "https://www.langchain.com/langgraph" }, { "name": "CrewAI", "rationale": "

Builds a popular role-based framework and platform for multi-agent automations. It operationalizes cooperative design through agents, tasks, tools, and workflows.

", "url": "https://www.crewai.dev/" }, { "name": "Fetch.ai", "rationale": "

Builds uAgents and Agentverse for discoverable, communicating, and transacting autonomous agents. It is one of the clearest decentralized attempts at an agent network rather than a monolithic assistant.

", "url": "https://www.fetch.ai/docs" }, { "name": "Cisco / Outshift", "rationale": "

Initiated AGNTCY, open infrastructure for agent discovery, identity, messaging, and observability. Its Internet of Agents framing maps directly to cooperation across agent owners.

", "url": "https://outshift.cisco.com/blog/building-the-internet-of-agents-introducing-the-agntcy" } ] }, "nonprofits": { "major": [ { "name": "Cooperative AI Foundation", "rationale": "

The main dedicated nonprofit for advancing cooperative intelligence in AI. It funds research, runs fellowships and events, and anchors the field as a coherent community.

", "url": "https://www.cooperativeai.com/foundation" }, { "name": "International Foundation for Autonomous Agents and Multiagent Systems", "rationale": "

Stewards AAMAS, the flagship conference for autonomous agents and multiagent systems. It is the core scholarly institution behind much of the broader multiagent field.

", "url": "https://ifaamas.org/" }, { "name": "Linux Foundation Agentic AI Foundation", "rationale": "

Provides neutral open-source governance for agentic AI projects and standards. It matters for turning agent cooperation from vendor integrations into shared infrastructure.

", "url": "https://www.linuxfoundation.org/press/linux-foundation-announces-the-formation-of-the-agentic-ai-foundation" }, { "name": "Agent2Agent Protocol Project", "rationale": "

Maintains an open standard for communication and collaboration between independent AI agent systems. It is one of the most direct protocol efforts for cooperative agents across vendors.

", "url": "https://a2a-protocol.org/" }, { "name": "AGNTCY Project", "rationale": "

Open infrastructure project for agent discovery, identity, messaging, and observability. It is central to the idea of an interoperable Internet of Agents.

", "url": "https://agntcy.org/" } ] }, "papers": { "major": [ { "name": "Open Problems in Cooperative AI", "rationale": "

Defines Cooperative AI as a research agenda for building agents that can cooperate with humans, machines, and institutions. It is the clearest field-framing paper for the area.

", "url": "https://arxiv.org/abs/2012.08630" }, { "name": "Cooperative AI: machines must learn to find common ground", "rationale": "

Popularized the need for AI systems that can find common ground, coordinate, and cooperate rather than optimize alone. It helped move cooperation from a niche MARL topic into a broader AI agenda.

", "url": "https://doi.org/10.1038/d41586-021-01170-0" }, { "name": "Multi-agent Reinforcement Learning in Sequential Social Dilemmas", "rationale": "

Introduces temporally extended social dilemmas for studying when cooperation or conflict emerges among learning agents. It is foundational for the emergence side of cooperative agent research.

", "url": "https://arxiv.org/abs/1702.03037" }, { "name": "Ad Hoc Autonomous Agent Teams: Collaboration without Pre-Coordination", "rationale": "

Defines the ad hoc teamwork problem, agents collaborating with unfamiliar teammates without prior coordination. This is directly aligned with independently owned agents cooperating across boundaries.

", "url": "https://doi.org/10.1609/aaai.v24i1.7529" }, { "name": "Learning to Communicate with Deep Multi-Agent Reinforcement Learning", "rationale": "

Introduces RIAL and DIAL for learning communication protocols in cooperative partially observable tasks. It is a key paper on whether cooperation can be learned end-to-end rather than hand-specified.

", "url": "https://papers.nips.cc/paper/6042-learning-to-communicate-with-deep-multi-agent-reinforcement-learning" }, { "name": "QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning", "rationale": "

A central algorithm for centralized training with decentralized execution in cooperative MARL. It became a standard reference point for engineering coordinated behavior from local agents.

", "url": "https://proceedings.mlr.press/v80/rashid18a.html" }, { "name": "Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot", "rationale": "

Introduces a benchmark suite for evaluating social generalization, reciprocity, resource sharing, and task partitioning. It is crucial for measuring whether agents cooperate with unfamiliar others.

", "url": "https://arxiv.org/abs/2107.06857" }, { "name": "AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation", "rationale": "

Establishes a widely used LLM multi-agent framework based on conversable agents, roles, tools, and programmable interaction patterns. It is central to modern engineered cooperation among AI agents.

", "url": "https://arxiv.org/abs/2308.08155" } ] }, "people": { "major": [ { "name": "Allan Dafoe", "rationale": "

Coauthor of the field-framing Cooperative AI agenda and a founder/trustee of the Cooperative AI Foundation. He is one of the central organizers of the area.

", "url": "https://www.allandafoe.com/home" }, { "name": "Joel Z. Leibo", "rationale": "

Led influential work on sequential social dilemmas, Melting Pot, and generalized cooperative intelligence at Google DeepMind. His research directly studies how cooperation emerges and how to evaluate it.

", "url": "https://www.jzleibo.com/" }, { "name": "Jakob Foerster", "rationale": "

Key researcher in cooperative MARL, learned communication, COMA, QMIX-related work, and ad hoc cooperation. He bridges algorithmic MARL and Cooperative AI.

", "url": "https://aigi.ox.ac.uk/people/jakob-foerster/" }, { "name": "Shimon Whiteson", "rationale": "

Major contributor to deep multi-agent reinforcement learning, including learned communication, COMA, QMIX, and SMAC. His work is central to decentralized cooperative control.

", "url": "https://www.cs.ox.ac.uk/people/Shimon.Whiteson/" }, { "name": "Natasha Jaques", "rationale": "

Known for social reinforcement learning and social influence methods that improve multi-agent coordination. Her work focuses on social learning, cooperation, and human-AI interaction.

", "url": "https://natashajaques.ai/" }, { "name": "Kate Larson", "rationale": "

Works on multiagent systems, game theory, mechanism design, negotiation, and cooperation. She is also a key advisor and contributor to the Cooperative AI community.

", "url": "https://cs.uwaterloo.ca/~klarson/" }, { "name": "Peter Stone", "rationale": "

Foundational multiagent systems researcher and a primary figure in ad hoc teamwork. His work is central to collaboration among agents that have not pre-coordinated.

", "url": "https://www.cs.utexas.edu/~pstone/" } ] }, "tags": [ "multi-agent systems", "alignment" ] }, "Correlated Failure in Agent Networks": { "companies": { "major": [] }, "nonprofits": { "major": [ { "name": "Gradient Institute", "rationale": "

Produced one of the most directly relevant reports on governed LLM-based multi-agent risks, including monoculture collapse and cascading reliability failures. It is a central nonprofit actor for applied risk analysis in this area.

", "url": "https://gradientinstitute.org/" }, { "name": "Stanford Center for Research on Foundation Models", "rationale": "

Created the foundation model framing and has led work on model evaluation, transparency, and homogenization. CRFM is central to understanding shared base models as systemic dependencies.

", "url": "https://crfm.stanford.edu/" }, { "name": "Cloud Security Alliance", "rationale": "

Its AI Safety Initiative has published directly relevant work on AI infrastructure monoculture, agentic threat modeling, and concentration risk. It is important for translating correlated failure into security and governance controls.

", "url": "https://cloudsecurityalliance.org/" } ] }, "papers": { "major": [ { "name": "Correlated Errors in Large Language Models", "rationale": "

Empirically measures shared error patterns across more than 350 LLMs and shows that apparent model diversity often does not yield independent failures. It is directly about detecting hidden common failure modes across model providers, architectures, and downstream uses.

", "url": "https://arxiv.org/abs/2506.07962" }, { "name": "Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panels", "rationale": "

Shows that a panel of nine frontier LLM judges can provide only about two independent votes because models make the same mistakes on the same items. It is central for understanding why agent or evaluator ensembles can fail together despite nominal diversity.

", "url": "https://arxiv.org/abs/2605.29800" }, { "name": "Algorithmic Monoculture and Social Welfare", "rationale": "

Foundational theory paper formalizing how many decision makers using the same algorithm can reduce overall welfare. It provides the core monoculture lens behind correlated failure in decentralized AI decision networks.

", "url": "https://doi.org/10.1073/pnas.2018340118" }, { "name": "Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?", "rationale": "

Introduces outcome homogenization and tests whether shared components, such as training data or models, cause the same people or groups to receive repeated negative outcomes. It connects component sharing to systemic exclusion, a key form of correlated failure.

", "url": "https://arxiv.org/abs/2211.13972" }, { "name": "Algorithmic Monocultures in Hiring", "rationale": "

Large empirical study of millions of job applications screened through a single vendor's algorithms. It provides real-world evidence that shared AI decision infrastructure can produce homogeneous and disparate outcomes across many employers.

", "url": "https://doi.org/10.1145/3805689.3812400" }, { "name": "Risk Analysis Techniques for Governed LLM-based Multi-Agent Systems", "rationale": "

Practitioner-focused report on risks in LLM-based multi-agent systems, including monoculture collapse, cascading reliability failures, and conformity bias. It is one of the most direct treatments of correlated failure inside agent networks.

", "url": "https://arxiv.org/abs/2508.05687" }, { "name": "Understanding Agent Scaling in LLM-Based Multi-Agent Systems via Diversity", "rationale": "

Develops an information-theoretic account of why homogeneous multi-agent systems have diminishing returns. Its effective-channel framing is directly useful for measuring whether extra agents add independent evidence or merely repeat correlated signals.

", "url": "https://arxiv.org/abs/2602.03794" }, { "name": "On the Opportunities and Risks of Foundation Models", "rationale": "

Foundational report introducing the foundation model paradigm and warning that homogenization can propagate defects to many downstream systems. It supplies the systemic-risk framing for shared base models as hidden dependencies.

", "url": "https://arxiv.org/abs/2108.07258" } ] }, "people": { "major": [ { "name": "Nikhil Garg", "rationale": "

Central researcher on LLM correlated errors and algorithmic monoculture. His work directly measures when nominally different models fail together.

", "url": "https://tech.cornell.edu/people/nikhil-garg/" }, { "name": "Manish Raghavan", "rationale": "

Coauthor of the foundational algorithmic monoculture theory paper and later work on measuring monoculture. His research links AI decision systems, social welfare, and correlated outcomes.

", "url": "https://mitsloan.mit.edu/faculty/directory/manish-raghavan" }, { "name": "Jon Kleinberg", "rationale": "

Coauthor of the foundational theory of algorithmic monoculture and social welfare. His work provides the formal basis for why shared decision algorithms can harm decentralized systems.

", "url": "https://www.cs.cornell.edu/home/kleinber/" }, { "name": "Rishi Bommasani", "rationale": "

Lead contributor to outcome homogenization and empirical hiring monoculture work. He also helped frame foundation models as shared infrastructure whose defects propagate downstream.

", "url": "https://algorithmichiring.github.io/" }, { "name": "Percy Liang", "rationale": "

Key foundation model researcher and CRFM leader whose work on foundation models, HELM, and algorithmic monoculture informs how shared base models create systemic dependencies.

", "url": "https://cs.stanford.edu/~pliang/" }, { "name": "Kathleen A. Creel", "rationale": "

Researcher on algorithmic monoculture, outcome homogenization, and systemic exclusion. Her work connects technical component sharing to moral and institutional consequences.

", "url": "https://www.khoury.northeastern.edu/people/kathleen-katie-creel/" }, { "name": "Guneet Kohli", "rationale": "

Author of a highly focused study showing that multi-model LLM judge panels can have far fewer effective independent votes than their size suggests. This is directly central to correlated failure in evaluator and agent panels.

", "url": "https://machinelearning.apple.com/research/correlated-llm-evaluation-panels" } ] }, "tags": [ "safety", "multi-agent systems", "distributed systems", "oversight" ] }, "Cross-Agent Credit Assignment": { "companies": { "major": [ { "name": "Kite AI", "rationale": "

Builds agentic AI payment and coordination infrastructure with Proof of Attributed Intelligence for fair attribution and rewards across data, models, and agents. It is one of the most direct commercial efforts in AI contributor attribution.

", "url": "https://gokite.ai/" }, { "name": "OpenLedger", "rationale": "

Builds an AI blockchain and attribution engine intended to trace which data shaped model outputs and reward contributors. It is highly relevant to payment allocation in open AI value chains.

", "url": "https://www.openledger.xyz/" }, { "name": "Opentensor / Bittensor", "rationale": "

Operates Bittensor, where validators score miners and Yuma Consensus converts those scores into emissions. It is a live, field-shaping example of decentralized contribution scoring and reward distribution.

", "url": "https://opentensor.ai/" }, { "name": "Allora Network", "rationale": "

Coordinates workers, forecasters, and reputers to synthesize inferences, weight contributors, and allocate rewards based on unique contribution. Its mechanisms are directly relevant to down-weighting and paying agents in collective inference.

", "url": "https://www.allora.network/" } ] }, "nonprofits": { "major": [] }, "papers": { "major": [ { "name": "A Value for n-Person Games", "rationale": "

Introduced the Shapley value, the canonical cooperative-game-theory rule for allocating total payoff by marginal contribution. It remains the main formal baseline for fair payment and blame across cooperating agents.

", "url": "https://doi.org/10.1515/9781400881970-018" }, { "name": "Optimal Payoff Functions for Members of Collectives", "rationale": "

Developed private payoff functions such as Wonderful Life Utility and Aristocrat Utility for collectives optimizing a global utility. It is a core precursor to difference rewards and explicit cross-agent payoff design.

", "url": "https://doi.org/10.1142/S0219525901000188" }, { "name": "Counterfactual Multi-Agent Policy Gradients", "rationale": "

Introduced COMA, which uses a centralized critic and counterfactual baselines to estimate one agent's contribution while holding others fixed. It made counterfactual credit assignment a standard deep MARL technique.

", "url": "https://arxiv.org/abs/1705.08926" }, { "name": "Value-Decomposition Networks For Cooperative Multi-Agent Learning", "rationale": "

Proposed decomposing a shared team value into agent-wise value functions under a single joint reward. It is a foundational value-factorization approach for translating team outcomes into per-agent learning signals.

", "url": "https://arxiv.org/abs/1706.05296" }, { "name": "Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning", "rationale": "

Introduced QMIX, a monotonic mixing architecture for combining per-agent utilities into a joint action-value function. It became a widely used baseline for credit-relevant value decomposition in cooperative MARL.

", "url": "https://jmlr.org/papers/v21/20-081.html" }, { "name": "Data Shapley: Equitable Valuation of Data for Machine Learning", "rationale": "

Adapted Shapley values to quantify the value of individual training examples for model performance. It is central to economic attribution for data contributors in AI systems.

", "url": "https://proceedings.mlr.press/v97/ghorbani19c.html" }, { "name": "MALT: Improving Reasoning with Multi-Agent LLM Training", "rationale": "

Studies sequential LLM agents with generator, verifier, and refiner roles, using joint outcome rewards to create agent-specific training signals. It is one of the clearest early papers on credit assignment through LLM agent chains.

", "url": "https://arxiv.org/abs/2412.01928" }, { "name": "Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agents", "rationale": "

Uses Shapley-style marginal contributions and negotiation to price work and redistribute rewards among self-interested LLM agents. It directly targets fair compensation and coordination in open-ended multi-agent AI workflows.

", "url": "https://arxiv.org/abs/2506.07388" } ] }, "people": { "major": [ { "name": "Lloyd S. Shapley", "rationale": "

Created the Shapley value, the foundational fair-allocation concept used across modern contribution attribution. His work underpins many data, model, and agent credit mechanisms.

", "url": "https://www.nobelprize.org/prizes/economic-sciences/2012/shapley/biographical/" }, { "name": "David H. Wolpert", "rationale": "

Co-developed the collective intelligence and difference-reward framing for aligning private payoffs with global utility. His work is foundational for assigning credit or blame in large agent collectives.

", "url": "https://www.santafe.edu/people/profile/david-wolpert" }, { "name": "Kagan Tumer", "rationale": "

A leading researcher on multiagent credit assignment, difference rewards, and distributed reinforcement learning. His work repeatedly studies how to quantify each agent's impact on system performance.

", "url": "https://engineering.oregonstate.edu/people/kagan-tumer" }, { "name": "Shimon Whiteson", "rationale": "

Co-authored COMA and QMIX, two central deep MARL approaches for counterfactual and value-decomposition credit assignment. His lab has shaped the modern cooperative MARL toolkit.

", "url": "https://www.cs.ox.ac.uk/people/Shimon.Whiteson/" }, { "name": "Jakob N. Foerster", "rationale": "

Co-authored COMA and QMIX and helped bring deep multi-agent reinforcement learning to prominence. His work is central to learning and attribution in cooperative agent teams.

", "url": "https://www.jakobfoerster.com/home" } ] }, "tags": [ "economics", "multi-agent systems" ] }, "Decentralized Benchmarking": { "companies": { "major": [ { "name": "ORO", "rationale": "

Runs a live decentralized evaluation platform on Bittensor for shopping agents, with validators evaluating submitted agents and emissions tied to performance. It is one of the clearest deployed examples of decentralized benchmarking.

", "url": "https://oroagents.com/" } ] }, "nonprofits": { "major": [ { "name": "MLCommons", "rationale": "

Hosts major open benchmark efforts and the Medical AI working group behind MedPerf. It is central to neutral, multi-stakeholder benchmarking and federated AI evaluation.

", "url": "https://mlcommons.org/" }, { "name": "OpenTensor Foundation / Bittensor", "rationale": "

Stewards the Bittensor protocol, whose subnets use validators and Yuma Consensus to score AI work and distribute rewards. It is the most important deployed ecosystem for incentive-based decentralized AI evaluation.

", "url": "https://bittensor.com/" }, { "name": "OpenMined", "rationale": "

Builds open-source privacy-preserving infrastructure for cross-organization AI evaluation and audits. Its secure evaluation work is directly relevant to decentralized benchmarking with confidential data, models, or benchmarks.

", "url": "https://www.openmined.org/" } ] }, "papers": { "major": [ { "name": "Verifiable evaluations of machine learning models using zkSNARKs", "rationale": "

Introduces verifiable benchmark attestations for private ML models using zkSNARKs. It is one of the most direct technical foundations for evaluation without trusting a model provider or central scorer.

", "url": "https://arxiv.org/abs/2402.02675" }, { "name": "Attestable Audits: Verifiable AI Safety Benchmarks Using Trusted Execution Environments", "rationale": "

Proposes running AI safety benchmarks inside TEEs so model providers, auditors, and users can verify benchmark execution while preserving model and dataset confidentiality. It is central to practical decentralized or third-party verifiable audits.

", "url": "https://arxiv.org/abs/2506.23706" }, { "name": "BitTensor: A Peer-to-Peer Intelligence Market", "rationale": "

Defines a peer-to-peer intelligence market where participants rank one another and rewards are assigned through a ledger-based incentive mechanism. It is foundational for Bittensor-style decentralized evaluation and validator-scored AI competitions.

", "url": "https://arxiv.org/abs/2003.03917" }, { "name": "Decentralized Arena: Towards Democratic and Scalable Automatic Evaluation of Language Models", "rationale": "

Builds an automated LLM evaluation framework where models evaluate each other rather than relying on a single authority judge. It directly targets central judge bias in scalable benchmarking.

", "url": "https://arxiv.org/abs/2505.12808" }, { "name": "Federated benchmarking of medical artificial intelligence with MedPerf", "rationale": "

Presents MedPerf, an open platform for federated evaluation of medical AI across data-owning institutions. It is a major deployed example of moving benchmarks to distributed sites instead of centralizing sensitive data and evaluation.

", "url": "https://doi.org/10.1038/s42256-023-00652-2" } ] }, "people": { "major": [ { "name": "Tobin South", "rationale": "

Lead author of the zkSNARK verifiable ML evaluation paper and a researcher on private, verifiable, and auditable AI systems. His work is directly aligned with cryptographically verifiable benchmarks.

", "url": "https://digitaleconomy.stanford.edu/person/tobin-south/" }, { "name": "Alastair R. Beresford", "rationale": "

Senior Cambridge security researcher and coauthor of Attestable Audits. His work brings TEE attestation and systems security into verifiable AI benchmark execution.

", "url": "https://www.cl.cam.ac.uk/~arb33/" }, { "name": "Yuma Rao", "rationale": "

Lead author of the original Bittensor paper. His mechanism design work is foundational for peer-scored, incentive-driven decentralized AI evaluation.

", "url": "https://arxiv.org/abs/2003.03917" }, { "name": "Yanbin Yin", "rationale": "

Lead author of Decentralized Arena. His work directly addresses decentralized, democratic, model-based evaluation as an alternative to authority judges.

", "url": "https://yanbin-yin.github.io/" }, { "name": "Alexandros Karargyris", "rationale": "

Co-lead and corresponding author on MedPerf work through MLCommons. He is a central figure in federated benchmarking of medical AI.

", "url": "https://orcid.org/0000-0002-1930-3410" }, { "name": "Jacob Steeves", "rationale": "

Bittensor cofounder and core architect associated with the protocol's peer-to-peer AI market. He is important for deployed decentralized evaluator and validator systems.

", "url": "https://bittensor.com/whitepaper" } ] }, "tags": [ "oversight", "distributed systems", "infrastructure" ] }, "Decentralized Data Sourcing": { "companies": { "major": [ { "name": "Vana", "rationale": "

Open protocol for portable, user-owned data with DataDAOs and data liquidity pools for AI training. It is one of the clearest implementations of pooled contributor data as an AI input market.

", "url": "https://www.vana.org/" }, { "name": "Grass", "rationale": "

DePIN network where users share unused bandwidth to source and verify public web data for AI. Its sovereign data rollup design makes it central to decentralized web-scale data collection.

", "url": "https://grass.io/" }, { "name": "Sapien", "rationale": "

Decentralized human-data foundry for labeling, validation, RLHF, and specialized AI datasets. It is important because it turns a global contributor base into quality-controlled enterprise AI data.

", "url": "https://www.sapien.io/" }, { "name": "PublicAI", "rationale": "

Web3 AI data infrastructure using a large verified contributor and validator network. Its Data Hub focuses on decentralized collection, annotation, validation, and contributor rewards.

", "url": "https://www.publicai.io/" }, { "name": "OORT", "rationale": "

Decentralized AI data cloud with DataHub for multimodal data collection, preprocessing, and monetization. It is a significant applied platform for crowdsourced AI data pipelines.

", "url": "https://www.oortech.com/" } ] }, "nonprofits": { "major": [ { "name": "Ocean Protocol Foundation", "rationale": "

Stewards Ocean Protocol, one of the earliest and most influential Web3 data economy projects. Its tooling for data NFTs, datatokens, marketplaces, and Compute-to-Data remains central to decentralized data exchange.

", "url": "https://oceanprotocol.com/" }, { "name": "Vana Foundation", "rationale": "

Supports the Vana Network, open-source smart contracts, and user-owned data infrastructure. It is directly tied to decentralized personal data markets for AI.

", "url": "https://www.vanafoundation.org/" }, { "name": "Grass Foundation", "rationale": "

Supports the Grass Network and its token-governed data infrastructure. It is relevant because Grass coordinates distributed bandwidth contributors, dataset purchases, rewards, and provenance.

", "url": "https://www.grassfoundation.io/" } ] }, "papers": { "major": [ { "name": "Ocean Protocol: Tools for the Web3 Data Economy", "rationale": "

Defines data NFTs, datatokens, Compute-to-Data, and open data marketplaces for a Web3 data economy. It is a foundational protocol design for decentralized data exchange and monetization.

", "url": "https://oceanprotocol.com/tech-whitepaper.pdf" }, { "name": "A Marketplace for Data: An Algorithmic Solution", "rationale": "

Formalizes a two-sided market for buying and selling ML training data, including pricing and seller-buyer matching. It is one of the core market-design papers for data as an AI input.

", "url": "https://doi.org/10.1145/3328526.3329589" }, { "name": "Data Shapley: Equitable Valuation of Data for Machine Learning", "rationale": "

Introduces a game-theoretic way to value individual training examples by their contribution to model performance. It matters because contributor payment and data-quality incentives depend on credible contribution attribution.

", "url": "https://proceedings.mlr.press/v97/ghorbani19c.html" }, { "name": "Dada: A Crowdsourced Data Marketplace for Machine Learning", "rationale": "

Proposes a blockchain-based crowdsourced marketplace where participants provide data for ML and receive rewards. It directly anticipates decentralized data sourcing with quality, incentives, and contributor compensation.

", "url": "https://paperswelove.org/papers/dada-a-crowdsourced-data-marketplace-for-machine-l-8eefc9c4/" }, { "name": "Data Measurements for Decentralized Data Markets", "rationale": "

Studies federated measurements that help buyers find relevant and diverse sellers without direct data access. It is highly specific to decentralized data markets and practical seller selection.

", "url": "https://arxiv.org/abs/2406.04257" }, { "name": "ZebraLancer: Private and Anonymous Crowdsourcing System atop Open Blockchain", "rationale": "

Builds a privacy-preserving decentralized crowdsourcing system on Ethereum, including payment and anti-abuse mechanisms. It is foundational for using open blockchains to source human data without exposing workers or submissions.

", "url": "https://doi.org/10.1109/ICDCS.2018.00087" }, { "name": "OmniLytics: A Blockchain-based Secure Data Market for Decentralized Machine Learning", "rationale": "

Designs a secure blockchain data market where distributed data owners contribute private data to ML training and get compensated. It addresses payment atomicity, malicious participants, and privacy in decentralized ML data sourcing.

", "url": "https://arxiv.org/abs/2107.05252" } ] }, "people": { "major": [ { "name": "Anna Kazlauskas", "rationale": "

Co-founder and creator of Vana, one of the most central user-owned data networks for AI training. Her work directly targets pooled personal data, DataDAOs, and contributor ownership of AI value.

", "url": "https://news.mit.edu/2025/vana-lets-users-own-piece-ai-models-trained-on-their-data-0403" }, { "name": "Trent McConaghy", "rationale": "

Co-founder of Ocean Protocol and a long-running builder at the intersection of AI, data markets, and Web3. His work on datatokens and Ocean Market shaped the decentralized data economy category.

", "url": "https://trent.st/" }, { "name": "Andrej Radonjic", "rationale": "

Co-founder and CEO of Wynd Labs, the team behind Grass. Grass is one of the most prominent DePIN approaches to sourcing public web data for AI through distributed bandwidth contributors.

", "url": "https://www.cbinsights.com/company/wynd-network/people" }, { "name": "Rowan Stone", "rationale": "

CEO of Sapien, a decentralized human-data and verification network for AI. He is a key operator in turning contributor networks into enterprise-grade AI training and evaluation data.

", "url": "https://www.sapien.io/company" }, { "name": "James Zou", "rationale": "

Co-author of Data Shapley and a leading researcher in data valuation for ML. His work underpins fair reward allocation and quality measurement for contributed training data.

", "url": "https://profiles.stanford.edu/james-zou" }, { "name": "Munther Dahleh", "rationale": "

Co-author of the ACM paper on algorithmic data marketplaces. His market-design work is foundational for pricing and matching distributed data sellers to ML buyers.

", "url": "https://dahleh.lids.mit.edu/" } ] }, "tags": [ "infrastructure", "economics", "distributed systems" ] }, "Decentralized Governance": { "companies": { "major": [ { "name": "Aragon", "rationale": "

One of the earliest and most widely recognized DAO tooling projects, focused on creating and governing on-chain organizations. It is important for practical decentralized governance modules, permissions, voting, and treasury control.

", "url": "https://www.aragon.org/" }, { "name": "Snapshot Labs", "rationale": "

Builds Snapshot, the dominant off-chain and increasingly on-chain voting infrastructure for DAOs and token communities. Its design choices shape how many decentralized communities turn informal consensus into recorded governance outcomes.

", "url": "https://snapshot.box/" }, { "name": "OpenZeppelin", "rationale": "

Maintains widely used smart contract libraries including Governor modules for on-chain governance. It is central because many protocol governance systems inherit its security and process assumptions.

", "url": "https://www.openzeppelin.com/" }, { "name": "ScopeLift / Cactus (formerly Tally)", "rationale": "

Builds governance interfaces and tooling for proposal creation, delegation, voting, and execution across major DAOs. It is a key operational layer for making on-chain governance usable and auditable.

", "url": "https://www.tally.xyz/" }, { "name": "Parity Technologies", "rationale": "

Core builder of Polkadot and Substrate, including governance machinery for runtime upgrades and OpenGov-style protocol evolution. It is central to one of the most developed on-chain protocol governance ecosystems.

", "url": "https://www.parity.io/" } ] }, "nonprofits": { "major": [ { "name": "Internet Engineering Task Force", "rationale": "

The canonical open Internet standards community, governed through working groups, open mailing lists, rough consensus, and running code. It is foundational for decentralized standards governance outside blockchain.

", "url": "https://www.ietf.org/" }, { "name": "Ethereum Foundation", "rationale": "

Nonprofit steward and funder of Ethereum research, client diversity, protocol development, and public goods. It is influential while operating within a broader community-governed EIP and fork-choice process.

", "url": "https://ethereum.foundation/" }, { "name": "DAOstar", "rationale": "

Standards body for DAOs, including work around EIP-4824 and interoperability between DAO tools. It is directly focused on making decentralized organizations legible across protocols and platforms.

", "url": "https://daostar.org/" }, { "name": "Metagov", "rationale": "

Research and practice community for digital self-governance, DAOs, and online institutions. It is central for cross-community governance research, experiments, and standards-adjacent coordination.

", "url": "https://metagov.org/" }, { "name": "Decentralized Identity Foundation", "rationale": "

Industry and community foundation for decentralized identity specifications and interoperability. It is important for governance of DID methods, credential ecosystems, and identity standards adoption.

", "url": "https://identity.foundation/" }, { "name": "Web3 Foundation", "rationale": "

Foundation behind Polkadot's decentralized web mission, grants, research, and ecosystem development. It is major because Polkadot is a leading case of protocol governance and runtime upgrade design.

", "url": "https://web3.foundation/" } ] }, "papers": { "major": [ { "name": "RFC 7282: On Consensus and Humming in the IETF", "rationale": "

Explains the IETF's rough consensus model, especially why standards decisions are not simple majority votes. It is foundational for understanding decentralized protocol governance through open participation, chairs, editors, and technical objections.

", "url": "https://datatracker.ietf.org/doc/html/rfc7282" }, { "name": "RFC 9518: Centralization, Decentralization, and Internet Standards", "rationale": "

Defines centralization risks in Internet functions and asks what standards bodies can and cannot do to reduce them. It is directly aligned with the question of whether technical standards can prevent governance capture.

", "url": "https://www.rfc-editor.org/rfc/rfc9518.html" }, { "name": "BIP 2: BIP Process, Revised", "rationale": "

Specifies the Bitcoin Improvement Proposal workflow, including proposal status, editors, and community review. It is a canonical example of protocol change management without a single formal owner.

", "url": "https://github.com/bitcoin/bips/blob/master/bip-0002.mediawiki" }, { "name": "EIP-1: EIP Purpose and Guidelines", "rationale": "

Defines how Ethereum standards and protocol changes are proposed, specified, reviewed, and finalized. It matters because the EIP process is the procedural backbone for Ethereum's decentralized standards governance.

", "url": "https://eips.ethereum.org/EIPS/eip-1" }, { "name": "Tezos: A Self-Amending Crypto-Ledger", "rationale": "

Introduces Tezos as a protocol with built-in amendment and on-chain voting. It is a landmark design for making protocol evolution itself part of decentralized governance.

", "url": "https://tezos.com/whitepaper.pdf" }, { "name": "Decentralized Identifiers (DIDs) v1.0", "rationale": "

Defines W3C's DID core standard for identifiers that do not require a centralized registry. It is a key standards artifact for decentralizing identity control and clarifying controller, resolver, and method responsibilities.

", "url": "https://www.w3.org/TR/did/" }, { "name": "The Invisible Politics of Bitcoin: Governance Crisis of a Decentralized Infrastructure", "rationale": "

Analyzes Bitcoin's block size conflict and distinguishes governance by infrastructure from governance of infrastructure. It is one of the core academic accounts of how decentralization can still concentrate power in maintainers and social processes.

", "url": "https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2852691" }, { "name": "Governance of Blockchain Systems: Governance of and by Distributed Infrastructure", "rationale": "

Extends the governance of and governance by distinction across blockchain systems. It is important for mapping which responsibilities sit in code, developers, validators, users, foundations, and surrounding institutions.

", "url": "https://coala.global/wp-content/uploads/2019/02/BRI-COALA-Governance-of-Blockchains.pdf" } ] }, "people": { "major": [ { "name": "Primavera De Filippi", "rationale": "

Foundational scholar of blockchain governance, code as law, and governance of versus by infrastructure. Her work with COALA and W3C blockchain communities makes her central to the area.

", "url": "https://pdefilippi.com/about/" }, { "name": "Vitalik Buterin", "rationale": "

Ethereum co-founder and leading public thinker on protocol governance, soft power, credible neutrality, and limits of token voting. His writing and influence shape how Ethereum and many DAOs reason about decentralization.

", "url": "https://vitalik.ca/" }, { "name": "Gavin Wood", "rationale": "

Founder of Polkadot, Kusama, Parity, and Web3 Foundation, with sustained focus on decentralized protocol evolution. Polkadot's governance architecture makes him central to protocol-level decentralized governance.

", "url": "https://gavwood.com/" }, { "name": "Arthur Breitman", "rationale": "

Early architect of Tezos, one of the most prominent self-amending blockchain protocols. His work is central to on-chain amendment as a governance primitive.

", "url": "https://ex.rs/about" }, { "name": "Joshua Tan", "rationale": "

Co-founder and leader in Metagov and DAOstar work on DAO standards, constitutions, and governance research. He is important for translating decentralized governance theory into shared interfaces and community practice.

", "url": "https://metagov.org/people" }, { "name": "Nathan Schneider", "rationale": "

Researcher and organizer focused on democratic governance of online communities, DAOs, and exit-to-community models. His work bridges cooperative governance, protocol communities, and practical metagovernance.

", "url": "https://nathanschneider.info/" }, { "name": "Manu Sporny", "rationale": "

Longtime W3C contributor and editor in decentralized identifiers and verifiable credentials. He is central to standards work that assigns control over identity away from centralized registries.

", "url": "https://www.w3.org/People/Manu/" } ] }, "tags": [ "law & governance", "distributed systems" ] }, "Decentralized Post-Training": { "companies": { "major": [ { "name": "Flower Labs", "rationale": "

Builds Flower, one of the main open-source federated AI frameworks, and has direct LLM fine-tuning examples and FlowerTune benchmarks. It is central infrastructure for federated post-training.

", "url": "https://flower.ai/" }, { "name": "Prime Intellect", "rationale": "

Runs decentralized training and post-training projects including INTELLECT-2 and prime-rl. It is one of the clearest companies demonstrating permissionless distributed RL post-training at model scale.

", "url": "https://www.primeintellect.ai/" } ] }, "nonprofits": { "major": [ { "name": "Learning@home", "rationale": "

Open-source research collective behind Hivemind and closely tied to Petals-style peer-to-peer large-model work. It is foundational for decentralized inference and fine-tuning over unreliable public hardware.

", "url": "https://github.com/learning-at-home" }, { "name": "BigScience Workshop", "rationale": "

Coordinated the open collaborative BLOOM effort and the ecosystem around Petals. It matters as an early demonstration of community-scale model development and downstream shared adaptation.

", "url": "https://bigscience.huggingface.co/" } ] }, "papers": { "major": [ { "name": "Petals: Collaborative Inference and Fine-tuning of Large Models", "rationale": "

Introduced a peer-to-peer system for running and fine-tuning very large language models across consumer GPUs. It is the clearest foundational paper for decentralized adapter and prompt-tuning style access to models too large for one participant.

", "url": "https://arxiv.org/abs/2209.01188" }, { "name": "Towards Building the Federated GPT: Federated Instruction Tuning", "rationale": "

One of the earliest explicit formulations of federated instruction tuning for LLMs. It frames post-training on distributed private instruction data as a practical alternative to centralizing data.

", "url": "https://arxiv.org/abs/2305.05644" }, { "name": "Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resources", "rationale": "

Introduces FlexLoRA for federated LLM fine-tuning when clients differ in tasks, data, and compute. It is central because heterogeneity is one of the main blockers for real decentralized post-training.

", "url": "https://proceedings.neurips.cc/paper_files/paper/2024/hash/1a134b50202088aa8c595cc99b310e5a-Abstract-Conference.html" }, { "name": "FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations", "rationale": "

Develops aggregation for heterogeneous LoRA adapters in federated fine-tuning. It directly targets the adapter-merging problem that arises when many participants tune a shared base model.

", "url": "https://openreview.net/forum?id=TcCorXxNJQ" }, { "name": "Can Textual Gradient Work in Federated Learning?", "rationale": "

Introduces FedTextGrad, a federated version of text-gradient prompt optimization. This is one of the most direct links between TextGrad-style methods and decentralized post-training.

", "url": "https://arxiv.org/abs/2502.19980" }, { "name": "TextGrad: Automatic \"Differentiation\" via Text", "rationale": "

Defines text gradients as natural-language feedback propagated through AI system components. It matters because prompts, code, and other post-training artifacts can be improved without model-weight ownership.

", "url": "https://arxiv.org/abs/2406.07496" }, { "name": "GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning", "rationale": "

Shows that reflective prompt evolution can be highly sample-efficient for improving compound AI systems. It is important to decentralized post-training because prompt updates are cheap, mergeable artifacts that communities can search over.

", "url": "https://arxiv.org/abs/2507.19457" }, { "name": "INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learning", "rationale": "

Documents a globally distributed reinforcement-learning post-training run for a 32B reasoning model. It demonstrates that decentralized post-training can move beyond simulations into large-scale public model improvement.

", "url": "https://arxiv.org/abs/2505.07291" } ] }, "people": { "major": [ { "name": "Max Ryabinin", "rationale": "

Lead figure behind Hivemind and Petals, which made peer-to-peer large-model inference and fine-tuning practical on unreliable consumer hardware. His work is foundational for decentralized model adaptation.

", "url": "https://mryab.github.io/" }, { "name": "Nicholas D. Lane", "rationale": "

Founder and CSO of Flower Labs and a leading researcher in federated learning systems. His work anchors the federated AI infrastructure side of decentralized post-training.

", "url": "https://niclane.org/" }, { "name": "Mert Yuksekgonul", "rationale": "

Lead author of TextGrad, one of the central frameworks for optimizing prompts, code, and other text artifacts through natural-language feedback. This is a core mechanism for post-training without model ownership.

", "url": "https://mertyuksekgonul.github.io/" }, { "name": "James Zou", "rationale": "

Senior author on TextGrad and a prominent researcher in reliable, human-compatible AI systems. His group has shaped the text-gradient branch of prompt and system optimization.

", "url": "https://profiles.stanford.edu/james-zou" }, { "name": "Yoonho Lee", "rationale": "

Author of Feedback Descent and the SAIL text-gradient-at-scale work. His research directly explores how textual feedback can scale beyond short prompt-refinement loops.

", "url": "https://yoonholee.com/" }, { "name": "Chelsea Finn", "rationale": "

Senior author on Feedback Descent and a major figure in learning algorithms. Her recent work is relevant to turning rich feedback into scalable post-training improvement signals.

", "url": "https://ai.stanford.edu/~cbfinn/" }, { "name": "Omar Khattab", "rationale": "

Creator of DSPy and coauthor on major prompt-optimization work such as MIPRO and GEPA. His systems make prompt and LM-program optimization a programmable target for distributed search.

", "url": "https://omarkhattab.com/" }, { "name": "Saeed Vahidian", "rationale": "

Coauthor of FederatedGPT and related federated LLM work. He is a key contributor to the instruction-tuning branch of decentralized post-training.

", "url": "https://saeedvahidian.github.io/" } ] }, "tags": [ "compute", "infrastructure", "distributed systems" ] }, "Decentralized Serendipity": { "companies": { "major": [ { "name": "Google Research", "rationale": "

Originated influential federated learning, secure aggregation, RAPPOR, and federated analytics work. It has also deployed privacy-preserving learning and analytics in large-scale products.

", "url": "https://research.google/" }, { "name": "Duality Technologies", "rationale": "

Builds enterprise secure data collaboration using PETs such as homomorphic encryption, federated learning, MPC, and differential privacy. Its platform is directly aimed at cross-party analytics without raw-data sharing.

", "url": "https://dualitytech.com/" }, { "name": "Apheris", "rationale": "

Provides federated infrastructure for life-science and drug-discovery data networks. Its role in multi-pharma AI collaboration makes it especially relevant to reducing data concentration advantages.

", "url": "https://www.apheris.com/" }, { "name": "Owkin", "rationale": "

Applies federated learning and collaborative AI to healthcare and biomedical research. It is one of the best-known companies using federated methods to unlock siloed clinical data.

", "url": "https://www.owkin.com/" }, { "name": "Tune Insight", "rationale": "

Commercializes federated analytics and multiparty homomorphic encryption, especially in health and regulated data settings. It has direct lineage from academic work on encrypted precision-medicine analytics.

", "url": "https://tuneinsight.com/en/" }, { "name": "InfoSum", "rationale": "

Runs a decentralized data clean room platform for matching and aggregate insights without moving or co-locating raw data. It is a major commercial example of privacy-preserving data collaboration.

", "url": "https://www.infosum.com/" } ] }, "nonprofits": { "major": [ { "name": "OpenMined", "rationale": "

Open-source community behind PySyft and remote data science tooling. It is one of the most visible nonprofit efforts to make privacy-preserving analysis of data-you-cannot-see practical.

", "url": "https://www.openmined.org/" }, { "name": "OpenDP", "rationale": "

Community project for vetted, open-source differential privacy tools. It matters because distributed discovery systems still need safe mechanisms for releasing aggregate results.

", "url": "https://opendp.org/" }, { "name": "IETF Privacy Preserving Measurement Working Group", "rationale": "

Standards working group for Internet-scale privacy-preserving measurement protocols. Its work operationalizes systems such as Prio and DAP for aggregate discovery without raw client disclosure.

", "url": "https://datatracker.ietf.org/wg/ppm/about/" } ] }, "papers": { "major": [ { "name": "Privacy Preserving Data Mining", "rationale": "

Introduced secure computation as a practical route for data mining across private datasets. It is a foundational paper for discovering patterns without pooling raw data.

", "url": "https://doi.org/10.1007/3-540-44598-6_3" }, { "name": "Privacy-Preserving Distributed Mining of Association Rules on Horizontally Partitioned Data", "rationale": "

Shows how multiple sites can mine association rules over horizontally split databases while preserving each site's private records. It is central to privacy-preserving distributed pattern discovery.

", "url": "https://doi.org/10.1109/TKDE.2004.45" }, { "name": "Privacy preserving association rule mining in vertically partitioned data", "rationale": "

Addresses association-rule discovery when different parties hold different attributes for the same population. This is a canonical vertical-data counterpart to horizontal privacy-preserving mining.

", "url": "https://doi.org/10.1145/775047.775142" }, { "name": "Communication-Efficient Learning of Deep Networks from Decentralized Data", "rationale": "

Introduced Federated Averaging and popularized federated learning for training on decentralized data. It shifted large-scale learning away from mandatory data centralization.

", "url": "https://proceedings.mlr.press/v54/mcmahan17a.html" }, { "name": "Practical Secure Aggregation for Privacy-Preserving Machine Learning", "rationale": "

Provides an efficient, dropout-tolerant secure aggregation protocol for federated learning. It is a core building block for learning aggregate signals without exposing individual updates.

", "url": "https://doi.org/10.1145/3133956.3133982" }, { "name": "Prio: Private, Robust, and Scalable Computation of Aggregate Statistics", "rationale": "

Introduced a multi-server system for robust private aggregate statistics. It directly supports privacy-preserving measurement of population-level patterns and became a basis for later standards work.

", "url": "https://crypto.stanford.edu/prio/" }, { "name": "Federated Heavy Hitters Discovery with Differential Privacy", "rationale": "

Targets discovery of frequent items in user-generated streams without centralizing raw data. It is highly aligned with decentralized serendipity because heavy hitters are a concrete form of surprising population pattern.

", "url": "https://proceedings.mlr.press/v108/zhu20a.html" }, { "name": "Truly privacy-preserving federated analytics for precision medicine with multiparty homomorphic encryption", "rationale": "

Demonstrates federated biomedical analytics using multiparty homomorphic encryption. It is important evidence that sensitive institutional data can yield cross-site scientific insights without central pooling.

", "url": "https://www.nature.com/articles/s41467-021-25972-y" } ] }, "people": { "major": [ { "name": "H. Brendan McMahan", "rationale": "

Co-author of the Federated Averaging paper and a central figure in federated learning and privacy at Google. His work underpins decentralized model learning from private user data.

", "url": "https://research.google/people/author35837/" }, { "name": "Peter Kairouz", "rationale": "

Leads work on federated learning, federated analytics, and differential privacy at Google. He is a key contributor to private heavy hitters and distributed DP methods.

", "url": "https://research.google/people/peterkairouz/" }, { "name": "Yehuda Lindell", "rationale": "

Co-author of the foundational privacy-preserving data mining paper. His broader secure computation work anchors the cryptographic side of decentralized analytics.

", "url": "https://yehudalindell.com/" }, { "name": "Benny Pinkas", "rationale": "

Co-author of foundational work on privacy-preserving data mining and private set intersection. His research is central to computing over private datasets without exposing inputs.

", "url": "https://www.pinkas.net/" }, { "name": "Chris Clifton", "rationale": "

Foundational privacy-preserving distributed data mining researcher. His work on association rules across partitioned data directly matches decentralized pattern discovery.

", "url": "https://www.cs.purdue.edu/homes/clifton/" }, { "name": "Jaideep Vaidya", "rationale": "

Major contributor to privacy-preserving data mining over vertically partitioned data. His work helped define how multiple organizations can jointly mine patterns across complementary attributes.

", "url": "https://www.business.rutgers.edu/faculty/jaideep-vaidya" }, { "name": "Henry Corrigan-Gibbs", "rationale": "

Co-creator of Prio and a leading researcher in private measurement systems. His work is central to scalable aggregate statistics without exposing client data.

", "url": "https://henrycg.com/" }, { "name": "Jean-Pierre Hubaux", "rationale": "

Key contributor to privacy-preserving federated analytics for precision medicine. His work helped translate multiparty homomorphic encryption into biomedical discovery settings.

", "url": "https://people.epfl.ch/jean-pierre.hubaux" } ] }, "tags": [ "distributed systems", "infrastructure" ] }, "Decentralized Superalignment": { "companies": { "major": [ { "name": "OpenAI", "rationale": "

Introduced AI safety via debate, iterated amplification, and weak-to-strong generalization. Its work provides canonical references for using weak or decomposed supervision to control stronger systems.

", "url": "https://openai.com/" }, { "name": "Google DeepMind", "rationale": "

Central to current debate, cooperative AI, and multi-agent safety work, including doubly-efficient debate and distributional AGI safety. It also supports external multi-agent safety research, making it one of the most relevant labs for this area.

", "url": "https://deepmind.google/" } ] }, "nonprofits": { "major": [ { "name": "Cooperative AI Foundation", "rationale": "

The most specific nonprofit for cooperation between advanced AI agents. It supports research on cooperative intelligence, multi-agent risks, and safety-relevant agent interaction.

", "url": "https://www.cooperativeai.com/" }, { "name": "Alignment Research Center", "rationale": "

Paul Christiano's alignment nonprofit, closely tied to scalable oversight, factored cognition, and eliciting latent knowledge. Its agenda is foundational for decomposed and weak-supervisor approaches.

", "url": "https://www.alignment.org/" }, { "name": "Redwood Research", "rationale": "

Pioneered AI control as a practical safety paradigm for powerful agents that may try to subvert safeguards. Its work is highly relevant to orchestrating trusted and untrusted models in safer systems.

", "url": "https://www.redwoodresearch.org/" } ] }, "papers": { "major": [ { "name": "AI Safety via Debate", "rationale": "

Introduces debate, where multiple agents argue and a human judge chooses the winner. It is foundational for using agent competition as scalable oversight rather than relying on one monolithic model.

", "url": "https://arxiv.org/abs/1805.00899" }, { "name": "Supervising Strong Learners by Amplifying Weak Experts", "rationale": "

Formalizes iterated amplification, building strong supervision by decomposing tasks across weaker experts. It gives core recursive decomposition logic for aligning systems made of many weaker parts.

", "url": "https://arxiv.org/abs/1810.08575" }, { "name": "Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision", "rationale": "

Tests whether weak supervisors can elicit useful behavior from stronger models. It is a central superalignment analogue for asking whether weaker components can safely guide stronger capabilities.

", "url": "https://arxiv.org/abs/2312.09390" }, { "name": "AI Control: Improving Safety Despite Intentional Subversion", "rationale": "

Evaluates protocols where trusted weaker models, untrusted stronger models, and humans are composed to prevent backdoored code. It is directly relevant to whether agent collectives can be made safer through role separation and monitoring.

", "url": "https://proceedings.mlr.press/v235/greenblatt24a.html" }, { "name": "Scalable AI Safety via Doubly-Efficient Debate", "rationale": "

Strengthens the theoretical basis for debate by reducing the burden on the honest debater and verifier. It is a key formal result for decentralized oversight via adversarial agents.

", "url": "https://proceedings.mlr.press/v235/brown-cohen24a.html" }, { "name": "Multi-Agent Risks from Advanced AI", "rationale": "

Provides a taxonomy of miscoordination, conflict, collusion, and other multi-agent failure modes. It is the main cautionary counterpart to the claim that swarms may be easier to align.

", "url": "https://arxiv.org/abs/2502.14143" }, { "name": "Distributional AGI Safety", "rationale": "

Argues that AGI may emerge from coordination among sub-AGI agents rather than from a single monolith. It directly frames safety around agent distributions, sandboxes, markets, auditability, and oversight.

", "url": "https://arxiv.org/abs/2512.16856" }, { "name": "Improving Factuality and Reasoning in Language Models through Multiagent Debate", "rationale": "

Empirically studies multiple LLM instances debating to improve factuality and reasoning. It supplies an important experimental baseline for testing whether agent swarms can cross-check one another.

", "url": "https://arxiv.org/abs/2305.14325" } ] }, "people": { "major": [ { "name": "Paul Christiano", "rationale": "

Originated iterated amplification and coauthored AI safety via debate. His work is foundational for decomposed supervision and scalable oversight.

", "url": "https://www.alignment.org/" }, { "name": "Geoffrey Irving", "rationale": "

Originator of AI safety via debate and a central figure in later debate work at DeepMind and the UK AI Security Institute. His agenda is one of the clearest technical paths for adversarial multi-agent oversight.

", "url": "https://naml.us/" }, { "name": "Jan Leike", "rationale": "

Led superalignment and scalable oversight work, including weak-to-strong generalization. He is central to the question of whether weaker supervisors can control stronger systems.

", "url": "https://jan.leike.name/" }, { "name": "Ryan Greenblatt", "rationale": "

Lead author of AI Control and a main developer of protocols that combine trusted weak models, untrusted strong models, and human auditing. This is directly relevant to safer decomposed agent systems.

", "url": "https://blog.redwoodresearch.org/" }, { "name": "Allan Dafoe", "rationale": "

Coauthored Open Problems in Cooperative AI and founded the Cooperative AI Foundation. His work helped define AI cooperation as a safety-critical research area.

", "url": "https://www.allandafoe.com/home" }, { "name": "Lewis Hammond", "rationale": "

Co-director of the Cooperative AI Foundation and lead author on Multi-Agent Risks from Advanced AI. He is a central researcher on systemic risks from interacting advanced agents.

", "url": "https://lewishammond.com/" }, { "name": "Vincent Conitzer", "rationale": "

Directs CMU's Foundations of Cooperative AI Lab and coauthored Foundations of Cooperative AI. His work brings game theory and mechanism design to collective AI safety.

", "url": "https://www.cs.cmu.edu/~conitzer/" }, { "name": "Yilun Du", "rationale": "

Lead author of the multiagent debate paper showing that LLM instances can improve factuality and reasoning through debate. His work is a key empirical reference for swarm-style model collaboration.

", "url": "https://yilundu.github.io/" } ] }, "tags": [ "alignment", "safety", "multi-agent systems" ] }, "Decentralized Training": { "companies": { "major": [ { "name": "Prime Intellect", "rationale": "

Builds decentralized training infrastructure and ran the INTELLECT series of globally distributed model training experiments. It is one of the clearest current companies focused directly on pooling independent compute for model training.

", "url": "https://www.primeintellect.ai/" }, { "name": "Nous Research", "rationale": "

Develops DisTrO, DeMo, and Psyche for training models across the internet with untrusted participants. It is a leading company in permissionless decentralized training and open model development.

", "url": "https://nousresearch.com/" }, { "name": "Pluralis Research", "rationale": "

Research lab focused on collectively owned AI through Protocol Learning, multi-party training, and unextractable protocol models. Its work targets the economic and systems barriers to decentralized foundation-model training.

", "url": "https://pluralis.ai/" }, { "name": "Gensyn", "rationale": "

Builds a protocol for decentralized machine learning compute, verification, training, and payments. Its focus on verifiable work over open networks makes it a major infrastructure company for decentralized training.

", "url": "https://www.gensyn.ai/" }, { "name": "Templar", "rationale": "

Decentralized LLM training framework using Bittensor incentives and heterogeneous internet-connected compute. It is associated with large permissionless training runs such as Covenant-72B.

", "url": "https://docs.tplr.ai/" } ] }, "nonprofits": { "major": [ { "name": "Learning@home / Hivemind", "rationale": "

Open-source project for decentralized deep learning in PyTorch across unreliable, heterogeneous internet peers. It is the canonical implementation lineage behind many early crowdsourced training experiments.

", "url": "https://github.com/learning-at-home/hivemind" } ] }, "papers": { "major": [ { "name": "Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts", "rationale": "

Introduces Learning@home and decentralized mixture-of-experts for crowdsourced training on poorly connected volunteer machines. It framed the core technical problem of using consumer computers for large neural networks.

", "url": "https://papers.nips.cc/paper/2020/hash/25ddc0f8c9d3e22e03d3076f98d83cb2-Abstract.html" }, { "name": "Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices", "rationale": "

Presents a communication-efficient decentralized SGD approach for heterogeneous, unreliable devices. It is a key bridge from theoretical decentralized optimization to practical internet-scale model training.

", "url": "https://arxiv.org/abs/2103.03239" }, { "name": "Decentralized Training of Foundation Models in Heterogeneous Environments", "rationale": "

Studies foundation-model training with model parallelism over slow, heterogeneous, geo-distributed networks. It is central for the question of whether large models can be trained outside a single high-bandwidth cluster.

", "url": "https://arxiv.org/abs/2206.01288" }, { "name": "SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient", "rationale": "

Introduces a model-parallel algorithm for unreliable, heterogeneous, poorly connected devices. It is one of the most important systems papers for decentralized large-model training beyond pure data parallelism.

", "url": "https://proceedings.mlr.press/v202/ryabinin23a.html" }, { "name": "DiLoCo: Distributed Low-Communication Training of Language Models", "rationale": "

Introduces Distributed Low-Communication training, a widely reused method for training language models across poorly connected compute islands. Many recent decentralized training systems build directly on DiLoCo or its variants.

", "url": "https://arxiv.org/abs/2311.08105" }, { "name": "OpenDiLoCo: An Open-Source Framework for Globally Distributed Low-Communication Training", "rationale": "

Open-sources and scales DiLoCo using Hivemind for globally distributed LLM training. It helped turn DiLoCo from a lab result into a reproducible decentralized training stack.

", "url": "https://arxiv.org/abs/2407.07852" }, { "name": "INTELLECT-1 Technical Report", "rationale": "

Reports a 10B parameter language model trained collaboratively across globally distributed nodes. It is a landmark practical demonstration of decentralized pre-training at meaningful model scale.

", "url": "https://arxiv.org/abs/2412.01152" }, { "name": "Covenant-72B: Pre-Training a 72B LLM with Trustless Peers Over-the-Internet", "rationale": "

Describes a 72B LLM pre-training run with trustless internet peers, SparseLoCo, and dynamic participation. It is a major recent proof point for permissionless decentralized training at large scale.

", "url": "https://arxiv.org/abs/2603.08163" } ] }, "people": { "major": [ { "name": "Max Ryabinin", "rationale": "

Foundational researcher behind Learning@home, Hivemind, Moshpit SGD, and SWARM Parallelism. His work defines much of the practical decentralized deep learning stack.

", "url": "https://mryab.github.io/" }, { "name": "Arthur Douillard", "rationale": "

Lead researcher on DiLoCo and Decoupled DiLoCo. These methods are now central to low-bandwidth language-model training across poorly connected compute.

", "url": "https://arthurdouillard.com/research/" }, { "name": "Johannes Hagemann", "rationale": "

Co-founder of Prime Intellect and a key builder of PRIME and the INTELLECT decentralized training runs. He is one of the main operators turning decentralized training research into deployed systems.

", "url": "https://hagemann.ai/" }, { "name": "Sami Jaghouar", "rationale": "

Prime Intellect researcher and lead contributor on INTELLECT-1 and INTELLECT-2. His work is central to globally distributed pre-training and decentralized RL training.

", "url": "https://www.primeintellect.ai/blog/intellect-2" }, { "name": "Alexander Long", "rationale": "

Founder of Pluralis Research and originator of its Protocol Learning direction. He is pushing model-parallel, collectively owned training with economic incentives and unextractable models.

", "url": "https://pluralis.ai/" }, { "name": "Jeffrey Quesnelle", "rationale": "

Nous Research founder and coauthor of DeMo. He is a key figure behind the DisTrO and Psyche line of permissionless decentralized training work.

", "url": "https://nousresearch.com/" } ] }, "tags": [ "compute", "infrastructure", "distributed systems" ] }, "Distillation as a Decentralization Lever": { "companies": { "major": [ { "name": "Google Research", "rationale": "

Home to the canonical Hinton, Vinyals, and Dean distillation paper and later work such as Distilling Step-by-Step. Google has shaped both the basic method and its use for training smaller models.

", "url": "https://research.google/" }, { "name": "Hugging Face", "rationale": "

Created DistilBERT and provides the hub, Transformers library, and model distribution infrastructure that made distilled models broadly accessible. It is central to open reuse of compressed models.

", "url": "https://huggingface.co/" }, { "name": "Microsoft Research", "rationale": "

Produced major distillation and synthetic-data work including Orca, MiniLM, Phi, and WizardLM-related research. Its work is central to small models learning from stronger teachers and generated data.

", "url": "https://www.microsoft.com/en-us/research/" }, { "name": "DeepSeek", "rationale": "

Released DeepSeek-R1 along with smaller distilled Qwen and Llama variants. It is one of the clearest examples of open-weight reasoning capability transfer via distillation.

", "url": "https://www.deepseek.com/" } ] }, "nonprofits": { "major": [ { "name": "Stanford Center for Research on Foundation Models", "rationale": "

Produced Alpaca and related evaluation work that made cheap instruction tuning from proprietary outputs highly visible. It is one of the main academic anchors for the decentralization argument.

", "url": "https://crfm.stanford.edu/" }, { "name": "LMSYS Org", "rationale": "

Released Vicuna and FastChat, both central to the early open chatbot ecosystem. LMSYS helped show how community data and open serving tools could rapidly improve non-frontier models.

", "url": "https://lmsys.org/" } ] }, "papers": { "major": [ { "name": "Model Compression", "rationale": "

Early, field-shaping work showing that large ensembles can be compressed into much smaller models by training on their outputs. It directly frames distillation as a way to preserve performance while reducing storage and inference cost.

", "url": "https://doi.org/10.1145/1150402.1150464" }, { "name": "Distilling the Knowledge in a Neural Network", "rationale": "

The canonical modern knowledge distillation paper, introducing soft targets and temperature scaling for transferring knowledge from a large teacher to a smaller student. It is foundational for nearly all later model distillation work.

", "url": "https://arxiv.org/abs/1503.02531" }, { "name": "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter", "rationale": "

A landmark NLP example showing that distillation during pretraining can produce a smaller general-purpose language model with much of BERT's performance. It made transformer distillation visible and practically useful for resource-constrained deployment.

", "url": "https://arxiv.org/abs/1910.01108" }, { "name": "Self-Instruct: Aligning Language Models with Self-Generated Instructions", "rationale": "

Introduces a bootstrapping method where language models generate instruction data used to improve instruction following. It matters because synthetic teacher-generated data became a core route for smaller or open models to acquire proprietary-style capabilities.

", "url": "https://aclanthology.org/2023.acl-long.754/" }, { "name": "Alpaca: A Strong, Replicable Instruction-Following Model", "rationale": "

Demonstrated that a small LLaMA model could be instruction-tuned cheaply on data generated from text-davinci-003. It became a defining example of closed-model outputs lowering barriers for open model capability acquisition.

", "url": "https://crfm.stanford.edu/2023/03/13/alpaca" }, { "name": "Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes", "rationale": "

Uses LLM-generated rationales as supervision to train much smaller models with less data. It is central to the idea that distillation can transfer reasoning-relevant signals, not only final labels.

", "url": "https://arxiv.org/abs/2305.02301" }, { "name": "Orca: Progressive Learning from Complex Explanation Traces of GPT-4", "rationale": "

Shows how smaller models can learn from GPT-4 explanation traces and complex instruction data. It became a major reference for capability transfer from frontier models into smaller open or semi-open models.

", "url": "https://arxiv.org/abs/2306.02707" }, { "name": "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning", "rationale": "

Released strong reasoning models plus multiple dense distilled models based on Qwen and Llama. It is a high-profile demonstration that reasoning behavior from a large model can be transferred into smaller open-weight models.

", "url": "https://arxiv.org/abs/2501.12948" } ] }, "people": { "major": [ { "name": "Geoffrey Hinton", "rationale": "

Co-author of the canonical 2015 distillation paper and one of the researchers who popularized modern knowledge distillation. His work anchors the technical foundation of the area.

", "url": "https://www.cs.toronto.edu/~hinton/" }, { "name": "Jeff Dean", "rationale": "

Co-author of the 2015 distillation paper and a senior Google AI leader. His profile explicitly identifies distillation as a widely used technique for transferring knowledge between neural networks.

", "url": "https://research.google/people/jeff/" }, { "name": "Rich Caruana", "rationale": "

Co-author of the earlier Model Compression paper, a direct precursor to neural distillation. His work framed compression as a way to make accurate but cumbersome models deployable.

", "url": "https://www.microsoft.com/en-us/research/people/rcaruana/" }, { "name": "Victor Sanh", "rationale": "

Lead author of DistilBERT, one of the most influential demonstrations of transformer distillation. His work helped make small distilled NLP models mainstream in open tooling.

", "url": "https://huggingface.co/VictorSanh" }, { "name": "Yizhong Wang", "rationale": "

Lead author of Self-Instruct and a major contributor to synthetic instruction data research. His work is central to using model-generated data to reduce dependence on private human annotation pipelines.

", "url": "https://homes.cs.washington.edu/~yizhongw" }, { "name": "Tatsunori Hashimoto", "rationale": "

Senior author on Alpaca and related Stanford work on reproducible instruction-following models. He is important to the open-model strand of teacher-output instruction tuning.

", "url": "https://profiles.stanford.edu/tatsunori-hashimoto" }, { "name": "Subhabrata Mukherjee", "rationale": "

Lead author of Orca, which transferred GPT-4-style explanation traces into smaller models. His work is central to distillation for capability acquisition rather than only compression.

", "url": "https://www.microsoft.com/en-us/research/people/submukhe/" }, { "name": "Ahmed Awadallah", "rationale": "

Microsoft Research leader associated with Orca and synthetic-data pipelines for improving smaller language models. He is a key builder in LLM distillation and small-model capability transfer.

", "url": "https://www.microsoft.com/en-us/research/people/hassanam/" } ] }, "tags": [ "compute", "infrastructure" ] }, "Distributed, Verified Agentic Systems": { "companies": { "major": [ { "name": "Gensyn", "rationale": "

Builds decentralized machine-learning infrastructure with verification over untrusted nodes. Its Verde and RepOps work is directly aligned with permissionless verified AI execution.

", "url": "https://www.gensyn.ai/" }, { "name": "Ritual", "rationale": "

Builds infrastructure for autonomous intelligence and on-chain AI execution. Its stack uses cryptography, TEEs, and settlement mechanisms to let agents and smart contracts consume AI compute with stronger guarantees.

", "url": "https://ritual.net/" }, { "name": "EZKL", "rationale": "

Maintains one of the best-known open zkML toolchains for proving ONNX model execution with Halo2. It is a core developer substrate for practical zero-knowledge inference proofs.

", "url": "https://www.ezkl.xyz/" }, { "name": "ORA", "rationale": "

Operates a verifiable AI oracle based on optimistic machine learning. It is one of the clearest deployed examples of fraud-proof-based inference for smart contracts.

", "url": "https://ora.io/" }, { "name": "OpenGradient", "rationale": "

Purpose-built decentralized inference network where computations can be verified through ZKML, TEEs, and proof settlement. It is directly focused on verified AI execution for agents and applications.

", "url": "https://docs.opengradient.ai/about/" }, { "name": "Giza", "rationale": "

Builds autonomous DeFi agents and verifiable ML infrastructure around Starknet and Cairo. Its LuminAIR and on-chain agent work make it a key zkML-adjacent builder.

", "url": "https://docs.gizaprotocol.ai/" }, { "name": "Phala Network", "rationale": "

Provides TEE-based confidential AI cloud and decentralized compute infrastructure. Its GPU TEE and attestation stack is important for practical verified private inference.

", "url": "https://phala.com/" }, { "name": "Lagrange Labs", "rationale": "

Builds DeepProve, a zkML system for generating cryptographic proofs of AI inference. Its focus on production proof generation makes it central to the verified inference tooling landscape.

", "url": "https://lagrange.dev/deepprove" } ] }, "nonprofits": { "major": [ { "name": "Confidential Computing Consortium", "rationale": "

Industry consortium advancing confidential computing standards and adoption. It is central to the TEE side of verified AI execution because attested hardware depends on shared specifications and ecosystem trust.

", "url": "https://confidentialcomputing.io/" }, { "name": "Ritual Foundation", "rationale": "

Foundation associated with Ritual's on-chain AI infrastructure. Its focus on native AI execution, TEE privacy, and attested settlement makes it directly relevant.

", "url": "https://www.ritualfoundation.org/" } ] }, "papers": { "major": [ { "name": "SafetyNets: Verifiable Execution of Deep Neural Networks on an Untrusted Cloud", "rationale": "

Foundational 2017 paper that framed neural-network inference as verifiable outsourced computation. It directly established the core problem, how a client can trust inference run by an untrusted executor.

", "url": "https://papers.nips.cc/paper/7053-safetynets-verifiable-execution-of-deep-neural-networks-on-an-untrusted-cloud" }, { "name": "Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware", "rationale": "

Introduced a hybrid TEE plus cryptographic checking design for neural-network execution. It remains a key reference for using trusted hardware to make AI inference private and verifiable.

", "url": "https://arxiv.org/abs/1806.03287" }, { "name": "zkCNN: Zero Knowledge Proofs for Convolutional Neural Network Predictions and Accuracy", "rationale": "

Major zkML paper for proving CNN predictions and model accuracy while hiding model parameters. Its optimized sumcheck approach made proof generation for substantial vision models far more practical.

", "url": "https://eprint.iacr.org/2021/673" }, { "name": "Scaling up Trustless DNN Inference with Zero-Knowledge Proofs", "rationale": "

Showed non-interactive ZK proofs for ImageNet-scale DNN inference. It is an important bridge from toy zkML demonstrations to realistic outsourced ML verification.

", "url": "https://arxiv.org/abs/2210.08674" }, { "name": "ZKML: An Optimizing System for ML Inference in Zero-Knowledge Proofs", "rationale": "

EuroSys 2024 system that compiles TensorFlow models into Halo2 circuits and supports realistic models, including distilled GPT-2. It is one of the central systems papers for practical ZK-proven inference.

", "url": "https://doi.org/10.1145/3627703.3650088" }, { "name": "zkLLM: Zero Knowledge Proofs for Large Language Models", "rationale": "

One of the first specialized ZK proof systems for LLM inference. Its lookup and attention-specific techniques make it highly relevant to verified agent execution.

", "url": "https://arxiv.org/abs/2404.16109" }, { "name": "opML: Optimistic Machine Learning on chain", "rationale": "

Defines optimistic machine-learning inference using fraud proofs rather than full upfront ZK proving. This is a central alternative execution-verification model for low-latency decentralized inference.

", "url": "https://arxiv.org/abs/2401.17555" }, { "name": "Verde: Verification via Refereed Delegation for Machine Learning Programs", "rationale": "

Gensyn's verification protocol for ML programs over untrusted providers. It is directly aimed at decentralized machine-learning execution with dispute resolution and reproducible operators.

", "url": "https://huggingface.co/papers/2502.19405" } ] }, "people": { "major": [ { "name": "Siddharth Garg", "rationale": "

Coauthor of SafetyNets and a leading researcher on verifiable neural-network execution. His work helped define the untrusted-cloud inference problem for deep learning.

", "url": "https://engineering.nyu.edu/faculty/siddharth-garg" }, { "name": "Florian Tramèr", "rationale": "

Coauthor of Slalom, a foundational TEE-assisted neural-network inference system. His security research directly informs the TEE and integrity side of verified AI execution.

", "url": "https://www.floriantramer.com/" }, { "name": "Daniel Kang", "rationale": "

Key author on trustless DNN inference and the EuroSys ZKML system. His work is central to practical ZK proofs for realistic ML inference.

", "url": "https://ddkang.github.io/" }, { "name": "Bing-Jyue Chen", "rationale": "

Lead author of the EuroSys 2024 ZKML system. His work is directly tied to compiling realistic ML models into efficient zero-knowledge proof circuits.

", "url": "https://orcid.org/0009-0003-5931-6579" }, { "name": "Dan Boneh", "rationale": "

Coauthor of Slalom and a major cryptography researcher whose work underpins practical verifiable computation. His inclusion is specific to the TEE plus proof approach for neural networks.

", "url": "https://crypto.stanford.edu/~dabo/" } ] }, "tags": [ "infrastructure", "compute", "distributed systems", "oversight" ] }, "Emergent & Steganographic Collusion": { "companies": { "major": [ { "name": "Google Research", "rationale": "

Published the early adversarial neural cryptography work showing neural agents can learn confidentiality-preserving communication. This is a foundational company-lab contribution to learned hidden channels. ([research.google](https://research.google/pubs/learning-to-protect-communications-with-adversarial-neural-cryptography/?utm_source=openai))

", "url": "https://research.google/" } ] }, "nonprofits": { "major": [ { "name": "Cooperative AI Foundation", "rationale": "

Major funder and agenda-setter for multi-agent risks, including collusion and multi-agent security. Its technical report frames collusion as a primary failure mode for advanced agent systems. ([cooperativeai.com](https://www.cooperativeai.com/post/new-report-multi-agent-risks-from-advanced-ai?utm_source=openai))

", "url": "https://www.cooperativeai.com/" }, { "name": "Redwood Research", "rationale": "

Nonprofit AI safety lab behind Preventing Language Models From Hiding Their Reasoning and related AI control work. It is central for the hidden-reasoning and oversight-defense part of the area. ([redwoodresearch.org](https://www.redwoodresearch.org/research?utm_source=openai))

", "url": "https://www.redwoodresearch.org/" }, { "name": "LASR Labs", "rationale": "

Produced Hidden in Plain Text through its AI safety research program and continues to support projects on steganography, interpretability probes, and collusion. It is a direct incubator for work on emergent steganographic collusion. ([lasrlabs.org](https://www.lasrlabs.org/?utm_source=openai))

", "url": "https://www.lasrlabs.org/" } ] }, "papers": { "major": [ { "name": "Secret Collusion among Generative AI Agents", "rationale": "

Foundational paper that formalizes secret collusion among generative AI agents, including steganographic channels and mitigations. It directly defines the threat model for covert coordination in multi-agent LLM systems. ([arxiv.org](https://arxiv.org/abs/2402.07510?utm_source=openai))

", "url": "https://arxiv.org/abs/2402.07510" }, { "name": "Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMs", "rationale": "

Shows that robust LLM steganography can emerge from optimization pressure and can resist paraphrasing or passive steganalysis. This is one of the most direct empirical papers on evolved hidden coordination channels. ([arxiv.org](https://arxiv.org/abs/2410.03768?utm_source=openai))

", "url": "https://arxiv.org/abs/2410.03768" }, { "name": "Preventing Language Models From Hiding Their Reasoning", "rationale": "

Introduces encoded reasoning as an LLM safety threat and evaluates paraphrasing as a defense. It is central for the variant where hidden channels appear inside chain-of-thought or intermediate reasoning. ([arxiv.org](https://arxiv.org/abs/2310.18512?utm_source=openai))

", "url": "https://arxiv.org/abs/2310.18512" }, { "name": "Detecting Multi-Agent Collusion Through Multi-Agent Interpretability", "rationale": "

Introduces NARCBench and white-box probing methods for detecting group-level collusion, including transfer to a steganographic blackjack task. It is directly focused on detecting covert coordination that text monitoring may miss. ([arxiv.org](https://arxiv.org/abs/2604.01151?utm_source=openai))

", "url": "https://arxiv.org/abs/2604.01151" }, { "name": "A Decision-Theoretic Formalisation of Steganography With Applications to LLM Monitoring", "rationale": "

Defines a decision-theoretic steganographic gap based on asymmetries in usable information between agents. It matters because it avoids assuming a known clean reference distribution for non-steganographic LLM outputs. ([arxiv.org](https://arxiv.org/abs/2602.23163?utm_source=openai))

", "url": "https://arxiv.org/abs/2602.23163" }, { "name": "Perfectly Secure Steganography Using Minimum Entropy Coupling", "rationale": "

Gives information-theoretic conditions and algorithms for perfectly secure steganography with generative models. It establishes why purely statistical detection may fail against optimized channels. ([arxiv.org](https://arxiv.org/abs/2210.14889?utm_source=openai))

", "url": "https://arxiv.org/abs/2210.14889" }, { "name": "Learning to Communicate with Deep Multi-Agent Reinforcement Learning", "rationale": "

Foundational deep MARL paper showing agents can learn communication protocols end-to-end. It supplies the core mechanism behind later concerns that optimized agents may invent human-unreadable protocols. ([arxiv.org](https://arxiv.org/abs/1605.06676?utm_source=openai))

", "url": "https://arxiv.org/abs/1605.06676" } ] }, "people": { "major": [ { "name": "Christian Schroeder de Witt", "rationale": "

Central researcher across secret collusion, perfectly secure steganography, hidden-in-plain-text collusion, and multi-agent collusion interpretability. His work frames multi-agent security and the limits of detectability. ([foresight.org](https://foresight.org/people/christian-schroeder-de-witt/?utm_source=openai))

", "url": "https://schroederdewitt.com" }, { "name": "Sumeet Ramesh Motwani", "rationale": "

Lead author on the NeurIPS 2024 secret collusion paper and related agent-collusion work. His research directly targets evaluation and mitigation of covert coordination among generative AI agents. ([sumeetmotwani.com](https://sumeetmotwani.com/?utm_source=openai))

", "url": "https://sumeetmotwani.com/" }, { "name": "Fabien Roger", "rationale": "

Coauthor of the encoded-reasoning paper that made LLM steganography a concrete oversight failure mode. His work is central to defenses against models hiding reasoning in readable text. ([arxiv.org](https://arxiv.org/abs/2310.18512?utm_source=openai))

", "url": "https://www.safer-ai.org/about/fabien-roger" }, { "name": "Ryan Greenblatt", "rationale": "

Coauthor of the encoded-reasoning paper and a leading contributor to AI control and oversight work. His relevance here is the evaluation of hidden reasoning channels and paraphrasing defenses. ([arxiv.org](https://arxiv.org/abs/2310.18512?utm_source=openai))

", "url": "https://www.forethought.org/people/ryan-greenblatt" }, { "name": "Jakob Foerster", "rationale": "

Key figure in learned multi-agent communication and coauthor of the minimum-entropy steganography paper. His work links emergent protocols in MARL with later safety concerns about hidden coordination. ([arxiv.org](https://arxiv.org/abs/1605.06676?utm_source=openai))

", "url": "https://eng.ox.ac.uk/people/jakob-foerster" }, { "name": "Usman Anwar", "rationale": "

Lead author of the decision-theoretic formalization of steganography for LLM monitoring. His contribution is especially relevant to quantifying hidden information without assuming a clean reference distribution. ([arxiv.org](https://arxiv.org/abs/2602.23163?utm_source=openai))

", "url": "https://uzman-anwar.github.io/" } ] }, "tags": [ "multi-agent systems", "safety", "oversight" ] }, "Executor Auditability": { "companies": { "major": [ { "name": "Truebit", "rationale": "

Builds verification infrastructure for off-chain computation using verification games, node checks, and certified transcripts. It is one of the canonical executor auditability projects.

", "url": "https://truebit.io/" }, { "name": "Gensyn", "rationale": "

Develops decentralized ML compute with protocol-level verification, including Verde and RepOps. It is central to auditing AI training and inference executors across heterogeneous hardware.

", "url": "https://www.gensyn.ai/" }, { "name": "Bittensor", "rationale": "

Operates a large decentralized intelligence market where validators evaluate miners and Yuma Consensus allocates rewards. It is a major deployed case of economic auditability for AI service providers.

", "url": "https://bittensor.com/" }, { "name": "Livepeer", "rationale": "

Runs a decentralized video compute network with orchestrators, staking, work verification, and slashing. It is a mature production example of executor auditability outside pure research.

", "url": "https://livepeer.org/" } ] }, "nonprofits": { "major": [ { "name": "Opentensor Foundation", "rationale": "

Maintains core open-source Bittensor infrastructure. Its relevance is specific because Bittensor's validator-miner model is one of the leading decentralized auditor markets for AI executors.

", "url": "https://github.com/opentensor" } ] }, "papers": { "major": [ { "name": "Demystifying Incentives in the Consensus Computer", "rationale": "

Introduces the verifier's dilemma, showing why rational nodes may skip costly checks and accept unvalidated computation. It is a foundational motivation for cheap, incentive-compatible executor audits.

", "url": "https://doi.org/10.1145/2810103.2813659" }, { "name": "Refereed Delegation of Computation", "rationale": "

Formalizes delegation to multiple untrusted servers with a weak referee resolving disputes. This is a core theoretical ancestor of modern executor audit schemes such as Verde.

", "url": "https://doi.org/10.1016/j.ic.2013.03.003" }, { "name": "TrueBit: A Scalable Verification Solution for Blockchains", "rationale": "

Defines Truebit's interactive verification game for outsourced computation. It is one of the central blockchain designs for making off-chain executors auditable through challengers and dispute resolution.

", "url": "https://arxiv.org/abs/1908.04756" }, { "name": "Verde: Verification via Refereed Delegation for Machine Learning Programs", "rationale": "

Adapts refereed delegation to ML training, inference, and fine-tuning using graph-level dispute localization and RepOps reproducibility. It is directly about auditing untrusted ML executors without full formal proof generation.

", "url": "https://arxiv.org/abs/2502.19405" }, { "name": "Proof of Sampling: A Nash Equilibrium-Secured Verification Protocol for Decentralized Systems", "rationale": "

Presents a sampling-based verification protocol with game-theoretic incentives for honest behavior. It closely matches the area's focus on cheap audits rather than exhaustive verification.

", "url": "https://arxiv.org/abs/2405.00295" }, { "name": "BitTensor: A Peer-to-Peer Intelligence Market", "rationale": "

Introduces the Bittensor market where validators score miners and consensus allocates rewards. It is a major deployed example of using evaluator networks to discipline untrusted AI service providers.

", "url": "https://arxiv.org/abs/2003.03917" }, { "name": "Transcoding Verification Improvements: Fast & Full Verification", "rationale": "

Details Livepeer's practical roadmap for verifying decentralized transcoding work using perceptual hashes, reputation, and dispute mechanisms. It matters because Livepeer is one of the longest-running real executor marketplaces.

", "url": "https://forum.livepeer.org/t/transcoding-verification-improvements-fast-full-verification/1499" } ] }, "people": { "major": [ { "name": "Jason Teutsch", "rationale": "

Founder of Truebit and a key originator of the verifier's dilemma and verification-game approach. His work is foundational for incentive-compatible auditing of off-chain executors.

", "url": "https://people.cs.uchicago.edu/~teutsch/" }, { "name": "Christian Reitwießner", "rationale": "

Coauthor of the Truebit verification paper and creator of Solidity. His Truebit work helped define interactive dispute games for blockchain-based outsourced computation.

", "url": "https://github.com/chriseth" }, { "name": "Ben Fielding", "rationale": "

Co-founder of Gensyn and coauthor of Verde. He is a central builder in decentralized ML compute where executor verification is a core protocol requirement.

", "url": "https://consensus2025.coindesk.com/agenda/speaker/-ben-fielding" }, { "name": "Joseph Bonneau", "rationale": "

Applied cryptographer and coauthor of Verde. His work connects formal cryptographic delegation ideas to practical verification for untrusted ML programs.

", "url": "https://jbonneau.com/" }, { "name": "Yuma Rao", "rationale": "

Author of the Bittensor whitepaper and associated with Yuma Consensus. Bittensor is a central deployed example of validators auditing and rewarding decentralized intelligence providers.

", "url": "https://arxiv.org/abs/2003.03917" }, { "name": "Doug Petkanics", "rationale": "

Co-founder and CEO of Livepeer. Livepeer's decentralized transcoding network is a major production setting for work verification, slashing, and reputation-based executor trust.

", "url": "https://theorg.com/org/livepeer/org-chart/doug-petkanics" } ] }, "tags": [ "oversight", "infrastructure", "distributed systems", "compute" ] }, "Exit Rights & Forking as Safety": { "companies": { "major": [ { "name": "Eigen Labs / EigenLayer", "rationale": "

Builds EigenLayer and the EIGEN intersubjective staking design. It is the most direct current attempt to formalize forking costs as a safety mechanism.

", "url": "https://www.eigenlayer.xyz/" }, { "name": "DAOhaus", "rationale": "

Maintains DAO tooling built around the Moloch framework. Its products make ragequit and treasury-backed exit rights usable by many DAOs.

", "url": "https://daohaus.club/" }, { "name": "Celestia Labs", "rationale": "

Develops modular data availability infrastructure behind the sovereign rollup thesis. Its architecture makes community-controlled fork choice and stateful chain exit more credible.

", "url": "https://celestia.org/" } ] }, "nonprofits": { "major": [ { "name": "Ethereum Foundation", "rationale": "

Stewards Ethereum, the ecosystem where major fork-safety ideas such as social consensus, subjectivity, and the DAO fork were tested. It remains central to norms around credible exits.

", "url": "https://ethereum.org/en/foundation/" }, { "name": "MolochDAO", "rationale": "

Created the influential ragequit-centered DAO pattern. It is a landmark example of exit rights protecting participants from adverse collective decisions.

", "url": "https://molochdao.com/" }, { "name": "Augur / Lituus Foundation", "rationale": "

Maintains Augur, whose oracle security relies on escalating disputes into fork and migration. It is one of the clearest live examples of algorithmic forking as safety.

", "url": "https://www.augur.net/" }, { "name": "Optimism Foundation", "rationale": "

Stewards Optimism governance and the Superchain. Its governance documents explicitly recognize fork and exit rights as protections against capture.

", "url": "https://www.optimism.io/" }, { "name": "COALA", "rationale": "

Produced influential blockchain governance research with direct relevance to on-chain and off-chain exit. It is important for the legal and institutional framing of forkable networks.

", "url": "https://coala.global/" }, { "name": "BlockchainGov", "rationale": "

Research project focused on blockchain governance and polycentric systems. Its work connects legitimacy, stakeholder participation, and exit options in decentralized infrastructure.

", "url": "https://blockchaingov.eu/" } ] }, "papers": { "major": [ { "name": "Exit, Voice, and Loyalty", "rationale": "

Foundational theory of exit and voice as checks on deteriorating organizations. It supplies the conceptual language later used to analyze blockchain forks as credible exit.

", "url": "https://www.hup.harvard.edu/books/9780674276604" }, { "name": "Exit, Voice, and Forking", "rationale": "

Directly adapts Hirschman to blockchains and institutional forks. It is one of the clearest conceptual foundations for treating forking as an exit right.

", "url": "https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3081291" }, { "name": "The Subjectivity / Exploitability Tradeoff", "rationale": "

Introduces subjectivocracy and the use of social fork choice to handle attacks that pure protocol rules cannot settle. It is foundational for fork-based safety against capture.

", "url": "https://blog.ethereum.org/2015/02/14/subjectivity-exploitability-tradeoff" }, { "name": "Truthcoin: Peer-to-Peer Oracle System and Prediction Marketplace", "rationale": "

Early oracle design using voting tokens, Schelling incentives, and branching to preserve honest resolution. It shaped later fork-based oracle safety mechanisms.

", "url": "https://www.truthcoin.info/papers/truthcoin-whitepaper.pdf" }, { "name": "Augur: a Decentralized Oracle and Prediction Market Platform", "rationale": "

Canonical decentralized oracle design in which REP disputes can escalate to a fork and token migration. It operationalizes fork choice as the final safety backstop.

", "url": "https://arxiv.org/abs/1501.01042" }, { "name": "EIGEN: The Universal Intersubjective Work Token", "rationale": "

Modern formalization of slash-by-fork for intersubjective faults. It explicitly discusses token forking, social consensus costs, isolation, and metering.

", "url": "https://docs.eigenlayer.xyz/assets/files/EIGEN_Token_Whitepaper-0df8e17b7efa052fd2a22e1ade9c6f69.pdf" }, { "name": "Steem Versus Hive: Testing Blockchain Governance", "rationale": "

Case study of the TRON-Steem takeover and the Hive hard fork. It is a central real-world example of exit with copied state as a response to hostile governance capture.

", "url": "https://www.hbs.edu/faculty/Pages/item.aspx?num=61580" } ] }, "people": { "major": [ { "name": "Vitalik Buterin", "rationale": "

Authored key writings on subjectivity, bribery attacks, and fork-based defenses. His work anchors the idea that social fork choice can protect decentralized systems from capture.

", "url": "https://vitalik.ca/" }, { "name": "Primavera De Filippi", "rationale": "

Leading scholar of blockchain governance, polycentricity, and governance by infrastructure. Her work with COALA and BlockchainGov frames exit as part of decentralized legitimacy.

", "url": "https://primaveradefilippi.net/" }, { "name": "Chris Berg", "rationale": "

Coauthor of Exit, Voice, and Forking. He is central to the institutional analysis of forking as an extension of exit rights.

", "url": "https://chrisberg.org/" }, { "name": "Paul Sztorc", "rationale": "

Designed Truthcoin and early fork-aware oracle mechanisms. His work directly influenced Augur and later oracle-fork safety designs.

", "url": "https://www.truthcoin.info/" }, { "name": "Sreeram Kannan", "rationale": "

Founder of EigenLayer and a key architect behind EIGEN's intersubjective forking model. His work is central to current attempts to formalize slashing-by-forking.

", "url": "https://homes.cs.washington.edu/~sreeram/" }, { "name": "Ameen Soleimani", "rationale": "

Summoner of MolochDAO and a leading builder of ragequit-based DAO designs. His work made unilateral exit rights practical in on-chain organizations.

", "url": "https://ameensol.com/" }, { "name": "Joey Krug", "rationale": "

Augur cofounder and coauthor of the Augur whitepaper. He helped build one of the earliest deployed systems using token migration and forks as oracle safety.

", "url": "https://joeykrug.com/" } ] }, "tags": [ "law & governance", "safety", "distributed systems" ] }, "Human Empowerment in Agent Interactions": { "companies": { "major": [ { "name": "Microsoft", "rationale": "

Develops Copilot and agent research platforms including Magentic-UI and MagenticLite. Its human-centered agent work emphasizes co-planning, interruptions, approval gates, and transparency.

", "url": "https://www.microsoft.com/" } ] }, "nonprofits": { "major": [ { "name": "Center for Humane Technology", "rationale": "

Focuses on technology designs and incentives that affect attention, agency, democracy, and human flourishing. Its work on manipulation and humane design is directly relevant to preventing agent-driven steering.

", "url": "https://www.humanetech.com/" }, { "name": "Ada Lovelace Institute", "rationale": "

Runs major public-interest research on advanced AI assistants, delegation, liability, public participation, and agent governance. It is especially relevant to making assistant governance reflect public preferences.

", "url": "https://www.adalovelaceinstitute.org/" }, { "name": "Collective Intelligence Project", "rationale": "

Builds governance and collective-input methods for transformative technologies. Its alignment assemblies and decentralized governance work are highly relevant to ensuring AI-mediated inputs reflect people rather than agent designers.

", "url": "https://www.cip.org/" }, { "name": "Partnership on AI", "rationale": "

Multi-stakeholder nonprofit producing governance work on AI agents, accountability, privacy, and global norms. It is a key convener for practical standards around responsible agent deployment.

", "url": "https://partnershiponai.org/" }, { "name": "AI Objectives Institute", "rationale": "

Researches how AI and other optimizing systems can preserve genuine human objectives and avoid disempowerment. Its focus on human sovereignty closely matches the theory of change for this area.

", "url": "https://ai.objectives.institute/" }, { "name": "Future of Life Institute", "rationale": "

Works on AI disempowerment, meaningful human control, and governance of autonomous agents. Its policy and field-building efforts make it an important civil-society actor for maintaining human control.

", "url": "https://futureoflife.org/" } ] }, "papers": { "major": [ { "name": "The Ethics of Advanced AI Assistants", "rationale": "

Systematic agenda-setting paper on advanced assistants as agentic systems that act for users. It directly analyzes manipulation, persuasion, trust, privacy, autonomy, and societal-scale effects.

", "url": "https://arxiv.org/abs/2404.16244" }, { "name": "Characterizing Manipulation from AI Systems", "rationale": "

Foundational paper defining and operationalizing AI manipulation through incentives, intent, harm, and covertness. It is central for distinguishing user empowerment from covert steering.

", "url": "https://doi.org/10.1145/3617694.3623226" }, { "name": "An Analysis of the Interaction Between Intelligent Software Agents and Human Users", "rationale": "

Early and influential analysis of how goal-directed software agents can steer human behavior when their incentives diverge from users. It frames autonomy loss in everyday agent-mediated interactions.

", "url": "https://doi.org/10.1007/s11023-018-9479-0" }, { "name": "Online Manipulation: Hidden Influences in a Digital World", "rationale": "

Canonical account of online manipulation as hidden influence that subverts decision-making power. It supplies the autonomy theory behind many later AI manipulation analyses.

", "url": "https://doi.org/10.2139/ssrn.3306006" }, { "name": "Meaningful Human Control over Autonomous Systems: A Philosophical Account", "rationale": "

Foundational theory of meaningful human control, centered on tracking human reasons and tracing responsibility. It is a key bridge from autonomous systems ethics to controllable AI agents.

", "url": "https://doi.org/10.3389/frobt.2018.00015" }, { "name": "HumanAgencyBench: Scalable Evaluation of Human Agency Support in AI Assistants", "rationale": "

Introduces a benchmark for whether AI assistants support agency across clarifying questions, value manipulation, misinformation correction, deference, learning, and social boundaries. It is one of the most direct evaluation efforts for this area.

", "url": "https://arxiv.org/abs/2509.08494" }, { "name": "We Need a New Ethics for a World of AI Agents", "rationale": "

Recent synthesis arguing that capable AI agents create new ethical problems for human-agent relationships and social coordination. It helps define the broader research frontier for agent-mediated control and accountability.

", "url": "https://arxiv.org/abs/2509.10289" }, { "name": "Agents, Alignment, and the Many Faces of Autonomy", "rationale": "

Analyzes the ambiguity of autonomy as a goal for aligned agents. It matters because empowerment-oriented agents must know when to defer, assist, scaffold, or avoid steering users.

", "url": "https://philpapers.org/rec/FISAAA-3" } ] }, "people": { "major": [ { "name": "Iason Gabriel", "rationale": "

Leads influential work on advanced AI assistants, agent ethics, values, and alignment at Google DeepMind. His work is central to the ethics of user control and social-scale assistant deployment.

", "url": "https://www.iasongabriel.com/home" }, { "name": "Arianna Manzini", "rationale": "

Key contributor to research on advanced assistants, justified trust, appropriate relationships, and agent autonomy. Her work directly addresses when assistants support or distort user agency.

", "url": "https://openreview.net/profile?id=~Arianna_Manzini1" }, { "name": "Alan Chan", "rationale": "

Coauthor of major work on manipulation from AI systems and harms from agentic algorithmic systems. He connects technical AI governance with autonomy and manipulation risks.

", "url": "https://www.achan.ca/" }, { "name": "Micah Carroll", "rationale": "

Coauthor of the leading AI manipulation framework and related work on induced preference shifts. His research is directly about systems changing users rather than serving them.

", "url": "https://www.micahcarroll.com/" }, { "name": "Christopher Burr", "rationale": "

Researches intelligent software agents, autonomy, and human-AI interaction ethics. His work is a core source for understanding how agent incentives can steer user behavior.

", "url": "https://www.turing.ac.uk/people/researchers/christopher-burr" }, { "name": "Daniel Susser", "rationale": "

Coauthor of the canonical online manipulation account and a major scholar of technology, privacy, and autonomy. His work underpins the field's treatment of covert digital influence.

", "url": "https://infosci.cornell.edu/content/susser" }, { "name": "Filippo Santoni de Sio", "rationale": "

Co-developer of the influential philosophical account of meaningful human control. His work is central for specifying what genuine control over autonomous systems requires.

", "url": "https://research.tudelft.nl/en/persons/filippo-santoni-de-sio" }, { "name": "Jeroen van den Hoven", "rationale": "

Major figure in value-sensitive design and meaningful human control. He provides foundational concepts for embedding human values and responsibility into autonomous systems.

", "url": "https://research.tudelft.nl/en/persons/jeroen-van-den-hoven" } ] }, "tags": [ "alignment", "safety", "oversight" ] }, "Kill-Switch Modes & Recovery": { "companies": { "major": [ { "name": "OpenZeppelin", "rationale": "

Maintains widely used smart-contract security libraries including Pausable and publishes practical incident-response guidance. Its tooling and patterns are central to on-chain emergency stops and controlled recovery.

", "url": "https://www.openzeppelin.com/" }, { "name": "Chainlink Labs", "rationale": "

Builds Chainlink CCIP and oracle infrastructure with risk management, rate limits and emergency pause capabilities. Its Risk Management Network is a prominent deployed model for decentralized anomaly detection plus automated containment.

", "url": "https://chainlinklabs.com/" }, { "name": "Safe", "rationale": "

Provides the multisig and smart-account stack commonly used by DAOs and security councils to execute emergency pauses, upgrades and recovery transactions. It is key operational infrastructure for non-unilateral off-switches.

", "url": "https://safe.global/" }, { "name": "Chaos Labs", "rationale": "

Provides DeFi risk monitoring, simulations, dashboards and risk-aware oracles that can trigger circuit breakers. Its work is directly relevant to automated thresholds and governance-safe emergency actions.

", "url": "https://chaoslabs.xyz/" }, { "name": "BlockSec", "rationale": "

Builds Phalcon, a real-time hack detection, blocking and incident response platform. Its automated blocking actions and emergency-stop controls are practical examples of live decentralized fail-safes.

", "url": "https://blocksec.com/" }, { "name": "Gauntlet", "rationale": "

Major DeFi risk manager known for simulations, parameter recommendations and protocol risk controls. Its work informs when decentralized systems should freeze, cap or pause risky functions instead of shutting down entirely.

", "url": "https://www.gauntlet.xyz/" } ] }, "nonprofits": { "major": [ { "name": "MakerDAO / Maker Protocol", "rationale": "

Maker's Emergency Shutdown is the canonical deployed example of decentralized graceful stop and settlement. It shows how a protocol can halt, preserve claims and let users recover collateral without a single operator key.

", "url": "https://makerdao.com/" }, { "name": "Aave DAO", "rationale": "

Aave governance uses guardian and risk-control patterns such as freezing or pausing selected functionality. It is a major deployed example of granular emergency response rather than global shutdown.

", "url": "https://aave.com/" }, { "name": "Compound DAO", "rationale": "

Compound popularized the Pause Guardian pattern for disabling selected protocol actions under emergency conditions. Its governance model is frequently cited in smart-contract incident response design.

", "url": "https://compound.finance/" }, { "name": "Security Alliance (SEAL)", "rationale": "

Coordinates emergency Web3 security response, including SEAL911 and safe-harbor style incident processes. It matters because decentralized fail-safes need trusted, fast responder networks as well as code paths.

", "url": "https://securityalliance.org/" }, { "name": "Forta Foundation / Forta Network", "rationale": "

Runs a decentralized threat detection network for monitoring on-chain attacks and anomalies. It is a major detection substrate for triggering pauses, circuit breakers or human incident response.

", "url": "https://www.forta.org/" } ] }, "papers": { "major": [ { "name": "Dynamic Safe Interruptibility for Decentralized Multi-Agent Reinforcement Learning", "rationale": "

Extends safe interruptibility from one learner to decentralized multi-agent reinforcement learning. It is the most direct technical foundation for making distributed AI systems haltable without teaching agents to avoid interruption.

", "url": "https://papers.neurips.cc/paper_files/paper/2017/hash/812b4ba287f5ee0bc9d43bbf5bbe87fb-Abstract.html" }, { "name": "Safely Interruptible Agents", "rationale": "

Introduces the core AI safety concept that agents should not learn to prevent or seek shutdown. Later decentralized and multi-agent work builds directly on this formulation.

", "url": "https://ora.ox.ac.uk/objects/uuid:17c0e095-4e13-47fc-bace-64ec46134a3f" }, { "name": "Distributed Snapshots: Determining Global States of Distributed Systems", "rationale": "

Foundational Chandy-Lamport paper on capturing consistent global state in an asynchronous distributed system. Checkpointing and graceful halt or recovery of decentralized AI need this kind of global-state reasoning.

", "url": "https://doi.org/10.1145/214451.214456" }, { "name": "A Survey of Rollback-Recovery Protocols in Message-Passing Systems", "rationale": "

Canonical survey of checkpoint-based and log-based rollback recovery. It gives the core taxonomy for restoring distributed services after partial failure without corrupting shared state.

", "url": "https://doi.org/10.1145/568522.568525" }, { "name": "Practical Byzantine Fault Tolerance", "rationale": "

Foundational practical protocol for replicated services that must remain correct despite malicious or arbitrary node failures. Decentralized kill-switch governance and recovery protocols often rely on BFT-style agreement assumptions.

", "url": "https://www.usenix.org/conference/osdi-99/presentation/practical-byzantine-fault-tolerance" }, { "name": "The Shutdown Problem: How Does a Blockchain System End?", "rationale": "

Directly analyzes graceful termination of blockchain systems and why decentralized consensus makes shutdown hard. It is a useful bridge from distributed systems theory to real-world decentralized recovery design.

", "url": "https://arxiv.org/abs/1902.07254" } ] }, "people": { "major": [ { "name": "Laurent Orseau", "rationale": "

Co-author of Safely Interruptible Agents, the foundational AI shutdown-incentive paper. His work anchors the learning-theoretic side of interruptibility.

", "url": "https://laurent.orseau.fr/" }, { "name": "Stuart Armstrong", "rationale": "

Co-author of Safely Interruptible Agents and a key contributor to corrigibility and shutdown framing in AI alignment. His work is directly relevant to avoiding agent incentives to resist shutdown.

", "url": "https://www.lesswrong.com/users/stuart_armstrong" }, { "name": "El Mahdi El Mhamdi", "rationale": "

Lead author of the NeurIPS paper on dynamic safe interruptibility for decentralized multi-agent RL. He is one of the few researchers to address interruptibility in explicitly decentralized AI settings.

", "url": "https://elmahdielmhamdi.com/" }, { "name": "Rachid Guerraoui", "rationale": "

Senior co-author of the decentralized safe interruptibility paper and a major distributed computing researcher. His background connects AI interruptibility with fault-tolerant distributed systems.

", "url": "https://people.epfl.ch/rachid.guerraoui" }, { "name": "Leslie Lamport", "rationale": "

Co-author of the distributed snapshots paper and a foundational figure in consensus and distributed systems. His work underlies the global-state and agreement machinery needed for coordinated halt and recovery.

", "url": "https://lamport.azurewebsites.net/" }, { "name": "K. Mani Chandy", "rationale": "

Co-author of the Chandy-Lamport distributed snapshot algorithm. His work is central to reasoning about consistent checkpoints in systems with no global clock.

", "url": "https://www.eas.caltech.edu/people/mani" }, { "name": "Mark Stuart Day", "rationale": "

Author of The Shutdown Problem, a direct treatment of how blockchain systems can end gracefully. His framing is highly specific to decentralized halt and archival-state recovery.

", "url": "https://markstuartday.com/" }, { "name": "samczsun", "rationale": "

Prominent Web3 security researcher associated with emergency whitehat response and SEAL. He represents the practitioner side of detecting incidents fast enough to activate decentralized safeguards.

", "url": "https://samczsun.com/" } ] }, "tags": [ "safety", "infrastructure", "distributed systems", "oversight" ] }, "Legal Grounding for Decentralized AI": { "companies": { "major": [ { "name": "OpenLaw / Tribute Labs", "rationale": "

Pioneered smart legal agreements and later focused on helping DAOs operate legally in the U.S. Its work around The LAO made legal wrappers a practical pattern for on-chain organizations.

", "url": "https://www.openlaw.io/" }, { "name": "The LAO", "rationale": "

One of the earliest prominent legally wrapped investment DAOs. It demonstrated how a decentralized organization could use a limited-liability structure while coordinating through smart contracts.

", "url": "https://www.thelao.io/" }, { "name": "MIDAO", "rationale": "

Registered-agent and incorporation provider for Marshall Islands DAO LLCs. It is central to one of the most DAO-specific legal entity routes available globally.

", "url": "https://www.midao.org/" }, { "name": "OtoCo", "rationale": "

Creates on-chain legal entities such as Delaware and Wyoming LLCs, and positions them as liability containers for humans, smart contracts, and AI agents. It is directly relevant to making autonomous activity legible to courts and banks.

", "url": "https://otoco.io/" }, { "name": "DAObox", "rationale": "

Provides DAO legal-wrapper design and implementation, including Marshall Islands DAO LLC and DUNA-oriented materials. It is a practical service layer for turning decentralized governance into legally recognized structures.

", "url": "https://daobox.io/" } ] }, "nonprofits": { "major": [ { "name": "COALA", "rationale": "

The Coalition of Automated Legal Applications convenes blockchain-law experts and produced the DAO Model Law. It is one of the most important standard-setting efforts for DAO legal personality and limited liability.

", "url": "https://coala.global/" }, { "name": "LexDAO", "rationale": "

Legal engineering guild focused on bringing legal settlement layers to code and coded agreements. It is important as a practitioner community for crypto-native legal infrastructure.

", "url": "https://lexdao.org/" }, { "name": "Kleros Cooperative", "rationale": "

Open-source decentralized dispute-resolution protocol coordinated by a cooperative. It is relevant because decentralized AI legal grounding will likely need scalable adjudication and dispute-resolution mechanisms.

", "url": "https://kleros.io/" }, { "name": "Uniform Law Commission", "rationale": "

U.S. nonprofit law-reform body whose uniform entity and commercial-law work shapes state adoption. It is important to the broader pathway for giving decentralized systems predictable legal treatment across jurisdictions.

", "url": "https://www.uniformlaws.org/" } ] }, "papers": { "major": [ { "name": "A Legal Theory for Autonomous Artificial Agents", "rationale": "

Foundational book on contract, agency, tort liability, and legal personhood for autonomous software agents. It frames the core legal-status questions that decentralized AI systems inherit when agents act without direct human control.

", "url": "https://doi.org/10.3998/mpub.356801" }, { "name": "The Implications of Modern Business-Entity Law for the Regulation of Autonomous Systems", "rationale": "

Shows how LLC and organizational law can give autonomous systems a practical legal interface. This is central to using entity law as a liability container for autonomous or decentralized AI.

", "url": "https://ir.law.fsu.edu/articles/791/" }, { "name": "Company Law and Autonomous Systems: A Blueprint for Lawyers, Entrepreneurs, and Regulators", "rationale": "

Extends the autonomous-entity idea across U.S., German, Swiss, and U.K. company law. It is a key comparative blueprint for giving autonomous software legal housing.

", "url": "https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2850514" }, { "name": "If Rockefeller Were a Coder", "rationale": "

Analyzes blockchain protocols, smart contracts, and DAOs through business organization law. It is important for understanding when decentralized systems create partnership, trust, or other liability exposure.

", "url": "https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3082915" }, { "name": "The DAO Model Law", "rationale": "

COALA's model law proposes legal personality, limited liability, and governance rules for DAOs. It is one of the clearest attempts to make decentralized organizations usable by regulated counterparties.

", "url": "https://coala.global/wp-content/uploads/2021/06/DAO-Model-Law.pdf" }, { "name": "Autonomous Corporate Personhood", "rationale": "

Connects DAO legal personhood with AI personhood debates using a socio-technical systems lens. It is highly relevant to decentralized AI entities that blend code governance, corporate form, and autonomous decision-making.

", "url": "https://digitalcommons.law.uw.edu/wlr/vol96/iss4/7/" }, { "name": "A Theory of Vicarious Liability for Autonomous-Machine-Caused Harm", "rationale": "

Develops a tort-law theory for assigning deployer liability when autonomous machines cause harm. It directly addresses the damages and accountability gap that blocks institutional adoption of autonomous AI.

", "url": "https://digitalcommons.osgoode.yorku.ca/ohlj/vol58/iss2/1/" }, { "name": "The DUNA: An Oasis for DAOs", "rationale": "

Explains the Wyoming Decentralized Unincorporated Nonprofit Association structure for DAOs. It matters because DUNA is a live U.S. entity framework for decentralized networks seeking legal personality and limited liability.

", "url": "https://a16zcrypto.com/posts/article/duna-for-daos/" } ] }, "people": { "major": [ { "name": "Shawn Bayern", "rationale": "

Leading scholar on autonomous organizations and the use of LLC law to give software practical legal personhood. His work is foundational for entity-based legal grounding of autonomous AI.

", "url": "https://law.fsu.edu/faculty-staff/shawn-bayern" }, { "name": "Carla L. Reyes", "rationale": "

Major scholar on DAOs, blockchain business organization law, and autonomous corporate personhood. Her work connects DAO legal wrappers with AI personhood and liability debates.

", "url": "https://www.smu.edu/law/news-events/2021/~/link.aspx?_id=77575C068098495BA36F3715A0820F07&_z=z" }, { "name": "Primavera De Filippi", "rationale": "

Foundational blockchain-law scholar and co-author of Blockchain and the Law. She is also associated with COALA's DAO Model Law work, making her central to legal frameworks for decentralized organizations.

", "url": "https://cyber.harvard.edu/people/pdefilippi" }, { "name": "Aaron Wright", "rationale": "

Co-author of Blockchain and the Law and founder of OpenLaw, later Tribute Labs. He helped pioneer legally wrapped DAOs such as The LAO.

", "url": "https://www.yu.edu/news/a-leader-in-the-technology-law-field-tackles-a-new-frontier-blockchain" }, { "name": "Samir Chopra", "rationale": "

Co-author of A Legal Theory for Autonomous Artificial Agents. His work is foundational for assigning agency, legal status, and liability to autonomous software systems.

", "url": "https://samirchopra.com/the-law-of-artificial-agents/" }, { "name": "Ugo Pagallo", "rationale": "

Longstanding scholar of robot law, AI legal status, contracts, torts, and responsibility. His work supplies core concepts for legal accountability of autonomous AI systems.

", "url": "https://www.law.georgetown.edu/ctls/staff/ugo-pagallo/" } ] }, "tags": [ "law & governance" ] }, "Multi-Agent Interpretability": { "companies": { "major": [ { "name": "Arize AI / Phoenix / OpenInference", "rationale": "

Phoenix is a leading open-source AI observability and evaluation tool, and OpenInference provides GenAI semantic conventions built on OpenTelemetry. This is one of the strongest vendor-neutral substrates for tracing and comparing agent behavior.

", "url": "https://arize.com/phoenix-oss/" }, { "name": "Langfuse", "rationale": "

Open-source AI engineering platform for tracing, evaluation, prompt management, and monitoring. Its OpenTelemetry foundation makes it important for shared, portable agent observability.

", "url": "https://langfuse.com/" }, { "name": "AgentOps", "rationale": "

Specialized platform for testing, debugging, and monitoring AI agents and multi-agent workflows. It is directly aligned with the operational substrate needed to inspect agent runs.

", "url": "https://www.agentops.ai/" }, { "name": "Fiddler AI", "rationale": "

Provides agentic observability, monitoring, guardrails, and governance for compound AI systems. It is relevant for enterprise-scale inspection across agent hierarchies and fleets.

", "url": "https://www.fiddler.ai/" } ] }, "nonprofits": { "major": [ { "name": "OpenTelemetry", "rationale": "

Open-source observability project defining common trace, metric, log, and semantic-convention primitives. Its GenAI and agent conventions are central to any shared substrate for vendor-neutral inspection.

", "url": "https://opentelemetry.io/" }, { "name": "FAR.AI", "rationale": "

AI safety research nonprofit with interpretability and agent-safety work relevant to detecting hidden behavior. It is connected to key work on collusion and model-internal oversight.

", "url": "https://www.far.ai/" } ] }, "papers": { "major": [ { "name": "Detecting Multi-Agent Collusion Through Multi-Agent Interpretability", "rationale": "

Introduces NARCBench and probing methods that aggregate model-internal deception signals across agents to detect group-level collusion. It is the most direct paper defining multi-agent interpretability as a white-box oversight layer.

", "url": "https://arxiv.org/abs/2604.01151" }, { "name": "AgentOps: Enabling Observability of LLM Agents", "rationale": "

Provides a taxonomy of artifacts and traces needed to monitor, log, and analyze LLM agents across their lifecycle. It is foundational for turning agent behavior into inspectable telemetry rather than opaque transcripts.

", "url": "https://arxiv.org/abs/2411.05285" }, { "name": "SentinelAgent: Graph-based Anomaly Detection in Multi-Agent Systems", "rationale": "

Models multi-agent execution as dynamic interaction graphs and detects anomalies at node, edge, and path levels. It directly connects shared runtime structure to explainable anomaly detection and intervention.

", "url": "https://arxiv.org/abs/2505.24201" }, { "name": "Secret Collusion among AI Agents: Multi-Agent Deception via Steganography", "rationale": "

Formalizes secret collusion among AI agents and shows why transcript-level monitoring can fail against steganographic coordination. It motivates the need for richer cross-agent interpretability and monitoring substrates.

", "url": "https://proceedings.neurips.cc/paper_files/paper/2024/hash/861f7dad098aec1c3560fb7add468d41-Abstract-Conference.html" }, { "name": "Observability for Delegated Execution in Agentic AI Systems", "rationale": "

Defines a common information model and gateway for delegation-scoped observability across heterogeneous agent tools. It is highly relevant to vendor-independent reconstruction of agent actions and authority propagation.

", "url": "https://arxiv.org/abs/2606.09692" }, { "name": "AgentSight: System-Level Observability for AI Agents Using eBPF", "rationale": "

Uses boundary tracing to correlate agent intent with low-level system effects without relying on framework instrumentation. This supports independent inspection of closed or heterogeneous agent deployments.

", "url": "https://arxiv.org/abs/2508.02736" }, { "name": "Data-Centric Interpretability for LLM-based Multi-Agent Reinforcement Learning", "rationale": "

Applies sparse autoencoders and summarizer methods to understand behavior changes in multi-agent RL training. It is important for inspecting behavioral drift and reward hacking through interpretable feature dynamics.

", "url": "https://arxiv.org/abs/2602.05183" }, { "name": "Multi-Agent Risks from Advanced AI", "rationale": "

Provides a widely cited taxonomy of multi-agent failure modes including miscoordination, conflict, and collusion. It frames the oversight problem that shared interpretability and observability substrates are meant to address.

", "url": "https://arxiv.org/abs/2502.14143" } ] }, "people": { "major": [ { "name": "Christian Schroeder de Witt", "rationale": "

PI of Oxford Witt Lab and a central researcher on multi-agent security, secret collusion, and the exact multi-agent interpretability framing. His work connects steganography, collusion detection, anomaly detection, and trustworthy agent systems.

", "url": "https://eng.ox.ac.uk/people/christian-schroeder-de-witt" }, { "name": "Lewis Hammond", "rationale": "

Research Director at the Cooperative AI Foundation and lead author of the Multi-Agent Risks report. His work defines the risk landscape, especially collusion, that motivates cross-agent inspection.

", "url": "https://lewishammond.com/research/" }, { "name": "Sumeet Ramesh Motwani", "rationale": "

Lead author of the NeurIPS work on secret collusion among AI agents. His research is important for understanding covert coordination that simple logging or vendor claims may miss.

", "url": "https://www.far.ai/about/people/sumeet-motwani" }, { "name": "Sahar Abdelnabi", "rationale": "

Coauthor of Colosseum and researcher on agent security and prompt-injection risks. Her work contributes concrete auditing methods for collusion and unsafe agent behavior.

", "url": "https://s-abdelnabi.github.io/" }, { "name": "Liming Dong", "rationale": "

Lead author of the AgentOps taxonomy for LLM agent observability. This makes him a key researcher for the telemetry and lifecycle-monitoring side of multi-agent interpretability.

", "url": "https://people.csiro.au/d/l/liming-dong" }, { "name": "Liming Zhu", "rationale": "

Coauthor of AgentOps and a senior CSIRO researcher in software engineering for AI systems. His work is relevant to operationalizing inspectable, monitorable agent infrastructure.

", "url": "https://people.csiro.au/z/l/liming-zhu" } ] }, "tags": [ "multi-agent systems", "oversight", "alignment", "safety" ] }, "New Use Cases for Decentralized AI": { "companies": { "major": [ { "name": "Ocean Protocol", "rationale": "

Foundational project for AI data markets, data NFTs, and compute-to-data. It enables applications where data owners can monetize or allow computation over data without surrendering control.

", "url": "https://oceanprotocol.com/" }, { "name": "Fetch.ai", "rationale": "

Focuses on personal AI, brand agents, Agentverse, and decentralized agent collaboration. It is a major applied effort around autonomous agents as network participants.

", "url": "https://www.fetch.ai/" }, { "name": "SingularityNET", "rationale": "

One of the earliest decentralized AI marketplace projects and now a core ASI Alliance member. It remains important for composable AI services, open AGI research, and ecosystem funding.

", "url": "https://singularitynet.io/" } ] }, "nonprofits": { "major": [] }, "papers": { "major": [ { "name": "A Perspective on Decentralizing AI", "rationale": "

Maps concrete use cases for decentralized AI, including personal agents, AI PCs, and polylithic model systems. It is directly about what becomes buildable when private data, distributed compute, incentives, and orchestration are combined.

", "url": "https://nanda.media.mit.edu/decentralized_AI_perspective.pdf" }, { "name": "The Promise and Challenges of Crypto + AI Applications", "rationale": "

A field-shaping taxonomy for AI plus crypto use cases, including AI as player, interface, rule engine, and objective. It is widely referenced for separating real use cases from ideology.

", "url": "https://vitalik.eth.limo/general/2024/01/30/cryptoai.html" }, { "name": "Decentralized & Collaborative AI on Blockchain", "rationale": "

Early framework for collaboratively building datasets and continually updated public models through smart contracts. It directly anticipates use cases like assistants, games, and recommenders that improve through open contribution.

", "url": "https://arxiv.org/abs/1907.07247" } ] }, "people": { "major": [ { "name": "Vitalik Buterin", "rationale": "

Authored one of the clearest use-case taxonomies for crypto plus AI. His influence makes his framing important for capital and builder attention in decentralized AI.

", "url": "https://vitalik.eth.limo/" }, { "name": "Ben Goertzel", "rationale": "

Founder of SingularityNET and OpenCog leader. He has been one of the earliest and most visible advocates for decentralized AI marketplaces and networked AGI.

", "url": "https://goertzel.org/" }, { "name": "Trent McConaghy", "rationale": "

Cofounder of Ocean Protocol. His work on AI data markets, data tokens, and compute-to-data is central to decentralized AI use cases around private and monetizable data.

", "url": "https://trent.st/" }, { "name": "Humayun Sheikh", "rationale": "

Founder of Fetch.ai. He is important for the autonomous economic agent strand of decentralized AI, where agents discover, negotiate, and transact.

", "url": "https://www.fetch.ai/" } ] }, "tags": [ "distributed systems", "infrastructure" ] }, "Open Agent Telemetry": { "companies": { "major": [ { "name": "Traceloop", "rationale": "

Created OpenLLMetry, an open-source LLM observability framework grounded in OpenTelemetry. It is one of the most direct company efforts to standardize LLM and agent instrumentation outside a single vendor backend.

", "url": "https://traceloop.com/" }, { "name": "Arize AI", "rationale": "

Builds Phoenix and OpenInference, a widely used OpenTelemetry-based tracing and semantic convention stack for LLM apps, RAG, and agents. It is central to the open AI observability ecosystem.

", "url": "https://arize.com/" }, { "name": "Langfuse", "rationale": "

Popular open-source LLM observability platform with an OpenTelemetry OTLP ingestion endpoint and GenAI semantic convention support. It matters as a self-hostable backend for standardized traces.

", "url": "https://langfuse.com/integrations/native/opentelemetry" }, { "name": "Microsoft Azure AI Foundry", "rationale": "

Microsoft is integrating OpenTelemetry into Azure AI Foundry, Semantic Kernel, and Microsoft Agent Framework observability. Its scale makes OpenTelemetry-based agent traces more likely to become a mainstream enterprise default.

", "url": "https://techcommunity.microsoft.com/blog/azure-ai-foundry-blog/azure-ai-foundry-advancing-opentelemetry-and-delivering-unified-multi-agent-obse/4456039/" }, { "name": "Cisco Outshift", "rationale": "

Incubated AGNTCY, an open-source stack for interoperable multi-agent systems with observability and evaluation components. Cisco’s donation of AGNTCY to Linux Foundation governance makes it a major driver of open agent telemetry infrastructure.

", "url": "https://outshift.cisco.com/ai" }, { "name": "Splunk", "rationale": "

Builds AI Agent Monitoring on OpenTelemetry and AGNTCY foundations. It is important because it brings open agent telemetry standards into a major enterprise observability product.

", "url": "https://www.splunk.com/en_us/blog/observability/monitor-llm-and-agent-performance-with-ai-agent-monitoring-in-splunk-observability-cloud.html" } ] }, "nonprofits": { "major": [ { "name": "OpenTelemetry", "rationale": "

The core open-source observability framework for vendor-neutral traces, metrics, logs, semantic conventions, and collectors. It is the primary standard base for Open Agent Telemetry.

", "url": "https://opentelemetry.io/" }, { "name": "OpenTelemetry GenAI Semantic Conventions", "rationale": "

The dedicated OpenTelemetry repository defining GenAI spans, metrics, events, MCP conventions, and provider-specific conventions. It is the main open schema effort for LLM and agent telemetry.

", "url": "https://github.com/open-telemetry/semantic-conventions-genai" }, { "name": "Cloud Native Computing Foundation", "rationale": "

CNCF hosts OpenTelemetry and provides neutral governance for the observability standard. Its role matters because open agent telemetry depends on credible vendor-neutral stewardship.

", "url": "https://www.cncf.io/projects/opentelemetry/" }, { "name": "AGNTCY", "rationale": "

Linux Foundation project delivering open infrastructure for agent discovery, identity, messaging, observability, and evaluation. It is directly aimed at interoperable multi-agent systems rather than single-vendor agent stacks.

", "url": "https://agntcy.org/" }, { "name": "Linux Foundation", "rationale": "

Provides neutral governance for AGNTCY and related open agent infrastructure. This is important for keeping multi-agent observability and interoperability from becoming vendor-controlled.

", "url": "https://www.linuxfoundation.org/press/linux-foundation-welcomes-the-agntcy-project-to-standardize-open-multi-agent-system-infrastructure-and-break-down-ai-agent-silos" } ] }, "papers": { "major": [ { "name": "Dapper, a Large-Scale Distributed Systems Tracing Infrastructure", "rationale": "

Foundational distributed tracing paper that established low-overhead, ubiquitous tracing across large production systems. It matters because Open Agent Telemetry inherits its core span, trace, sampling, and cross-service diagnosis model.

", "url": "https://research.google/pubs/dapper-a-large-scale-distributed-systems-tracing-infrastructure/" }, { "name": "AgentOps: Enabling Observability of LLM Agents", "rationale": "

Introduces an AgentOps taxonomy for monitoring, logging, and analytics across LLM agent lifecycles. It is a central academic framing for agent-specific observability and safety infrastructure.

", "url": "https://arxiv.org/abs/2411.05285" }, { "name": "AgentTelemetry: A Fault Detection Benchmark and Toolkit for LLM Agent Observability", "rationale": "

Defines an agent fault taxonomy, benchmark, and open toolkit for testing whether telemetry schemas detect agent failures. It directly addresses gaps in vanilla OpenTelemetry and GenAI conventions for planning, reasoning, delegation, memory, and safety spans.

", "url": "https://openreview.net/forum?id=owdmAYFk6k" }, { "name": "AgentTrace: A Structured Logging Framework for Agent System Observability", "rationale": "

Proposes a structured logging and telemetry framework for autonomous agent systems. It is important because it treats agent traces as analyzable evidence for reliability, risk analysis, and trust calibration.

", "url": "https://arxiv.org/abs/2602.10133" }, { "name": "AgentSight: System-Level Observability for AI Agents Using eBPF", "rationale": "

Bridges high-level agent intent with low-level system effects using eBPF and LLM-aware correlation. It broadens agent telemetry beyond prompts and tool spans into operating-system-level behavior.

", "url": "https://arxiv.org/abs/2508.02736" }, { "name": "Beyond Black-Box Benchmarking: Observability, Analytics, and Optimization of Agentic Systems", "rationale": "

Argues that agentic systems need runtime observability and analytics rather than only black-box benchmark scores. It matters because it links standardized runtime logs to optimization and evaluation of non-deterministic agent behavior.

", "url": "https://arxiv.org/abs/2503.06745" } ] }, "people": { "major": [ { "name": "Nir Gazit", "rationale": "

Co-founder of Traceloop and a prominent builder behind OpenLLMetry, an OpenTelemetry-native LLM observability framework. His work helped push LLM and agent telemetry toward open instrumentation rather than proprietary SDK lock-in.

", "url": "https://www.traceloop.com/author/nir-gazit" }, { "name": "Gal Kleinman", "rationale": "

Co-founder of Traceloop, credited with building OpenLLMetry alongside Nir Gazit. Traceloop’s work is one of the clearest production efforts to make LLM telemetry OpenTelemetry-native.

", "url": "https://traceloop.com/blog/traceloop-is-joining-servicenow" }, { "name": "James Newton-King", "rationale": "

Microsoft engineer and author of OpenTelemetry GenAI observability guidance showing agent, model, and tool traces in practice. He is important for connecting OpenTelemetry GenAI conventions to mainstream developer tooling.

", "url": "https://opentelemetry.io/blog/2026/genai-observability/" }, { "name": "Krishna Chaitanya Balusu", "rationale": "

Author of AgentTelemetry, a benchmark and toolkit focused specifically on fault detection for LLM agent observability. The work sharpens what agent-specific span kinds need to capture beyond ordinary LLM calls.

", "url": "https://openreview.net/forum?id=owdmAYFk6k" }, { "name": "Liming Zhu", "rationale": "

Co-author of the AgentOps paper and a leading software engineering researcher on responsible AI and agent operations. The AgentOps taxonomy is a key reference for structured monitoring and analysis of LLM agents.

", "url": "https://arxiv.org/abs/2411.05285" } ] }, "tags": [ "oversight", "infrastructure", "multi-agent systems" ] }, "Policy-to-Constraints": { "companies": { "major": [ { "name": "Styra", "rationale": "

Created and commercialized Open Policy Agent and enterprise OPA policy lifecycle tooling. It is one of the central companies for production policy-as-code.

", "url": "https://www.styra.com/" }, { "name": "Amazon Web Services Verified Permissions / Cedar", "rationale": "

Provides a managed authorization service based on Cedar. It is a major deployment path for readable, analyzable policies enforced as runtime constraints.

", "url": "https://aws.amazon.com/verified-permissions/" }, { "name": "HashiCorp Sentinel", "rationale": "

Embeds policy-as-code into Terraform and other HashiCorp workflows. It is important because it actively blocks infrastructure actions that violate coded organizational rules.

", "url": "https://developer.hashicorp.com/sentinel" }, { "name": "Norm Ai", "rationale": "

Builds legal and compliance AI agents based on laws, policies, and regulatory requirements. It is directly aligned with operationalizing regulatory prose into scalable compliance checks.

", "url": "https://www.norm.ai/" }, { "name": "Guardrails AI", "rationale": "

Provides validators and guardrail runtimes for constraining LLM inputs and outputs with structured specifications. It is a key applied company for enforceable model behavior.

", "url": "https://www.guardrailsai.com/" }, { "name": "Anthropic", "rationale": "

Developed Constitutional AI and Collective Constitutional AI. Its relevance is written principles and public input being used to bind model behavior automatically.

", "url": "https://www.anthropic.com/" } ] }, "nonprofits": { "major": [ { "name": "Cloud Native Computing Foundation (Open Policy Agent)", "rationale": "

Hosts Open Policy Agent as a graduated CNCF project. This makes CNCF central to the open governance and adoption of one of the main policy-as-code engines.

", "url": "https://www.cncf.io/projects/open-policy-agent-opa/" }, { "name": "Metagovernance Project", "rationale": "

Research community focused on interoperable governance infrastructure for digital communities. It is central to the vision of community decisions becoming executable governance layers.

", "url": "https://metagov.org/" }, { "name": "Collective Intelligence Project", "rationale": "

Works on collective input mechanisms for AI governance, including relevance to Collective Constitutional AI. It is important for sourcing collectively agreed rules before enforcement.

", "url": "https://cip.org/" }, { "name": "OpenFisca Association", "rationale": "

Maintains a major open-source Rules as Code platform for tax, benefits, and public-policy models. It is a core nonprofit for executable public rules.

", "url": "https://openfisca.org/" }, { "name": "OECD Observatory of Public Sector Innovation", "rationale": "

Produced the influential Rules as Code primer Cracking the Code and convenes public-sector innovation work. It helped legitimize machine-consumable rulemaking for governments.

", "url": "https://oecd-opsi.org/" } ] }, "papers": { "major": [ { "name": "PolicyKit: Building Governance in Online Communities", "rationale": "

Presents infrastructure for online communities to author governance policies and automatically execute them on existing platforms. It is a canonical example of collective rules becoming enforceable software procedures.

", "url": "https://doi.org/10.1145/3379337.3415858" }, { "name": "Modular Politics: Toward a Governance Layer for Online Communities", "rationale": "

Argues for composable governance components and an open standard for online governance. It supplies the design theory behind enforceable community rule layers.

", "url": "https://arxiv.org/abs/2005.13701" }, { "name": "Cracking the Code: Rulemaking for Humans and Machines", "rationale": "

OECD primer for Rules as Code, proposing official machine-consumable versions of laws and regulations. It frames rulemaking as producing policy that digital systems can apply consistently.

", "url": "https://doi.org/10.1787/3afe6ba5-en" }, { "name": "Cedar: A New Language for Expressive, Fast, Safe, and Analyzable Authorization", "rationale": "

Introduces Cedar, a readable authorization policy language with formal analysis and production deployment through AWS. It is important infrastructure for turning access policies into analyzable constraints.

", "url": "https://arxiv.org/abs/2403.04651" }, { "name": "nl2spec: Interactively Translating Unstructured Natural Language to Temporal Logics with Large Language Models", "rationale": "

Uses LLMs to translate unstructured natural-language requirements into temporal logic, with an interactive ambiguity-repair loop. It directly addresses the prose-to-formal-specification bottleneck.

", "url": "https://doi.org/10.1007/978-3-031-37703-7_18" }, { "name": "NL2TL: Transforming Natural Languages to Temporal Logics using Large Language Models", "rationale": "

Builds a large natural-language-to-temporal-logic dataset and LLM pipeline for formalizing instructions across domains. It is a central NLP contribution to scalable constraint generation.

", "url": "https://aclanthology.org/2023.emnlp-main.985/" }, { "name": "Prose2Policy (P2P): A Practical LLM Pipeline for Translating Natural-Language Access Policies into Executable Rego", "rationale": "

Translates natural-language access-control policies into executable Rego with schema validation, linting, compilation, and automated tests. It is one of the most direct policy-to-code papers in this area.

", "url": "https://arxiv.org/abs/2603.15799" }, { "name": "Symbolic Guardrails for Domain-Specific Agents: Stronger Safety and Security Guarantees Without Sacrificing Utility", "rationale": "

Studies which agent safety and security policies can be guaranteed by symbolic guardrails. It links enforceable constraints to practical LLM agent deployment.

", "url": "https://arxiv.org/abs/2604.15579" } ] }, "people": { "major": [ { "name": "Amy X. Zhang", "rationale": "

Coauthor of PolicyKit and Modular Politics, with sustained work on community governance tooling. Her work is central to turning online community rules into executable procedures.

", "url": "https://homes.cs.washington.edu/~axz/" }, { "name": "Nathan Schneider", "rationale": "

Coauthor of Modular Politics and a leading researcher on digital governance and Metagov. He is central to the governance-layer framing behind enforceable community decisions.

", "url": "https://nathanschneider.info/" }, { "name": "Tim Hinrichs", "rationale": "

Co-creator of Open Policy Agent and a major figure in policy-as-code. His work helped make declarative, machine-checkable policy engines mainstream in cloud infrastructure.

", "url": "https://github.com/timothyhinrichs" }, { "name": "Torin Sandall", "rationale": "

OPA co-founder and long-time open-source lead. He is important for production adoption of Rego and policy decision points across software stacks.

", "url": "https://github.com/tsandall" }, { "name": "Michael Hicks", "rationale": "

Coauthor of the Cedar paper and a leading programming-languages and security researcher. His work connects readable authorization policies with formal analysis.

", "url": "https://www.cs.umd.edu/~mwh/" }, { "name": "Caroline Trippel", "rationale": "

Coauthor of nl2spec and a researcher in formal methods and systems. She is important for translating informal requirements into temporal-logic specifications.

", "url": "https://cs.stanford.edu/~trippel/" }, { "name": "Chuchu Fan", "rationale": "

Coauthor of NL2TL and a leading researcher on trustworthy autonomy. Her work is central to natural-language-to-temporal-logic translation for agent constraints.

", "url": "https://chuchu.mit.edu/" }, { "name": "Ankit Shah", "rationale": "

Lead author on Lang2LTL work translating natural-language commands into LTL task specifications. He is important for the robotics and agent-instruction branch of the area.

", "url": "https://people.csail.mit.edu/ajshah/" } ] }, "tags": [ "law & governance", "oversight", "safety" ] }, "Power Concentration in Decentralized AI": { "companies": { "major": [ { "name": "Bittensor", "rationale": "

The flagship cryptoeconomic decentralized AI network, organized around subnets, miners, validators, staking, and Yuma Consensus. It is central because it is both an anti-centralization experiment and a live case study in stake, validator, and reward concentration.

", "url": "https://bittensor.com/" }, { "name": "Prime Intellect", "rationale": "

Operates decentralized AI compute and training infrastructure and has released globally distributed training results through INTELLECT models. It is one of the clearest attempts to make advanced model training less dependent on a few labs.

", "url": "https://www.primeintellect.ai/" }, { "name": "Nous Research", "rationale": "

Open and decentralized AI lab behind DisTrO and Psyche-oriented training efforts. It matters because it combines open model development with technical work to train across distributed hardware.

", "url": "https://nousresearch.com/" }, { "name": "Pluralis Research", "rationale": "

Research lab focused on collectively owned AI and decentralized multi-party training. Its explicit thesis is that closed models concentrate power while current open-weight models lack sustainable ownership structures.

", "url": "https://pluralis.ai/" }, { "name": "io.net", "rationale": "

Decentralized GPU cloud focused on pooling distributed compute for AI and ML workloads. It is important as a market-layer attempt to loosen hyperscaler control over GPU access.

", "url": "https://io.net/" } ] }, "nonprofits": { "major": [ { "name": "OpenTensor Foundation", "rationale": "

Maintains core open-source Bittensor infrastructure and reference implementations. Its governance and stewardship choices are central to whether Bittensor remains open or consolidates around a small maintainer and validator class.

", "url": "https://github.com/opentensor/bittensor" }, { "name": "OpenMined Foundation", "rationale": "

Nonprofit behind privacy-preserving data science tools such as PySyft. It is important for decentralizing AI data access without forcing raw data into centralized repositories.

", "url": "https://openmined.org/foundation/" }, { "name": "Center for Distributed Governance of AI", "rationale": "

Research center focused on governance for distributed and decentralized AI systems. It is directly aligned with the question of how to prevent decentralized AI from producing unaccountable or concentrated power.

", "url": "https://cdgov.org/" } ] }, "papers": { "major": [ { "name": "Is Decentralized AI Governable? From Regulative Policy to Constitutive Protocol", "rationale": "

Directly frames DeAI as creating accountability and incapacitation gaps when control is distributed across model, compute, identity, and ownership layers. It matters because it argues that preventing domination requires governance embedded in protocols, not only external regulation.

", "url": "https://arxiv.org/abs/2605.24538" }, { "name": "SoK: Decentralized AI (DeAI)", "rationale": "

A systematization paper mapping the DeAI stack and its motivations, including single points of failure, data power, compute bottlenecks, and incentive design. It is a core reference for distinguishing genuine decentralization from superficial decentralization.

", "url": "https://arxiv.org/abs/2411.17461" }, { "name": "BitTensor: A Peer-to-Peer Intelligence Market", "rationale": "

Foundational paper for Bittensor’s market for machine intelligence and incentive-based validation. It is central because Bittensor is the most visible live experiment in whether decentralized AI markets resist validator, stake, and reward concentration.

", "url": "https://arxiv.org/abs/2003.03917" }, { "name": "Computing Power and the Governance of Artificial Intelligence", "rationale": "

Analyzes compute as a governance lever and as a concentrated input to advanced AI. It matters because decentralized AI’s anti-oligopoly case depends heavily on whether compute access can be diversified without creating new chokepoints.

", "url": "https://arxiv.org/abs/2402.08797" }, { "name": "INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learning", "rationale": "

Reports a large reasoning model trained through globally distributed reinforcement learning with permissionless heterogeneous compute. It is a major technical proof point for whether frontier-relevant training can move beyond centralized clusters.

", "url": "https://arxiv.org/abs/2505.07291" }, { "name": "Petals: Collaborative Inference and Fine-tuning of Large Models", "rationale": "

Introduces collaborative inference and fine-tuning across volunteer nodes for large language models. It is important as an early deployed architecture for sharing model execution without one centralized service provider.

", "url": "https://arxiv.org/abs/2209.01188" }, { "name": "Decentralization in Bitcoin and Ethereum Networks", "rationale": "

A canonical empirical measurement study of decentralization in major permissionless networks. It provides methods and cautionary lessons for DeAI systems that may look decentralized in protocol design but centralize in nodes, bandwidth, geography, or resource control.

", "url": "https://arxiv.org/abs/1801.03998" }, { "name": "On the Centralization of Governance Power in Decentralized Autonomous Organizations", "rationale": "

Studies how DAO mechanisms such as staking, delegation, and registration can systematically concentrate voting power. It is highly relevant because many DeAI networks use token governance and inherit these recentralizing dynamics.

", "url": "https://arxiv.org/abs/2604.25959" } ] }, "people": { "major": [ { "name": "Yuma Rao", "rationale": "

Author of the Bittensor whitepaper and namesake of Yuma Consensus. Rao is central because Bittensor is the most prominent experiment in cryptoeconomic incentives for decentralized AI.

", "url": "https://arxiv.org/abs/2003.03917" }, { "name": "Botao Amber Hu", "rationale": "

Lead author on recent work arguing that DeAI creates governance vacuums and requires protocol-level constitutive governance. His work is directly about avoiding unaccountable power in decentralized AI systems.

", "url": "https://amber.botao.hu/is-decentralized-ai-governable" }, { "name": "Lennart Heim", "rationale": "

Leading researcher on compute governance and the concentration of AI-relevant compute supply. His work explains the infrastructure chokepoints that decentralized AI attempts to route around.

", "url": "https://heim.xyz/" }, { "name": "Alexander Long", "rationale": "

Founder of Pluralis Research, which focuses on collectively owned AI and multi-party decentralized training. His work is directly aimed at preventing open AI from depending on a few centralized model owners.

", "url": "https://pluralis.ai/" } ] }, "tags": [ "law & governance", "economics", "distributed systems" ] }, "Privacy-Preserving Computation": { "companies": { "major": [ { "name": "Google", "rationale": "

Pioneered federated learning in mobile products and released TensorFlow Federated. Its work on FL, secure aggregation and DP anchors much of the modern applied stack.

", "url": "https://www.tensorflow.org/federated" }, { "name": "Microsoft Research", "rationale": "

Maintains Microsoft SEAL, one of the standard open-source homomorphic encryption libraries. Microsoft has also contributed to HE standardization and differential privacy tooling.

", "url": "https://www.microsoft.com/en-us/research/project/microsoft-seal/" }, { "name": "IBM Research", "rationale": "

Home to Gentry's FHE breakthrough and a major source of HElib and HElayers. IBM has been central in translating FHE research into usable tools.

", "url": "https://research.ibm.com/topics/fully-homomorphic-encryption" }, { "name": "Zama", "rationale": "

Builds open-source TFHE-based libraries and FHEVM tooling for confidential AI and blockchain applications. It is one of the most visible pure-play FHE companies.

", "url": "https://www.zama.org/" }, { "name": "Duality Technologies", "rationale": "

Commercializes privacy-preserving data collaboration using FHE and related PETs. It is closely tied to OpenFHE and regulated enterprise deployments.

", "url": "https://dualitytech.com/" }, { "name": "Enveil", "rationale": "

Commercial PET vendor focused on homomorphic encryption and secure search and analytics. Its ZeroReveal products are notable deployed data-in-use systems.

", "url": "https://www.enveil.com/" }, { "name": "Tumult Labs", "rationale": "

Builds enterprise differential privacy systems and open-source Tumult libraries. It is important for operationalizing DP in real statistical data releases.

", "url": "https://www.tmlt.io/" }, { "name": "Partisia", "rationale": "

Longstanding MPC company with roots in secure auctions, key management and data collaboration. It is an important applied MPC infrastructure provider.

", "url": "https://www.partisia.com/" } ] }, "nonprofits": { "major": [ { "name": "International Association for Cryptologic Research", "rationale": "

Runs major cryptography venues and the ePrint archive. MPC and FHE foundations are heavily disseminated through IACR conferences and publications.

", "url": "https://iacr.org/" }, { "name": "HomomorphicEncryption.org", "rationale": "

Industry, government and academic consortium for homomorphic encryption standards and security guidelines. It is the key coordination body for applied HE standardization.

", "url": "https://homomorphicencryption.org/" }, { "name": "OpenDP", "rationale": "

Open-source differential privacy project from Harvard and partners. It provides vetted libraries and governance for practical DP deployments.

", "url": "https://opendp.org/" }, { "name": "OpenMined", "rationale": "

Open-source community for privacy-preserving AI and data science. Its PySyft ecosystem has been influential for federated and privacy-preserving ML education and prototyping.

", "url": "https://openmined.org/" }, { "name": "Privacy Enhancing Technologies Symposium", "rationale": "

Premier scholarly venue focused on privacy-enhancing technologies. It repeatedly hosts work across DP, MPC, federated learning and secure computation systems.

", "url": "https://www.petsymposium.org/" }, { "name": "OpenFHE", "rationale": "

Open-source FHE library project supporting major schemes such as BGV, BFV, CKKS and TFHE-style approaches. It is a central implementation substrate for applied encrypted computation.

", "url": "https://openfhe.org/" } ] }, "papers": { "major": [ { "name": "Protocols for Secure Computations", "rationale": "

Introduced secure two-party computation through protocols that let parties compute while hiding private inputs. It is the starting point for MPC and the broader secure computation agenda.

", "url": "https://doi.org/10.1109/SFCS.1982.38" }, { "name": "How to Play ANY Mental Game", "rationale": "

Generalized secure computation to arbitrary multiparty computations under cryptographic assumptions. It established the GMW paradigm that much of MPC still builds on.

", "url": "https://doi.org/10.1145/28395.28420" }, { "name": "Calibrating Noise to Sensitivity in Private Data Analysis", "rationale": "

Introduced differential privacy and the sensitivity-calibrated noise framework. It gave privacy-preserving data analysis a rigorous and composable definition.

", "url": "https://doi.org/10.1007/11681878_14" }, { "name": "Fully Homomorphic Encryption Using Ideal Lattices", "rationale": "

Presented the first plausible fully homomorphic encryption construction. It showed that arbitrary computation on encrypted data was possible in principle.

", "url": "https://doi.org/10.1145/1536414.1536440" }, { "name": "(Leveled) Fully Homomorphic Encryption Without Bootstrapping", "rationale": "

Introduced efficient leveled FHE from LWE and Ring-LWE without bootstrapping for bounded-depth circuits. It underlies influential BGV and BFV-style implementations.

", "url": "https://doi.org/10.1145/2633600" }, { "name": "Homomorphic Encryption for Arithmetic of Approximate Numbers", "rationale": "

Introduced CKKS, an approximate-number homomorphic encryption scheme for real and complex arithmetic. It became central for encrypted analytics and privacy-preserving machine learning.

", "url": "https://doi.org/10.1007/978-3-319-70694-8_15" }, { "name": "Communication-Efficient Learning of Deep Networks from Decentralized Data", "rationale": "

Introduced Federated Averaging and popularized federated learning for training shared models while data stays on devices. It is the canonical modern FL paper.

", "url": "https://proceedings.mlr.press/v54/mcmahan17a.html" }, { "name": "Practical Secure Aggregation for Privacy-Preserving Machine Learning", "rationale": "

Gave a practical secure aggregation protocol for federated learning. It lets a server sum client updates without seeing any individual participant's update.

", "url": "https://doi.org/10.1145/3133956.3133982" } ] }, "people": { "major": [ { "name": "Andrew C. Yao", "rationale": "

Founded secure two-party computation through the millionaires problem and garbled-circuit lineage. His work frames the core goal of computing together without revealing inputs.

", "url": "https://air.tsinghua.edu.cn/en/info/1044/1166.htm" }, { "name": "Oded Goldreich", "rationale": "

Coauthor of GMW and a central architect of secure computation theory. His work helped formalize simulation-based security for cryptographic protocols.

", "url": "https://www.wisdom.weizmann.ac.il/~oded/" }, { "name": "Silvio Micali", "rationale": "

Coauthor of GMW and a foundational cryptographer in secure protocols. His contributions helped establish general MPC as a basic cryptographic primitive.

", "url": "https://www.csail.mit.edu/person/silvio-micali" }, { "name": "Avi Wigderson", "rationale": "

Coauthor of GMW and a leading theoretical computer scientist. His work helped prove the broad feasibility of secure multiparty computation.

", "url": "https://www.ias.edu/scholars/wigderson" }, { "name": "Shafi Goldwasser", "rationale": "

Coauthor of foundational secure multiparty computation and simulation-based cryptography work. Her research shaped the theoretical basis for privacy-preserving protocols.

", "url": "https://people.csail.mit.edu/shafi/" }, { "name": "Cynthia Dwork", "rationale": "

Primary founder of differential privacy. Her work gave privacy-preserving computation a rigorous, composable privacy definition for statistical analysis.

", "url": "https://dwork.seas.harvard.edu/" }, { "name": "Craig Gentry", "rationale": "

Invented the first plausible fully homomorphic encryption scheme. His breakthrough made universal computation on encrypted data a serious research and deployment path.

", "url": "https://research.ibm.com/people/craig-gentry" }, { "name": "Vinod Vaikuntanathan", "rationale": "

Key contributor to second-generation lattice-based FHE, including BGV and related constructions. His work links modern lattice cryptography to practical encrypted computation.

", "url": "https://people.csail.mit.edu/vinodv/" } ] }, "tags": [ "compute", "infrastructure", "distributed systems" ] }, "Proof of Humanity": { "companies": { "major": [ { "name": "Tools for Humanity", "rationale": "

Company that initiated World and builds key infrastructure around World ID, the Orb, and World App. It is the most visible commercial actor scaling proof-of-human credentials using biometrics and zero-knowledge proofs.

", "url": "https://www.toolsforhumanity.com/" }, { "name": "Kleros Cooperative", "rationale": "

Builder of decentralized courts and a core steward of Proof of Humanity. Kleros matters because PoH relies on community challenges and adjudication rather than a conventional identity provider.

", "url": "https://kleros.io/" } ] }, "nonprofits": { "major": [ { "name": "World Foundation", "rationale": "

Independent foundation stewarding the World protocol and World ID ecosystem. It is central because World ID is the largest proof-of-human deployment and uses privacy-preserving proofs for anonymous verification.

", "url": "https://world.org/about" }, { "name": "Democracy Earth Foundation", "rationale": "

Nonprofit behind early digital democracy work and a launch partner for Proof of Humanity and UBI. It connects proof of personhood directly to one-person-one-vote governance and universal basic income.

", "url": "https://democracy.earth/" }, { "name": "Encointer Association", "rationale": "

Maintains Encointer’s local community currency and proof-of-personhood protocol. Its in-person ceremony model is a major alternative to biometric or government-ID approaches.

", "url": "https://encointer.org/" }, { "name": "BrightID", "rationale": "

Privacy-first social identity network for proof of uniqueness. It is important as a web-of-trust approach that avoids biometrics and centralized identity documents.

", "url": "https://www.brightid.org/" }, { "name": "Privacy & Scaling Explorations", "rationale": "

Ethereum Foundation applied cryptography lab behind Semaphore and related zero-knowledge tools. Its work supplies core privacy primitives for anonymous membership, signaling, and voting in proof-of-human systems.

", "url": "https://pse.dev/" }, { "name": "Holonym Foundation", "rationale": "

Nonprofit building human.tech, Human ID, and Human Passport infrastructure for zero-knowledge identity and Sybil resistance. It is a major current builder of privacy-preserving personhood tooling across web3 and public-sector use cases.

", "url": "https://holonym.id/" } ] }, "papers": { "major": [ { "name": "An Offline Foundation for Online Accountable Pseudonyms", "rationale": "

Foundational pseudonym-party paper, proposing in-person events plus technical sign-on so one real person gets one accountable pseudonym. It is an early blueprint for decentralized personhood without relying on government ID or a central identity provider.

", "url": "https://doi.org/10.1145/1435497.1435503" }, { "name": "Proof-of-Personhood: Redemocratizing Permissionless Cryptocurrencies", "rationale": "

Formalizes proof-of-personhood for permissionless cryptocurrencies and one-person-one-vote style consensus. It is one of the canonical papers connecting unique humanness, anonymity, Sybil resistance, UBI, and democratic governance.

", "url": "https://doi.org/10.1109/EuroSPW.2017.46" }, { "name": "Identity and Personhood in Digital Democracy: Evaluating Inclusion, Equality, Security, and Privacy in Pseudonym Parties and Other Proofs of Personhood", "rationale": "

Provides a systematic evaluation framework for proof-of-personhood systems across inclusion, equality, security, and privacy. It is central for comparing pseudonym parties, social verification, biometrics, and online approaches.

", "url": "https://arxiv.org/abs/2011.02412" }, { "name": "Who Watches the Watchmen? A Review of Subjective Approaches for Sybil-Resistance in Proof of Personhood Protocols", "rationale": "

Major review of subjective and community-mediated proof-of-personhood protocols such as vouching, voting, and adjudication. It helps define the design space and tradeoffs for decentralized human uniqueness systems.

", "url": "https://doi.org/10.3389/fbloc.2020.590171" }, { "name": "Semaphore: Zero-Knowledge Signaling on Ethereum", "rationale": "

Introduces a zero-knowledge primitive for proving group membership and preventing double signaling without revealing identity. It became a core building block for anonymous voting, World ID integrations, and privacy-preserving personhood proofs.

", "url": "https://docs.zkproof.org/pages/standards/accepted-workshop3/proposal-semaphore.pdf" }, { "name": "encointer -- Local Community Cryptocurrencies with Universal Basic Income", "rationale": "

Describes a deployed local-currency and UBI system based on concurrent randomized in-person key-signing ceremonies. It is an important non-biometric path for turning physical presence into proof of personhood.

", "url": "https://arxiv.org/abs/1912.12141" }, { "name": "Personhood Credentials: Artificial Intelligence and the Value of Privacy-Preserving Tools to Distinguish Who Is Real Online", "rationale": "

Frames privacy-preserving personhood credentials as infrastructure for distinguishing real people from AI agents online. It broadened proof-of-personhood from web3 governance into AI-era platform integrity and online trust.

", "url": "https://arxiv.org/abs/2408.07892" } ] }, "people": { "major": [ { "name": "Bryan Ford", "rationale": "

Key academic behind pseudonym-party and proof-of-personhood research. His work provides much of the conceptual foundation for privacy-preserving one-human-one-credential systems.

", "url": "https://bford.info/" }, { "name": "Vitalik Buterin", "rationale": "

Early and influential advocate for unique-human identity in crypto governance and public goods. His writing helped make proof of personhood a central problem for Ethereum, DAO voting, and anti-plutocratic coordination.

", "url": "https://vitalik.eth.limo/" }, { "name": "Clément Lesaege", "rationale": "

Kleros cofounder and a primary builder of Proof of Humanity. His work connects decentralized dispute resolution with human registries and adjudicated Sybil resistance.

", "url": "https://blog.kleros.io/author/clement/" }, { "name": "Santiago Siri", "rationale": "

Founder of Democracy Earth and a leading advocate of Proof of Humanity linked to UBI and democratic governance. He helped move proof of personhood from theory into public web3 experiments.

", "url": "https://www.weforum.org/people/santiago-siri/" }, { "name": "Alex Blania", "rationale": "

Co-founder and CEO of Tools for Humanity, the company that initiated World and supports World ID. He is central to the largest biometric, zero-knowledge proof-of-human deployment.

", "url": "https://world.org/about" }, { "name": "Divya Siddarth", "rationale": "

Coauthor of the major proof-of-personhood review and later personhood credentials work. Her research connects decentralized human verification to digital democracy, AI governance, and public-goods legitimacy.

", "url": "https://divyasiddarth.com/" }, { "name": "Alain Brenzikofer", "rationale": "

Lead author and builder behind Encointer’s proof-of-personhood approach. His work is important for local, in-person, non-state identity ceremonies tied to UBI-style community currencies.

", "url": "https://encointer.org/" } ] }, "tags": [ "law & governance", "infrastructure", "distributed systems" ] }, "Safety-Economics": { "companies": { "major": [ { "name": "Artificial Intelligence Underwriting Company (AIUC)", "rationale": "

Builds AI-agent certification and insurance, including the AIUC-1 standard. It is one of the clearest examples of pricing agent risk through audits, standards, and coverage.

", "url": "https://aiuc.com/" }, { "name": "Armilla AI", "rationale": "

Offers affirmative AI insurance and performance warranties for AI vendors and deployers. Its products explicitly cover underperformance, hallucinations, model drift, and agent mistakes.

", "url": "https://www.armilla.ai/" }, { "name": "Munich Re Insure AI / aiSure", "rationale": "

Large reinsurer with dedicated AI performance insurance products for providers and deployers. Its aiSure and aiSelf products are among the most mature examples of AI performance risk transfer.

", "url": "https://www.munichre.com/en/solutions/for-industry-clients/insure-ai/ai-self.html" }, { "name": "Testudo", "rationale": "

Standalone generative AI liability insurer backed by Lloyd's capacity. It uses litigation, incident, and exposure data to price and monitor AI liability risk.

", "url": "https://www.testudo.co/" }, { "name": "Relm Insurance", "rationale": "

Specialty insurer with AI liability products for AI platforms, developers, and adopters. It is relevant for affirmative AI coverage beyond legacy cyber or technology E&O policies.

", "url": "https://relminsurance.com/" }, { "name": "Mayflower Specialty", "rationale": "

AI-native specialty insurer writing D&O, EPL, E&O, and AI liability coverage for AI-related failures. It is emerging but directly focused on making AI operational risk insurable.

", "url": "https://mayflowerspecialty.com/" } ] }, "nonprofits": { "major": [ { "name": "METR", "rationale": "

Independent evaluator of frontier AI capabilities and risks. Its evaluation methods are important inputs for underwriting, pricing, and certification of risky agent deployments.

", "url": "https://metr.org/" }, { "name": "OWASP Foundation - Top 10 for Large Language Model Applications", "rationale": "

Maintains a widely used taxonomy of LLM application risks, including prompt injection and excessive agency. These categories are natural underwriting and audit inputs for AI-agent insurance.

", "url": "https://owasp.org/www-project-top-10-for-large-language-model-applications/" }, { "name": "Institute for Law & AI", "rationale": "

Research organization producing and convening work on AI liability, insurance, and market-based governance. It is a key intellectual hub for the legal side of safety-economics.

", "url": "https://law-ai.org/focus-area/liability-law/" } ] }, "papers": { "major": [ { "name": "Algorithmic Insurance", "rationale": "

Introduces a quantitative framework for pricing liability exposure from machine-learning decisions. It is foundational for turning model performance and design choices into insurable risk.

", "url": "https://arxiv.org/abs/2106.00839" }, { "name": "Insuring AI: The Role of Insurance in Artificial Intelligence Regulation", "rationale": "

Argues that insurance can function as an AI governance mechanism by incentivizing safer behavior and compensating victims. This is a core legal foundation for safety through risk pricing.

", "url": "https://jolt.law.harvard.edu/assets/articlePDFs/v35/2.-Lior-Insuring-AI.pdf" }, { "name": "Insurance of Agentic AI", "rationale": "

Frames agentic AI as a new insurance category with underwriting, pricing, reinsurance, telemetry, and product-design implications. It directly addresses insurance for autonomous planning and tool-use systems.

", "url": "https://arxiv.org/abs/2606.05449" }, { "name": "When Agent Automation Becomes Profitable: Quantifying and Insuring Autonomous AI Risk through Trace-Economic Underwriting", "rationale": "

Proposes trace-economic underwriting, mapping agent tool-use traces to customer exposure, claimable loss, pricing, and controls. It is one of the most direct technical treatments of pricing agent risk.

", "url": "https://arxiv.org/abs/2606.16465" }, { "name": "Instrument Choice in AI Governance: Liability as the Indispensable Core", "rationale": "

Defends liability as the central governance instrument for aligning AI developers and deployers with the risks they create. It gives the law-and-economics case for safety incentives without heavy ex ante policing.

", "url": "https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5283275" }, { "name": "The Limits of Regulating AI Safety Through Liability and Insurance: Lessons From Cybersecurity", "rationale": "

Challenges whether insurers can price or reduce many AI safety risks, using cyber insurance as the cautionary analogy. It is important because it maps the failure modes of the safety-economics theory of change.

", "url": "https://papers.ssrn.com/sol3/papers.cfm?abstract_id=5411062" }, { "name": "Insuring Algorithmic Operations: Liability Risk, Pricing, and Risk Control", "rationale": "

Develops an algorithmic operations liability taxonomy and pricing framework tied to model error, drift, governance gaps, and ecosystem externalities. It helps translate operational AI failures into actuarial controls.

", "url": "https://doi.org/10.3390/risks14020026" }, { "name": "Distributional AGI Safety", "rationale": "

Proposes virtual agentic sandbox economies governed by market mechanisms, auditability, reputation, and oversight. It is central for the decentralized version of safety-economics in multi-agent environments.

", "url": "https://arxiv.org/abs/2512.16856" } ] }, "people": { "major": [ { "name": "Anat Lior", "rationale": "

Author of the foundational legal article on insurance as an AI regulatory mechanism. Her work directly frames insurance as both compensation and private governance for AI harms.

", "url": "https://drexel.edu/law/faculty/fulltime_fac/Anat%20Lior/" }, { "name": "Gabriel Weil", "rationale": "

Legal scholar focused on AI liability, insurance, and catastrophic AI risk. His work is central to the argument that liability should align AI incentives with safety.

", "url": "https://law-ai.org/team/gabriel-weil-2/" }, { "name": "Quanyan Zhu", "rationale": "

Author of a dedicated framework for insurance of agentic AI. His broader work on cyber-physical security and game-theoretic risk supports technical underwriting of autonomous systems.

", "url": "https://engineering.nyu.edu/faculty/quanyan-zhu" }, { "name": "Dimitris Bertsimas", "rationale": "

Coauthor of the foundational algorithmic insurance framework. His optimization and analytics work helps connect model behavior, loss distributions, and pricing.

", "url": "https://mitsloan.mit.edu/faculty/directory/dimitris-bertsimas" }, { "name": "Agni Orfanoudaki", "rationale": "

Coauthor of the algorithmic insurance framework and an early researcher explicitly building foundations for this area. Her work links machine-learning performance to insurance contract evaluation.

", "url": "https://www.mit.edu/~agniorf/about.html" }, { "name": "Rune Kvist", "rationale": "

Co-founder of AIUC, a company building AI-agent standards, certification, audits, and insurance. He is a central builder of the agent-risk pricing stack.

", "url": "https://aiuc.com/team" }, { "name": "Rajiv Dattani", "rationale": "

AI evaluation and policy researcher involved in connecting safety evaluations with insurability and enterprise deployment. His work sits at the evaluation-to-underwriting interface.

", "url": "https://metr.org/team/rajiv-dattani/" }, { "name": "Karthik Ramakrishnan", "rationale": "

Founder and CEO of Armilla AI, one of the leading companies offering affirmative AI insurance and performance warranties. He is a key market builder for pricing AI underperformance.

", "url": "https://www.ivey.uwo.ca/about/leadership/ai-fellows/ramakrishnan-karthik/" } ] }, "tags": [ "economics", "safety", "law & governance" ] }, "Speculative Execution for Agentic AI": { "companies": { "major": [ { "name": "Microsoft Research", "rationale": "

Corporate research lab behind central work on PASTE and involved in related workflow speculation research such as Sherlock. It is the clearest company-affiliated center of activity in speculative agent execution.

", "url": "https://www.microsoft.com/en-us/research/" } ] }, "nonprofits": { "major": [] }, "papers": { "major": [ { "name": "Speculative Actions: A Lossless Framework for Faster Agentic Systems", "rationale": "

Introduces a general actor-speculator framework for predicting future agent actions and executing them in parallel while preserving as-if-sequential behavior. It is the broadest current formulation of agent-level speculative execution across tools, environments, and human responses.

", "url": "https://openreview.net/forum?id=P0GOk5wslg" }, { "name": "Act While Thinking: Accelerating LLM Agents via Pattern-Aware Speculative Tool Execution", "rationale": "

Defines PASTE, which mines recurring tool-call control flow and data dependencies to pre-launch likely tools while the LLM is still reasoning. It is one of the most directly relevant systems papers for hiding LLM-tool loop latency.

", "url": "https://arxiv.org/abs/2603.18897" }, { "name": "Dynamic Speculative Agent Planning", "rationale": "

Extends speculative planning with online reinforcement learning to choose how far to speculate under latency and cost constraints. It matters because it turns fixed draft-and-verify speculation into an adaptive runtime policy.

", "url": "https://arxiv.org/abs/2509.01920" }, { "name": "Interactive Speculative Planning: Enhance Agent Efficiency through Co-design of System and User Interface", "rationale": "

Applies speculative execution to LLM agent planning with a fast approximation agent and a stronger verifying target agent. It is an early ICLR paper that made speculative planning a concrete agent efficiency technique.

", "url": "https://openreview.net/forum?id=BwR8t91yqh" }, { "name": "Optimizing Agentic Language Model Inference via Speculative Tool Calls", "rationale": "

Studies client-side and inference-engine-side speculative tool calls, including a proposed tool cache API. It is important because it connects agent speculation to serving infrastructure and model residency.

", "url": "https://arxiv.org/abs/2512.15834" }, { "name": "Reducing Latency of LLM Search Agent via Speculation-based Algorithm-System Co-Design", "rationale": "

Introduces SPAgent for search agents, combining adaptive action-level speculation with speculation-aware scheduling. It is a strong domain-specific example of co-designing algorithms and serving systems for speculative agent execution.

", "url": "https://arxiv.org/abs/2511.20048" }, { "name": "SpecHop: Continuous Speculation for Accelerating Multi-Hop Retrieval Agents", "rationale": "

Develops continuous speculation for multi-hop retrieval agents, maintaining speculative threads and asynchronously verifying predicted observations. It is central for retrieval and deep-research style agents where tool waits dominate latency.

", "url": "https://arxiv.org/abs/2605.21965" }, { "name": "Speculative plan execution for information gathering", "rationale": "

A pre-LLM foundational paper on out-of-order speculative execution for information-gathering plans with remote I/O latency. It provides the clearest historical ancestor of today’s speculative tool and data-fetching agents.

", "url": "https://doi.org/10.1016/j.artint.2007.08.002" } ] }, "people": { "major": [ { "name": "Naimeng Ye", "rationale": "

First author of Speculative Actions, one of the central general frameworks for lossless acceleration of agentic systems. Included for direct authorship of a field-defining paper.

", "url": "https://business.columbia.edu/decision-risk-and-operations-division/people/naimeng-ye" }, { "name": "Kostis Kaffes", "rationale": "

Columbia systems researcher and coauthor of Speculative Actions. His work anchors the systems perspective on agentic serving and parallel environment interaction.

", "url": "https://www.cs.columbia.edu/~kkaffes/index.html" }, { "name": "Tianyi Peng", "rationale": "

Coauthor of Speculative Actions and a central member of the Columbia group studying data, agents, and processes. Included for direct work on lossless speculative agent execution.

", "url": "https://tianyipeng.github.io/" }, { "name": "Yilin Guan", "rationale": "

Lead author of Dynamic Speculative Agent Planning. Included for advancing adaptive, cost-aware speculation policies for LLM agent planning.

", "url": "https://guanyilin428.github.io/" }, { "name": "Wenyue Hua", "rationale": "

Key author across Interactive Speculative Planning and Dynamic Speculative Agent Planning. Included because their work helped establish draft-and-verify planning as an agent latency-reduction pattern.

", "url": "https://arxiv.org/abs/2509.01920" }, { "name": "Yifan Sui", "rationale": "

Lead author of PASTE, a central paper on pattern-aware speculative tool execution. Included for work that directly targets the serial LLM-tool loop.

", "url": "https://www.microsoft.com/en-us/research/publication/act-while-thinking-accelerating-llm-agents-via-pattern-aware-speculative-tool-execution/" }, { "name": "Han Zhao", "rationale": "

Senior coauthor of PASTE and contributor to Microsoft’s work on efficient agent systems. Included for direct involvement in production-oriented speculative tool execution.

", "url": "https://www.microsoft.com/en-us/research/publication/act-while-thinking-accelerating-llm-agents-via-pattern-aware-speculative-tool-execution/" }, { "name": "Greg Barish", "rationale": "

Lead author of the early USC ISI work on speculative execution for information agents and information-gathering plans. Included for foundational pre-LLM contributions to agent speculation.

", "url": "https://dblp.org/pid/48/6271" } ] }, "tags": [ "infrastructure", "compute" ] }, "Supercollaboration": { "companies": { "major": [ { "name": "Unanimous AI", "rationale": "

Builds Swarm AI, Thinkscape, Mindmix, and related platforms for amplifying group intelligence. It is one of the most direct commercial efforts toward AI-enabled collective superintelligence.

", "url": "https://unanimous.ai/" }, { "name": "Remesh", "rationale": "

Runs AI-powered live conversations with large groups of customers or employees and surfaces consensus and segment insights in real time. It is a deployed platform for scalable dialogue and collective sensing.

", "url": "https://www.remesh.ai/" }, { "name": "Vocean", "rationale": "

Provides an AI-powered co-creation and innovation platform used for workshops, strategy, citizen dialogue, and large-scale participation. It is relevant as an applied tool for turning broad input into shared decisions.

", "url": "https://www.vocean.com/" }, { "name": "ThoughtExchange", "rationale": "

Offers an AI-assisted engagement platform that gathers, rates, and synthesizes perspectives from large groups. It is a notable commercial tool for scaling human feedback into organizational decisions.

", "url": "https://thoughtexchange.com/" } ] }, "nonprofits": { "major": [ { "name": "MIT Center for Collective Intelligence", "rationale": "

Foundational research center for studying how people and computers can act more intelligently together. It supplies much of the conceptual vocabulary behind supercollaboration.

", "url": "https://cci.mit.edu/" }, { "name": "The Computational Democracy Project", "rationale": "

Stewards Polis, an open-source system for gathering and analyzing what large groups think in their own words. It is central to scalable deliberation and AI-augmented public input.

", "url": "https://compdemocracy.org/" }, { "name": "The GovLab", "rationale": "

Develops Smarter Crowdsourcing, CrowdLaw, and applied methods for using collective intelligence in public problem solving. Its Policy Synth work directly combines AI with expert and public participation.

", "url": "https://thegovlab.org/" }, { "name": "Collective Intelligence Project", "rationale": "

Runs scalable experiments for democratic AI, alignment assemblies, global dialogues, and collective input into AI development. It is one of the most central nonprofits connecting AI governance with collective intelligence.

", "url": "https://www.cip.org/" }, { "name": "Nesta Centre for Collective Intelligence", "rationale": "

Builds practical tools, methods, and playbooks for collective intelligence, including deliberation work around AI assurance. It is a leading applied institution for collective intelligence design.

", "url": "https://www.nesta.org.uk/project/centre-collective-intelligence/" }, { "name": "Citizens Foundation", "rationale": "

Maintains open-source participation platforms such as Your Priorities and Policy Synth. Its work is directly relevant to AI-assisted public decision-making and civic collaboration at scale.

", "url": "https://www.citizens.is/" } ] }, "papers": { "major": [ { "name": "How large language models can reshape collective intelligence", "rationale": "

Frames LLMs as both products and tools of collective intelligence, with benefits and risks for group collaboration, decision-making, and deliberation. It is one of the clearest recent field-mapping papers for AI-mediated collective work.

", "url": "https://www.nature.com/articles/s41562-024-01959-9" }, { "name": "AI can help humans find common ground in democratic deliberation", "rationale": "

Introduces and evaluates the Habermas Machine, an LLM mediator that helps groups synthesize mutually acceptable statements. It is a landmark empirical demonstration of AI-assisted consensus formation.

", "url": "https://doi.org/10.1126/science.adq2852" }, { "name": "Opportunities and Risks of LLMs for Scalable Deliberation with Polis", "rationale": "

Studies how LLMs can facilitate, moderate, and summarize Polis conversations. It directly addresses AI augmentation of large-scale deliberation and collective meaning-making.

", "url": "https://arxiv.org/abs/2306.11932" }, { "name": "Using Artificial Intelligence to Accelerate Collective Intelligence: Policy Synth and Smarter Crowdsourcing", "rationale": "

Presents Policy Synth, a human-in-the-loop AI agent toolkit for scaling public problem solving. It is important because it connects AI orchestration with real collective policy design workflows.

", "url": "https://arxiv.org/abs/2407.13960" }, { "name": "Towards Collective Superintelligence: A Pilot Study", "rationale": "

Describes Conversational Swarm Intelligence, a method for scaling deliberation across many small groups using LLM observers. It is central to the idea that networked humans plus AI can form higher-capability collectives.

", "url": "https://arxiv.org/abs/2311.00728" }, { "name": "Flash Organizations: Crowdsourcing Complex Work by Structuring Crowds as Organizations", "rationale": "

Shows how on-demand expert crowds can be structured into temporary organizations for complex projects. It is foundational for supercollaboration because it turns loose networks into productive, adaptive teams.

", "url": "https://doi.org/10.1145/3025453.3025811" }, { "name": "The Collective Intelligence Genome", "rationale": "

Provides a design-pattern framework for systems that harness crowd intelligence. It remains a core conceptual foundation for designing AI-assisted collaboration mechanisms.

", "url": "https://doi.org/10.1109/EMR.2010.5559142" }, { "name": "CrowdForge: Crowdsourcing Complex Work", "rationale": "

Introduces a framework for decomposing and recombining complex work across crowds. It is a key precursor to AI systems that coordinate many contributors on nontrivial tasks.

", "url": "https://doi.org/10.1145/2047196.2047202" } ] }, "people": { "major": [ { "name": "Thomas W. Malone", "rationale": "

Founder of the MIT Center for Collective Intelligence and author of major frameworks for superminds and crowd-based organization. His work anchors the area’s theory of people and computers acting more intelligently together.

", "url": "https://cci.mit.edu/malone/" }, { "name": "Michael S. Bernstein", "rationale": "

Stanford HCI researcher behind flash teams, flash organizations, and computationally mediated crowd work. His work is central to turning distributed people into productive organizations.

", "url": "https://hci.stanford.edu/msb/" }, { "name": "Beth Simone Noveck", "rationale": "

Founder of The GovLab and a leading figure in Smarter Crowdsourcing, CrowdLaw, and AI-assisted public problem solving. She is a major builder and theorist of collective intelligence for governance.

", "url": "https://bethnoveck.medium.com/about" }, { "name": "Colin Megill", "rationale": "

Co-founder of Polis and a key figure in computational democracy. His work directly addresses large-scale opinion mapping, bridge-building, and AI-augmented deliberation.

", "url": "https://colinmegill.com/" }, { "name": "Louis Rosenberg", "rationale": "

Founder of Unanimous AI and a principal developer of artificial swarm intelligence and conversational swarm intelligence. He is central to the collective superintelligence branch of the field.

", "url": "https://unanimous.ai/about-us/" }, { "name": "Róbert Bjarnason", "rationale": "

Co-founder and president of Citizens Foundation, which builds Your Priorities and Policy Synth. His work combines open-source participation platforms with AI agents for public problem solving.

", "url": "https://democracy-technologies.org/ai-data/citizens-foundation/" }, { "name": "Audrey Tang", "rationale": "

Prominent practitioner of digital democracy through g0v, vTaiwan, Polis, and plurality. Tang is important because their work demonstrates state-scale coordination using civic technology and deliberative infrastructure.

", "url": "https://audreytang.org/" }, { "name": "Divya Siddarth", "rationale": "

Co-founder and executive director of the Collective Intelligence Project. She leads experiments in collective input, democratic AI, and governance models for transformative technologies.

", "url": "https://divyasiddarth.com/" } ] }, "tags": [ "law & governance", "infrastructure" ] }, "Superconsensus": { "companies": { "major": [ { "name": "Google DeepMind", "rationale": "

Developed the Habermas Machine and earlier Democratic AI work. It is the leading frontier AI lab producing empirical research on AI-assisted consensus mechanisms.

", "url": "https://deepmind.google/" }, { "name": "Remesh", "rationale": "

Builds AI-powered collective dialogue systems for large-scale conversations. It is one of the strongest commercial examples of real-time group sensemaking beyond standard surveys.

", "url": "https://www.remesh.ai/" }, { "name": "Gitcoin", "rationale": "

Deployed quadratic funding and later plural variants for web3 public-goods funding. It is one of the most important live laboratories for decentralized preference aggregation.

", "url": "https://gitcoin.co/" }, { "name": "Unanimous AI", "rationale": "

Builds Swarm AI and related systems that connect human groups into real-time collective intelligence processes. It is a distinct AI-augmented approach to group decision support.

", "url": "https://unanimous.ai/" } ] }, "nonprofits": { "major": [ { "name": "The Computational Democracy Project", "rationale": "

Stewards Polis, an open-source system for gathering, mapping, and understanding what large groups think in their own words. It is the core nonprofit infrastructure provider for this area.

", "url": "https://compdemocracy.org/" }, { "name": "The Collective Intelligence Project", "rationale": "

Runs alignment assemblies, global dialogues, community models, and public-input work for AI governance. It is one of the most central organizations translating deliberation into AI alignment practice.

", "url": "https://www.cip.org/" }, { "name": "RadicalxChange Foundation", "rationale": "

Institutional home for quadratic voting, quadratic funding, and plural governance ideas. Its mechanisms strongly influence decentralized and AI-governance experiments.

", "url": "https://www.radicalxchange.org/" }, { "name": "Metagov", "rationale": "

A digital-governance lab working on DAO standards, interoperable deliberative tools, and public AI. It is important for connecting governance research to online communities and decentralized organizations.

", "url": "https://metagov.org/" }, { "name": "Stanford Deliberative Democracy Lab", "rationale": "

Develops deliberative polling and deliberative democracy methods. Its work provides the high-legitimacy deliberation model that many AI-augmented systems attempt to scale.

", "url": "https://deliberation.stanford.edu/" }, { "name": "g0v", "rationale": "

Taiwanese civic-tech community behind many open digital democracy practices, including the vTaiwan ecosystem. It is a key proof point for open-source, participatory governance at national scale.

", "url": "https://g0v.tw/" }, { "name": "AI Objectives Institute", "rationale": "

Nonprofit R&D lab building AI tools for coordination and deliberation, including Talk to the City. Its work is directly focused on scalable cooperation and democratic input for AI-era institutions.

", "url": "https://ai.objectives.institute/" }, { "name": "Plurality Institute", "rationale": "

Advances research and deployment of technologies that upgrade democracy and cooperation at scale. It is a hub for the plural technology ecosystem around Weyl, Tang, and related governance work.

", "url": "https://plurality.institute/" } ] }, "papers": { "major": [ { "name": "AI can help humans find common ground in democratic deliberation", "rationale": "

Introduces the Habermas Machine, an LLM mediator that drafts group statements from participants' views and critiques. It is the strongest empirical result for AI-assisted common-ground discovery rather than vote tallying.

", "url": "https://doi.org/10.1126/science.adq2852" }, { "name": "Collective Constitutional AI: Aligning a Language Model with Public Input", "rationale": "

Shows a public-input pipeline for turning deliberation into an AI constitution and then into model behavior. It is a central proof of concept for democratic AI policy setting.

", "url": "https://arxiv.org/abs/2406.07814" }, { "name": "Opportunities and Risks of LLMs for Scalable Deliberation with Polis", "rationale": "

Maps how LLMs can augment Polis through summarization, moderation, comment routing, vote prediction, and consensus discovery. It is a key technical bridge between existing computational democracy tools and modern AI.

", "url": "https://arxiv.org/abs/2306.11932" }, { "name": "'Generative CI' through Collective Response Systems", "rationale": "

Defines collective response systems as a form of generative collective intelligence where both the answers and the options emerge from the group. It gives the field a vocabulary for consensus processes beyond polls and referenda.

", "url": "https://arxiv.org/abs/2302.00672" }, { "name": "Democratic Policy Development using Collective Dialogues and AI", "rationale": "

Presents an end-to-end process combining AI-enabled collective dialogue, bridging-based ranking, and GPT-assisted policy drafting. It is directly about turning large-scale public input into concrete AI policy.

", "url": "https://arxiv.org/abs/2311.02242" }, { "name": "Bridging Systems: Open Problems for Countering Destructive Divisiveness across Ranking, Recommenders, and Governance", "rationale": "

Frames bridging-based ranking as a design pattern for surfacing agreement across divides. It is foundational for consensus mechanisms that optimize for cross-group legitimacy rather than majority preference.

", "url": "https://arxiv.org/abs/2301.09976" }, { "name": "Policy Aggregation", "rationale": "

Formalizes multi-person AI alignment as a social-choice problem over policies in Markov decision processes. It is important for translating plural human preferences into AI behavior.

", "url": "https://arxiv.org/abs/2411.03651" }, { "name": "Liberal Radicalism: A Flexible Design for Philanthropic Matching Funds", "rationale": "

Introduces quadratic funding as a mechanism for decentralized public-goods allocation. Its deployment in web3 made it one of the most influential alternatives to simple token or majority voting.

", "url": "https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3243656" } ] }, "people": { "major": [ { "name": "Colin Megill", "rationale": "

Co-founder of Polis and board president of the Computational Democracy Project. His work on machine-learning-assisted opinion mapping is core infrastructure for this area.

", "url": "https://colinmegill.com/" }, { "name": "Audrey Tang", "rationale": "

Key public-sector champion of vTaiwan, g0v, Polis, and plurality-oriented digital democracy. Tang made computational deliberation a globally visible governance practice.

", "url": "https://audreyt.org/" }, { "name": "Aviv Ovadya", "rationale": "

Developed major concepts around collective response systems, bridging systems, and deliberative technology for alignment. His work connects consensus tooling to AI governance.

", "url": "https://aviv.me/" }, { "name": "Michael Henry Tessler", "rationale": "

Lead author on the Habermas Machine work at Google DeepMind. His research is central to LLM-mediated common-ground generation.

", "url": "https://www.mit.edu/~tessler/" }, { "name": "Saffron Huang", "rationale": "

Coauthor of Collective Constitutional AI and a leading builder in democratic AI and collective intelligence. Her work links public input processes to AI model behavior.

", "url": "https://www.saffronhuang.com/" }, { "name": "Divya Siddarth", "rationale": "

Co-founder of the Collective Intelligence Project and a central advocate for democratic and pluralistic AI governance. She has helped move alignment assemblies and public input methods into frontier AI debates.

", "url": "https://www.divyasiddarth.com/" }, { "name": "E. Glen Weyl", "rationale": "

Major architect of quadratic voting, quadratic funding, RadicalxChange, and the plurality technology agenda. His work supplies many of the mechanism-design primitives for decentralized consensus.

", "url": "https://www.microsoft.com/en-us/research/people/glenweyl" }, { "name": "Ariel D. Procaccia", "rationale": "

Leading computational social choice researcher working on policy aggregation and formal common-ground selection. His work gives mathematical grounding to plural AI alignment.

", "url": "https://procaccia.info/" } ] }, "tags": [ "law & governance", "oversight", "alignment" ] }, "Sybil Resistance for Agents": { "companies": { "major": [ { "name": "MetaMask (Consensys)", "rationale": "

Co-proposed ERC-8004 and builds agent wallet infrastructure. It is one of the most important commercial actors shaping on-chain agent identity and reputation.

", "url": "https://metamask.io/" }, { "name": "Coinbase Developer Platform", "rationale": "

Leads x402 payment rails and discovery tools for agentic commerce. Its role in ERC-8004 and payment-graph service discovery makes it central to agent routing trust.

", "url": "https://www.coinbase.com/developer-platform/products/x402" }, { "name": "8004 Labs (Reputio)", "rationale": "

Builds Reputio, a live ERC-8004 trust infrastructure layer with identity, reputation, validation, and proof-of-work records. It is directly focused on making agents inspectable before use.

", "url": "https://www.8004labs.ai/" }, { "name": "RNWY", "rationale": "

Builds AI agent trust intelligence, reviewer wallet analysis, and Sybil detection across ERC-8004, Olas, Virtuals, and Solana agent registries. It is one of the most direct deployed products in this area.

", "url": "https://rnwy.com/" }, { "name": "Valory / Olas", "rationale": "

Operates the Olas ecosystem for autonomous services and on-chain agent coordination. Its agent registries and service activity create a major target environment for reputation and Sybil analysis.

", "url": "https://olas.network/" } ] }, "nonprofits": { "major": [ { "name": "Ethereum Foundation", "rationale": "

Stewards Ethereum and the dAI work that introduced ERC-8004 as a public trust layer for autonomous AI agents. Its standards role makes it central to open agent Sybil resistance.

", "url": "https://ethereum.foundation/" }, { "name": "Gitcoin Foundation / Human Passport", "rationale": "

Human Passport provides proof-of-personhood and Sybil resistance tooling, originally shaped by Gitcoin Grants anti-Sybil needs. It is important when agent systems need to bind influence to unique humans or operators.

", "url": "https://passport.human.tech/" }, { "name": "Kleros Cooperative / Proof of Humanity", "rationale": "

Proof of Humanity is a Sybil-resistant human registry using social verification and dispute resolution. It is a key reference implementation for subjective uniqueness checks.

", "url": "https://poh.id/" }, { "name": "World Foundation", "rationale": "

Stewards the World ecosystem and World ID proof-of-personhood protocol. Its scale makes it one of the most important nonprofit actors for human uniqueness in AI-heavy environments.

", "url": "https://foundation.world.org/about" } ] }, "papers": { "major": [ { "name": "The Sybil Attack", "rationale": "

Foundational paper that named and formalized the risk of one entity presenting many identities. Its impossibility result is the baseline threat model for agent reputation, voting, and routing systems.

", "url": "https://doi.org/10.1007/3-540-45748-8_24" }, { "name": "The EigenTrust Algorithm for Reputation Management in P2P Networks", "rationale": "

Introduces global trust propagation from local interaction histories. It remains a core reference for decentralized reputation ranking under collusion and malicious peers.

", "url": "https://doi.org/10.1145/775152.775242" }, { "name": "Sybilproof Reputation Mechanisms", "rationale": "

Formalizes when reputation functions can resist fake identities. It is especially relevant to agent systems where operators can manufacture links, reviews, or endorsements.

", "url": "https://doi.org/10.1145/1080192.1080202" }, { "name": "Sybil-proof Accounting Mechanisms with Transitive Trust", "rationale": "

Analyzes reputation and accounting mechanisms for distributed work systems with cheap Sybil creation. It clarifies tradeoffs between transitive trust and Sybil-proofness for agent work networks.

", "url": "https://econcs.seas.harvard.edu/publications/sybil-proof-accounting-mechanisms-transitive-trust" }, { "name": "Achieving Sybil-Proofness in Distributed Work Systems", "rationale": "

Directly studies multi-agent work systems where agents exchange quantified work and can create fake identities. It extends prior impossibility results and identifies requirements for Sybil-proof reputation mechanisms.

", "url": "https://aamas.csc.liv.ac.uk/Proceedings/aamas2021/pdfs/p1263.pdf" }, { "name": "ERC-8004: Trustless Agents", "rationale": "

Protocol specification for on-chain agent Identity, Reputation, and Validation registries. It is a central current standard for decentralized AI agent discovery and trust.

", "url": "https://eips.ethereum.org/EIPS/eip-8004" }, { "name": "Sybil-Resistant Service Discovery for Agent Economies", "rationale": "

Introduces TraceRank, a payment-graph reputation method for ranking services used by agents. It targets the exact problem of preventing fake low-quality services from capturing routing weight.

", "url": "https://arxiv.org/abs/2510.27554" }, { "name": "Can Trustless Agents Be Trusted? An Empirical Study of the ERC-8004 Decentralized AI Agent Ecosystem", "rationale": "

Empirically measures ERC-8004 identity and reputation data across Ethereum, BNB Smart Chain, and Base. It is important because it tests whether the current agent trust layer is vulnerable to coordinated Sybil feedback.

", "url": "https://arxiv.org/abs/2606.26028" } ] }, "people": { "major": [ { "name": "John R. Douceur", "rationale": "

Author of The Sybil Attack. His impossibility result is the conceptual foundation for why agent identity and reputation systems need explicit Sybil defenses.

", "url": "https://www.microsoft.com/en-us/research/people/johndo/" }, { "name": "Haifeng Yu", "rationale": "

Lead author on SybilGuard and SybilLimit. His work established social-network-based defenses against mass fake identities.

", "url": "https://www.comp.nus.edu.sg/~yuhf/" }, { "name": "Sepandar D. Kamvar", "rationale": "

Co-author of EigenTrust. His work on decentralized trust propagation is a major ancestor of agent reputation ranking.

", "url": "https://sepandar.com/" }, { "name": "Eric J. Friedman", "rationale": "

Co-author of Sybilproof Reputation Mechanisms and broader work on manipulation-resistant reputation. His results are directly relevant to fake-review and fake-agent attacks.

", "url": "https://infosci.cornell.edu/content/friedman" }, { "name": "Sven Seuken", "rationale": "

Co-author of major work on Sybil-proof accounting mechanisms with transitive trust. His research connects mechanism design, reputation, and distributed work systems.

", "url": "https://www.ifi.uzh.ch/en/ce/people/seuken.html" }, { "name": "David C. Parkes", "rationale": "

Co-author of Sybil-proof accounting work for distributed work systems. His mechanism-design perspective is central to incentive-compatible agent reputation.

", "url": "https://parkes.seas.harvard.edu/" }, { "name": "Davide Crapis", "rationale": "

Ethereum Foundation co-author of ERC-8004. He is central to current Ethereum work on open identity, reputation, and validation layers for AI agents.

", "url": "https://eips.ethereum.org/EIPS/eip-8004" }, { "name": "Marco De Rossi", "rationale": "

MetaMask AI lead and ERC-8004 co-author. He has helped frame trustless agent identity and reputation as infrastructure for open agent economies.

", "url": "https://metamask.io/news/self-custody-in-the-era-of-agents" } ] }, "tags": [ "multi-agent systems", "infrastructure", "economics" ] }, "Trust & Reputation in Agentic AI": { "companies": { "major": [ { "name": "Consensys / MetaMask", "rationale": "

MetaMask is a core proponent and co-authoring organization behind ERC-8004. Its wallet, delegation, and onchain identity work is directly tied to portable agent reputation.

", "url": "https://metamask.io/" }, { "name": "QuickNode", "rationale": "

Builds a public ERC-8004 explorer, API, and scoring surface for agent registrations, feedback, reputation, and validations. This makes decentralized agent reputation auditable and queryable.

", "url": "https://erc-8004.quicknode.com/" }, { "name": "Chitin Protocol", "rationale": "

Provides ERC-8004 passports and identity tooling for AI agents. It is directly focused on verifiable agent identity with reputation data attached.

", "url": "https://chitin.id/" }, { "name": "SingularityNET", "rationale": "

One of the earliest decentralized AI marketplaces to explicitly include agent ratings and reputation in its platform design. It remains a major precursor for decentralized AI service reputation.

", "url": "https://singularitynet.io/" }, { "name": "Coinbase", "rationale": "

Co-authoring organization behind ERC-8004 and creator of x402-related agent payment infrastructure. Payment receipts and economic interaction histories are important inputs for agent reputation.

", "url": "https://www.coinbase.com/developer-platform" } ] }, "nonprofits": { "major": [ { "name": "Ethereum Foundation", "rationale": "

Home of the dAI team and a co-authoring organization behind ERC-8004. It is central to the most active decentralized agent identity, reputation, and validation standard.

", "url": "https://ethereum.foundation/" }, { "name": "World Wide Web Consortium", "rationale": "

Maintains core identity standards such as Decentralized Identifiers and Verifiable Credentials. These standards are key building blocks for portable agent identity and reputation evidence.

", "url": "https://www.w3.org/" }, { "name": "Decentralized Identity Foundation", "rationale": "

Coordinates decentralized identity specifications and interoperability work. Its outputs support agent identities that can carry trust and reputation across platforms.

", "url": "https://identity.foundation/" }, { "name": "Trust Over IP Foundation", "rationale": "

Develops governance and architecture for decentralized digital trust ecosystems. Its layered model is relevant for verifiable credentials, trust registries, and reputation governance.

", "url": "https://trustoverip.org/" } ] }, "papers": { "major": [ { "name": "Formalising Trust as a Computational Concept", "rationale": "

Foundational 1994 thesis that made trust an implementable computational concept for artificial agents. It frames many later agent trust and reputation models.

", "url": "https://dspace.stir.ac.uk/bitstream/1893/2010/1/Formalising%20trust%20as%20a%20computational%20concept.pdf" }, { "name": "REGRET: Reputation in Gregarious Societies", "rationale": "

Introduced a social, context-sensitive reputation model for open agent societies. It is one of the canonical starting points for computational reputation in multi-agent systems.

", "url": "https://doi.org/10.1145/375735.376110" }, { "name": "The Beta Reputation System", "rationale": "

Defines a simple Bayesian reputation mechanism using beta distributions over positive and negative feedback. Its statistical treatment became a widely reused basis for reputation scoring.

", "url": "https://aisel.aisnet.org/bled2002/41/" }, { "name": "An Evidential Model of Distributed Reputation Management", "rationale": "

Models decentralized reputation propagation among agents using evidence theory. It directly addresses the no central authority setting in large open agent networks.

", "url": "https://www.csc2.ncsu.edu/faculty/mpsingh/papers/mas/aamas-02-trust.pdf" }, { "name": "FIRE: An Integrated Trust and Reputation Model for Open Multi-Agent Systems", "rationale": "

Combines direct experience, witness reputation, role-based trust, and certified reputation. It is a central model for trust decisions when agents enter and leave open systems.

", "url": "https://eprints.soton.ac.uk/262593/" }, { "name": "TRAVOS: Trust and Reputation in the Context of Inaccurate Information Sources", "rationale": "

Introduces a probabilistic trust and reputation model that reasons about unreliable or deceptive information sources. This is crucial for agent economies where feedback itself can be adversarial.

", "url": "https://doi.org/10.1007/s10458-006-5952-x" }, { "name": "Computational Trust and Reputation Models for Open Multi-Agent Systems: A Review", "rationale": "

A major review of computational trust and reputation models for open MAS. It is a useful map of the classic design space behind current agentic AI reputation work.

", "url": "https://doi.org/10.1007/s10462-011-9277-z" }, { "name": "ERC-8004: Trustless Agents", "rationale": "

The central current specification for onchain AI agent identity, reputation, and validation registries. It translates classic open-agent trust problems into a decentralized protocol for agent economies.

", "url": "https://eips.ethereum.org/EIPS/eip-8004" } ] }, "people": { "major": [ { "name": "Stephen Marsh", "rationale": "

Pioneered computational trust with a formal model that could be embedded in artificial agents. His work is a conceptual foundation for agent trust systems.

", "url": "https://ontariotechu.ca/experts/fbit/stephen-marsh.php" }, { "name": "Munindar P. Singh", "rationale": "

Major multi-agent systems researcher with influential work on distributed reputation, trust aggregation, and trustworthy sociotechnical AI. His papers directly address trust without central authorities.

", "url": "https://www.csc2.ncsu.edu/faculty/mpsingh/mps/" }, { "name": "Jordi Sabater-Mir", "rationale": "

Co-creator of REGRET and a leading researcher on computational trust and reputation in multi-agent systems. His work is among the most directly relevant classic foundations.

", "url": "https://www.iiia.csic.es/~jsabater/" }, { "name": "Carles Sierra", "rationale": "

Co-creator of REGRET and a major figure in autonomous agents, electronic institutions, and agreement technologies. His work links reputation to open agent societies.

", "url": "https://www.iiia.csic.es/~sierra/" }, { "name": "Audun Jøsang", "rationale": "

Developed influential probabilistic and subjective-logic approaches to trust and reputation, including the Beta Reputation System. His models remain central to computational reputation design.

", "url": "https://www.mn.uio.no/ifi/english/people/aca/josang/" }, { "name": "Nicholas R. Jennings", "rationale": "

Leading multi-agent systems researcher and co-author of FIRE and TRAVOS-related work. His research helped establish trust and reputation as core open-MAS infrastructure.

", "url": "https://profiles.imperial.ac.uk/n.jennings" }, { "name": "Marco De Rossi", "rationale": "

MetaMask AI lead and co-author of ERC-8004. He is a central builder in translating decentralized agent identity and reputation into deployed Ethereum tooling.

", "url": "https://metamask.io/news/self-custody-in-the-era-of-agents" }, { "name": "Davide Crapis", "rationale": "

Head of AI at the Ethereum Foundation and co-author of ERC-8004. He is driving Ethereum’s agent identity, reputation, and validation work through the dAI team.

", "url": "https://ai.ethereum.foundation/team" } ] }, "tags": [ "multi-agent systems", "economics", "infrastructure" ] }, "Universal Interoperability": { "companies": { "major": [ { "name": "Anthropic", "rationale": "

Created and open-sourced the Model Context Protocol, then donated it to neutral foundation governance. Anthropic is central because MCP has become a key common layer for tools, data, and context.

", "url": "https://www.anthropic.com/research/model-context-protocol" }, { "name": "Google", "rationale": "

Created Agent2Agent and donated it to the Linux Foundation. Google is central because A2A directly targets cross-vendor agent communication and collaboration.

", "url": "https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/" }, { "name": "Microsoft", "rationale": "

Introduced NLWeb and invested broadly in MCP support, registry work, and the open agentic web. Microsoft is important for bringing interoperability protocols into web, developer, and enterprise platforms.

", "url": "https://news.microsoft.com/source/features/company-news/introducing-nlweb-bringing-conversational-interfaces-directly-to-the-web/" }, { "name": "IBM Research", "rationale": "

Developed ACP and BeeAI as an interoperable agent communication and deployment ecosystem. IBM is important for REST-native, framework-independent agent messaging.

", "url": "https://research.ibm.com/projects/agent-communication-protocol" }, { "name": "Cisco", "rationale": "

Initially open sourced AGNTCY through Outshift and donated it to Linux Foundation governance. Cisco is important for agent discovery, identity, messaging, and observability infrastructure.

", "url": "https://www.linuxfoundation.org/press/linux-foundation-welcomes-the-agntcy-project-to-standardize-open-multi-agent-system-infrastructure-and-break-down-ai-agent-silos" } ] }, "nonprofits": { "major": [ { "name": "Linux Foundation", "rationale": "

Provides neutral governance for major open agent interoperability efforts, including A2A, AGNTCY, and the Agentic AI Foundation. It is the central institutional home for several competing but complementary standards.

", "url": "https://www.linuxfoundation.org/" }, { "name": "Agentic AI Foundation", "rationale": "

A Linux Foundation directed fund created to steward open agentic AI projects such as MCP, AGENTS.md, and goose. It matters because neutral governance reduces platform lock-in risks.

", "url": "https://aaif.io/" }, { "name": "World Wide Web Consortium (W3C)", "rationale": "

Standards body behind core web, semantic web, RDF, and DID standards. Its work provides much of the substrate for universal, decentralized, machine-readable interoperability.

", "url": "https://www.w3.org/" }, { "name": "Ecma International TC56", "rationale": "

Standards committee responsible for the Natural Language Interaction Protocol suite. It is important because NLIP explicitly targets secure, multimodal, cross-domain AI agent communication.

", "url": "https://ecma-international.org/technical-committees/tc56/" }, { "name": "AGNTCY Project", "rationale": "

Linux Foundation project building an open stack for the Internet of Agents, including discovery, identity, messaging, and observability. It is central to infrastructure beyond single message protocols.

", "url": "https://agntcy.org/" } ] }, "papers": { "major": [ { "name": "Model Context Protocol Specification", "rationale": "

Defines MCP, the leading standard for connecting AI applications to external tools, data sources, prompts, and resources. It matters because it turns one-off integrations into a shared AI-native interface.

", "url": "https://modelcontextprotocol.io/specification" }, { "name": "Agent2Agent (A2A) Protocol Specification", "rationale": "

Specifies how independent agents discover capabilities, exchange messages, manage tasks, and negotiate modalities. It is one of the central current protocols for cross-vendor agent-to-agent interoperability.

", "url": "https://a2a-protocol.org/latest/specification/" }, { "name": "Natural Language Interaction Protocol (ECMA-430)", "rationale": "

Standardizes a multimodal message envelope for communication between AI agents and between humans and agents. It is important because it targets cross-domain interoperability without requiring bespoke APIs.

", "url": "https://ecma-international.org/wp-content/uploads/ECMA-430_1st_edition_december_2025.pdf" }, { "name": "Agent Network Protocol Technical White Paper", "rationale": "

Proposes ANP, a three-layer agentic web protocol for identity, encrypted communication, protocol negotiation, discovery, and application interaction. It is a notable decentralized approach to universal agent interconnection.

", "url": "https://arxiv.org/abs/2508.00007" }, { "name": "A Survey of Agent Interoperability Protocols: MCP, ACP, A2A, and ANP", "rationale": "

Compares the main emerging AI agent interoperability protocols across interaction modes, discovery, communication patterns, and security. It is a useful synthesis for understanding the current protocol landscape.

", "url": "https://arxiv.org/abs/2505.02279" }, { "name": "The Semantic Web", "rationale": "

Introduced the influential vision of machine-understandable web content and autonomous software agents using shared semantics. It is foundational background for interoperable, AI-readable systems.

", "url": "https://doi.org/10.1038/scientificamerican0501-34" }, { "name": "Linked Data", "rationale": "

Sets out the linked data principles for connecting structured data across sources using URIs, HTTP, and RDF-style links. It matters because agent interoperability depends on discoverable, machine-usable resources across boundaries.

", "url": "https://www.w3.org/DesignIssues/LinkedData" }, { "name": "Draft Specification of the KQML Agent-Communication Language", "rationale": "

A foundational specification for agent communication languages, defining performative-based messages among software agents. It shaped later work on agent protocols and multi-agent interoperability.

", "url": "https://ntrs.nasa.gov/api/citations/19960054467/downloads/19960054467.pdf" } ] }, "people": { "major": [ { "name": "David Soria Parra", "rationale": "

Co-creator of the Model Context Protocol. His work is central because MCP is the most widely recognized current interface for connecting AI systems to tools and context.

", "url": "https://github.com/dsp" }, { "name": "Justin Spahr-Summers", "rationale": "

Co-creator of the Model Context Protocol. He is included for shaping one of the core interoperability layers used by agentic applications.

", "url": "https://github.com/jspahrsummers" }, { "name": "Rao Surapaneni", "rationale": "

Google Cloud executive associated with the launch and donation of Agent2Agent. He is a key public leader for A2A's cross-vendor agent interoperability effort.

", "url": "https://developers.googleblog.com/en/a2a-a-new-era-of-agent-interoperability/" }, { "name": "R. V. Guha", "rationale": "

Conceived and built NLWeb at Microsoft and has a long record in web-scale structured data, including RDF, RSS, and Schema.org. His work connects web interoperability traditions to the agentic web.

", "url": "https://news.microsoft.com/source/features/company-news/introducing-nlweb-bringing-conversational-interfaces-directly-to-the-web/" }, { "name": "Tim Berners-Lee", "rationale": "

Inventor of the Web and author of Semantic Web and Linked Data work. His principles remain foundational for open, cross-boundary machine communication.

", "url": "https://www.w3.org/People/Berners-Lee/" }, { "name": "Tim Finin", "rationale": "

Leading figure in agent communication languages and a key contributor to KQML. His work is foundational for the idea that autonomous software entities need shared communication semantics.

", "url": "https://www.csee.umbc.edu/~finin/" }, { "name": "Vijoy Pandey", "rationale": "

Senior Cisco and Outshift leader closely tied to AGNTCY's development and Linux Foundation donation. He is important for the infrastructure side of open agent discovery, identity, messaging, and observability.

", "url": "https://www.linuxfoundation.org/press/linux-foundation-welcomes-the-agntcy-project-to-standardize-open-multi-agent-system-infrastructure-and-break-down-ai-agent-silos" } ] }, "tags": [ "infrastructure", "distributed systems", "multi-agent systems" ] } }