Cameron Kast

Superintelligence in what, exactly? Maybe it’s time to be more specific about the domains where we expect AI to go beyond human capabilities. @MicrosoftAI makes medicine an interesting place to start with its new MAI-DxO research and medical intelligence framework↓ The study, published in Science, explores sequential diagnosis from a partial clinical picture. MAI-DxO (Microsoft AI Diagnostic Orchestrator) uses a language model to simulate 5 clinical roles: proposing diagnoses, selecting questions and tests, challenging assumptions, checking reasoning, and weighing costs. Its assessment evolves as evidence arrives. It achieved 83.2% accuracy on complex NEJM cases and 80.3% on emergency-department cases, vs. 36.1% and 46.8% for participating generalist physicians. → But what would make this medical superintelligence? ▪️ Microsoft’s framework outlines 6 dimensions of medical intelligence. Its bar for superintelligence is high: outperforming the strongest human-led health systems in reasoning and coordination across the full course of care. 1. Medical knowledge It applies current evidence to individual cases and recognizes what it doesn’t know. 2. Diagnostic reasoning and triage It chooses questions, tests, and urgency, revising its view as evidence arrives. Sequential diagnosis fits here. 3. Therapeutic decision-making It weighs benefits, risks, costs, and availability, adapting treatment as the patient responds. 4. Longitudinal context It tracks health over time, preserving sources and uncertainty rather than starting fresh at every visit. 5. Clinical coordination It brings in expertise, determines who acts next, and checks that follow-up happened. 6. Human values and agency It respects patients’ goals, preferences, and right to refuse a recommendation. These capabilities must work together. Treatment responses can change diagnoses; patient priorities can change treatment choices, etc. That’s why a domain-specific view of superintelligence is useful - we can give a system clear tasks and criteria it needs to meet.

Two August 10 papers suggest that near-term "self-evolution" may happen mainly in editable skills and safety harnesses around fixed model weights. It's a meaningful step, but it should still be distinguished from recursive self-improvement (RSI). 1. @MSFTResearch’s OEO (Open-Ended Optimization) lets GPT-5.5 decide which failures to inspect, how to rewrite a reusable skill, what to retain and when to stop. It won 12/14 comparisons with prescribed pipelines. 2. Safety Harness Evolution (SHE) keeps a structured loop but lets failures update 4 harness artifacts: system prompt, rule bank, safety memory and tool policy. Attack success fell from 17.1% to 5.5% vs a static SafeHarness. Put them together, and you'll see a new direction: the next self-improving agent may improve mostly by rewriting the operational layer around fixed weights. BUT: We’d keep this separate from recursive self-improvement. RSI starts when one improvement actually makes the system better at carrying out the next round of improvement. In both papers, humans still set the boundaries: the model, objective, and evaluator stay fixed. → The agent gets better at operating within that setup, but it doesn’t get better at getting better. So we'd call this layer self-evolving infrastructure.