The benchmark dilemma, cheap diffusion models, and poisoning medical LLMs
In the News
We have quite a few new folks joining us for this week’s edition of Spill the GPTea. Welcome! Feel free to add any news I missed in the comments.
🗞 This Week in News
Who is funding the AI benchmarks? - The nonprofit benchmark creator Epoch AI revealed that OpenAI had supported the creation of FrontierMath, a benchmark for AI mathematical skills. Researchers claim that OpenAI had exclusive access to this benchmark until the launch of o3.
🥁 Interesting Products & Features
Codestral 25.01 from Mistral - a more efficient architecture and an improved tokenizer, it generates and completing code about 2 times faster than the previous version.
Kinetix - 3D Animations from Any Video. For Unity and Snap Lens. I was pretty impressed how slick the demos are. How did they start? With the data! They built their own 3D animation dataset.
📄 Interesting Papers
Training Large Language Models to Reason in a Continuous Latent Space - To explore the potential of LLM reasoning in an unrestricted latent space instead of using natural language, the authors introduce a new paradigm Coconut (Chain of Continuous Thought). They utilize the last hidden state of the LLM as a representation of the reasoning state (termed "continuous thought"). Rather than decoding this into a word token, they feed it back to the LLM as the subsequent input embedding directly in the continuous space. Experiments show that Coconut can effectively augment the LLM on several reasoning tasks. Authors from FAIR Meta and UC San Diego. GitHub.
Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budget - this research demonstrates very low-cost training of large-scale T2I diffusion transformer models. Because the computational cost of transformers increases with the number of patches in each image, they propose to randomly mask up to 75% of the image patches during training. Using only 37M publicly available real and synthetic images, they trained a 1.16 billion parameter sparse transformer with only $1,890 economical cost. The model achieves competitive FID and high-quality generations while incurring 118× lower cost than stable diffusion models and 14× lower cost than the current state-of-the-art approach that costs $28,400. Authors from Sony AI and University of California Riverside. GitHub.
Medical large language models are vulnerable to data-poisoning attacks - They found that replacement of just 0.001% of training tokens with medical misinformation results in harmful models more likely to propagate medical errors. They also discover that corrupted models match the performance of their corruption-free counterparts on open-source benchmarks routinely used to evaluate medical LLMs. Authors from New York University.
How human–AI feedback loops alter human perceptual, emotional and social judgements - in a series of experiments (n = 1,401 participants), the authors reveal a feedback loop where human–AI interactions alter processes underlying human perceptual, emotional and social judgements, subsequently amplifying biases in humans. This amplification is significantly greater than that observed in interactions between humans, due to both the tendency of AI systems to amplify biases and the way humans perceive AI systems. Participants are often unaware of the extent of the AI’s influence, rendering them more susceptible to it. Authors from University College London.
Neural Honeytrace: A Robust Plug-and-Play Watermarking Framework against Model Extraction Attacks - watermarking framework against model extraction attacks from an information-theoretic perspective. Guided by the model, they introduce a similarity-based training-free watermarking method for plug-and-play and flexible watermarking, and a distribution-based multi-step watermark information transmission strategy for robust watermarking. Experiments on four datasets demonstrate that Neural Honeytrace outperforms previous methods in efficiency and resisting adaptive attacks. Authors from Guangzhou University. GitHub.
Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Design - MCTS-AHD employs a Monte Carlo Tree Search (MCTS) for LLM-based heuristic evolution while preserving all LLM-generated heuristics in a tree structure. MCTS helps to comprehensively explore the space of heuristic functions and maintain the focus on better-performing ones. Moreover, as a structured data structure, the MCTS tree records the evolution history of heuristics, thus providing organized samples for heuristic evolution and LLMs' reasoning. Authors from National University of Singapore. GitHub.
AI vs AI: How effective are Turnitin, ZeroGPT, GPTZero, and Writer AI in detecting text generated by ChatGPT, Perplexity, and Gemini? - This study investigates the performance of four AI-detection tools (Turnitin, ZeroGPT, GPTZero, and Writer AI) in detecting AI-generated text. That text was generated using three LLMs (ChatGPT, Perplexity, and Gemini). Three adversarial techniques (edited through Grammarly, paraphrased through Quillbot, and 10%-20% editing by a human expert) were applied to see their effects on the performance of AI-detection tools. Turnitin had the highest accuracy and consistency, with a 100% AI score even with the adversarial techniques. Among the three adversarial techniques, paraphrasing through Quillbot affected the performance of AI-detection tools the most. Among the three LLMs, text generated through Perplexity was more accurately detected, while Gemini-generated text showed a relatively lower AI score. Authors from Shandong Vocational University of Foreign Affairs and Government of Punjab.
A Survey on Responsible LLMs: Inherent Risk, Malicious Use, and Mitigation Strategy - a review of recent advancements aimed at mitigating LLM challenges, organized across the four phases of LLM development and usage: data collecting and pre-training, fine-tuning and alignment, prompting and reasoning, and post-processing and auditing. Studies are shared focused on privacy protection, hallucination reduction, value alignment, toxicity elimination, and jailbreak defenses. Authors from Tsinghua University.
🧠 Sources of Inspiration
Train your own Sparse Autoencoder (implementation from OpenAI’s work scaling SAEs to ChatGPT).
Google Generative AI Accelerator - Six month accelerator with a pool of $30M, apply by Feb 10. Focus areas are ‘Knowledge, Skills, and Learning’, ‘Scientific Advancement’, and ‘Resilient Communities’.
Cover photo from Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budget.