🗞 This Week in News
ChatGPT Canvas - interactive interface for writing and coding. Currently in beta, but it will soon launch to free users too
California’s hotly debated AI bill SB 1047 is vetoed - the bill would have made companies that develop AI models liable for implementing safety protocols to prevent “critical harms.” Well, some companies - the rules would only apply to models that cost >$100 million and used 10^26 FLOPS during training. Once thing is true - the debate around this bill has brought AI safety to headlines globally.
Fighting deepfakes at Microsoft - This article describes the Coalition for Content Provenance and Authenticity (C2PA), an organization that Microsoft co-founded to develop an open technical standard for establishing the provenance (source and history) of digital content, including AI-generated content in the metadata of an image. Because metadata is invisible, Microsoft provides a public Content Integrity Check tool and a web browser extension for people to scan for credentials. Platforms like LinkedIn are using this to add Content Credentials icons on images and videos.
🥁 Interesting Products & Features
Grab the popcorn - it’s movie time! 🎞️
Flux 1.1 [pro] - six times faster generation than its predecessor FLUX.1 [pro] while also improving image quality, prompt adherence, and diversity
📄 Interesting Papers
Seeing Faces in Things: A Model and Dataset for Pareidolia - Humans detect faces in everything - from clouds to your lunch. This is called “pareidolia”. This paper presents an image dataset of "Faces in Things", consisting of five thousand web images with human-annotated pareidolic faces. Using this dataset, they examine the extent to which a SOTA human face detector exhibits pareidolia, and find a significant behavioral gap between humans and machines. Authors from MIT, Microsoft, Toyota, and NVIDIA.
A Survey on the Honesty of Large Language Models - Current LLMs exhibit dishonest behaviors, such as confidently presenting wrong answers or failing to express what they know. And research on the honesty of LLMs is challenging: there exist varying definitions of honesty, difficulties in distinguishing between known and unknown knowledge, and a lack of comprehensive understanding of related research. To address these issues, this article provides a survey on the honesty of LLMs, covering its clarification, evaluation approaches, and strategies for improvement. Authors from various institutions including Chinese University of Hong Kong, Tsinghua University, and University of Virginia.
Integrative Decoding: Improve Factuality via Implicit Self-consistency - Self-consistency-based approaches involve repeatedly sampling multiple outputs and selecting the most consistent one as the final response are effective in improving the factual accuracy of LLMs. This paper presents Integrative Decoding (ID) for self-consistency in open-ended generation tasks. ID operates by constructing a set of inputs, each prepended with a previously sampled response, and then processes them concurrently, with the next token being selected through aggregation of all their corresponding predictions at each decoding step. This simple approach implicitly incorporates self-consistency in the decoding objective. Evaluation shows that ID consistently enhances factuality over a wide range of language models. Authors from Hong Kong Polytechnic University, Microsoft, and UI Urbana-Champaign.
shapiq: Shapley Interactions for Machine Learning - This article introduces shapiq, an open-source Python package that unifies SOTA algorithms to efficiently compute Shapley Values. They also include a benchmarking suite containing 11 ML applications with pre-computed games and ground-truth values to systematically assess computational performance across domains. shapiq is able to explain and visualize any-order feature interactions in predictions of models, including vision transformers, language models, as well as XGBoost and LightGBM with TreeSHAP-IQ. Authors from Munich Center for Machine Learning and Bielefeld University.
🧠 Sources of Inspiration
A beautiful article written by my friend (and fellow Duke prof) Mark DeLong - a multifaceted exploration of LLMs, language, and human-AI interaction:
