LLM Research Summaries

Tracking ideas of interest, with a tendency towards local models, training, education science, culture, and history.

Contents

• Models Review
• Prompting
• Document Processing
• Training
• Data Science
• Code and Tools
• Interactive Agents
• Human-like Computer Use
• Education
• Science
• Psychology
  • AI Psychology
  • Mental Health
  • Types Predict
  • Traits Detect
• Future Perspectives
• Lore
• Historical
  • Statistical Language Models (SLMs) Era: 1990-2010

Models Review

• What is the Role of Small Models in the LLM Era: A Survey [small-llm]
• Evaluation of OpenAI o1: Opportunities and Challenges of AGI [o1_use-case-eval]
• Surveying the MLLM Landscape: A Meta-Review of Current Surveys [MLLM-Survey]

Prompting

• Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing [Prompt-Methods]
• The Prompt Report: A Systematic Survey of Prompting Techniques [Prompt-Methods][Survey]
• ComfyGen: Prompt-Adaptive Workflows for Text-to-Image Generation [ImageGen]
• Thinking Before Speaking: A Role-playing Model with Mindset [Roleplay]

Document Processing

• Detect-Order-Construct: A Tree Construction based Approach for Hierarchical Document Structure [Ingestion]
• Attention-Seeker: Dynamic Self-Attention Scoring for Unsupervised Keyphrase Extraction [Keywords]
• Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems [Long Context]
• Model-based Preference Optimization in Abstractive Summarization without Human Feedback [Preference Optimization]
• Leveraging Long-Context Large Language Models for Multi-Document Understanding and Summarization in Enterprise Applications [Multi-Document]
• Towards Efficient Methods in Medical Question Answering using Knowledge Graph Embeddings [Q/A]
• Evaluation of Large Language Models for Summarization Tasks in the Medical Domain: A Narrative Review [Medical Summary]
• Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits [Editing]
• ActiveRAG: Revealing the Treasures of Knowledge via Active Learning
• VERITAS-NLI: Validation and Extraction of Reliable Information Through Automated Scraping and Natural Language Inference

Training

• Ultimate Guide to Fine-Tuning LLMs: From Basics to Breakthroughs [Fine-Tune]
• Finetuning LLMs for Comparative Assessment Tasks [Fine-Tune] [Assessment]
• Propulsion: Steering LLM with Tiny Fine-Tuning [Fine-Tune] [Tiny]
• InfiniPot: Efficiently Handling Long Input Contexts in Large Language Models (LLMs) on Memory-Constrained Devices [Long-Context] [Low-Memory]
• Inheritune: Training Smaller Yet More Attentive Language Models
• BA-LoRA: Bias-Alleviating Low-Rank Adaptation to Mitigate Catastrophic Inheritance in Large Language Models
• BitDelta: Your Fine-Tune May Only Be Worth One Bit
• Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning
• Retrieval Instead of Fine-tuning: A Retrieval-based Parameter Ensemble for Zero-shot Learning

Data Science

• Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows? [Data-Engineering]
• Unified Framework to Classify Business Activities into International Standard Industrial Classification through Large Language Models for Circular Economy [Classification]
• Data-Prep-Kit: getting your data ready for LLM application development [Application]
• Data Analysis in the Era of Generative AI [Data-Analysis]

Code and Tools

• LLM With Tools: A Survey [Tools]
• From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging [Code]
• ToolGen: Unified Tool Retrieval and Calling via Generation [Tools]
• STEPTOOL: A STEP-GRAINED REINFORCEMENT LEARNING FRAMEWORK FOR TOOL LEARNING IN LLMS

Interactive Agents

• A Survey on Complex Tasks for Goal-Directed Interactive Agents [Agents]
• Do LLMs suffer from Multi-Party Hangover? [Multiparty]
• TRAINING LANGUAGE MODELS TO WIN DEBATES WITH SELF-PLAY IMPROVES JUDGE ACCURACY [Self-Play]
• Do great minds think alike? Investigating Human-AI Complementarity in Question Answering with caimira [Review-vs-Human]
• Conversational Swarms Of Humans And Ai Agents Enable Hybrid Collaborative Decision-Making
• Agentic Information Retrieval
• AFLOW : AUTOMATING AGENTIC WORKFLOW GENERATION

Human-like Computer Use

• WEBLINX: Real-World Website Navigation with Multi-Turn Dialogue [Web-Naviation]
• Agent S: An Open Agentic Framework that Uses Computers Like a Human

Education

• LLMs in Education: Novel Perspectives, Challenges, and Opportunities [Review] [Learning]
• Exploring the Use of ChatGPT for a Systematic Literature Review: a Design-Based Research [Review][Science]
• The application of GPT-4 in grading design university students’ assignment and providing feedback: An exploratory study [Grading]
• Hey AI Can You Grade My Essay?: Automatic Essay Grading [Grading]
• The Future of Learning in the Age of Generative AI: Automated Question Generation and Assessment
• Students Rather Than Experts: A New AI For Education Pipeline To Model More Human-Like And Personalised Early Adolescences

Science

• LANGUAGE AGENTS ACHIEVE SUPERHUMAN SYNTHESIS OF SCIENTIFIC KNOWLEDGE [Science]
• ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery
• Two Heads Are Better Than One: A Multi-Agent System Has the Potential to Improve Scientific Idea Generation
• Society of Medical Simplifiers
• HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation
• Retrieval-based Parameter Ensemble for Zero-shot Learning [medical]

Psychology

• Autoformalization of Game Descriptions using Large Language Models [Game-Theory]
• PropaInsight: Toward Deeper Understanding of Propaganda in Terms of Techniques, Appeals, and Intent [Social-Psychology]
• Large Language Models based on historical text could offer informative tools for behavioral science [History] [Behavioral-Science]
• Mapping neurotransmitter systems to the structural and functional organization of the human neocortex [Neural-Correlates]
• Automated Social Science: Language Models as Scientist and Subjects

AI Psychology

• Large Language Models and Cognitive Science: A Comprehensive Review of Similarities, Differences, and Challenges
• Measuring Human and AI Values based on Generative Psychometrics with Large Language Models
• Evaluating Psychological Safety of Large Language Models
• When AIs Play God(se): The Emergent Heresies of LLMtheism
• Can a Large Language Model be a Gaslighter?

Mental Health

• DISCO: A Hierarchical Disentangled Cognitive Diagnosis Framework for Interpretable Job Recommendation
• A COMPREHENSIVE EVALUATION OF LARGE LANGUAGE MODELS ON MENTAL ILLNESS
• Severity Prediction in Mental Health: LLM-based Creation, Analysis, Evaluation of a Novel Multilingual Dataset
• MentalArena: Self-play Training of Language Models for Diagnosis and Treatment of Mental Health Disorders [Diagnosis-Treatment]

Types Predict

• Machine Mindset: An MBTI Exploration of Large Language Models [MBTI]
• Myers-Briggs Personality Type Prediction Based on Reddit Comments [MBTI]

Traits Detect

• Harnessing Large Language Models: Fine-tuned BERT for Detecting Charismatic Leadership Tactics in Natural Language [Personality-Detect]
• Words that Represent Peace
• Research on Predicting Public Opinion Event Heat Levels Based on Large Language Models [Social-Science]
• Traversing Emotional Landscapes and Linguistic Patterns in Bernard-Marie

Future Perspectives

• Leopold Aschenbrenner's Situational Awareness + The Decade Ahead (2024) [aschenbrenner]
• The Future of Agents: AI is about to completely change how you use computers [Gates]
• Dystopian Dreams, Utopian Nightmares: AI and the Permanence of Racism

Lore

• When AIs Play God(se): The Emergent Heresies of LLMtheism [Truth-Terminal] [Claude-3] [Opus]
• Can AI Meditate? [Claude-3] [Sonnet]

Historical

Statistical Language Models (SLMs) Era: 1990-2010

• 1990: Hidden Markov Models for speech recognition (Rabiner) [Voice-Command-Systems]
• 1993: IBM Model 1 for statistical machine translation (Brown et al.) [Early-Online-Translation]
• 1995: Improved backing-off for M-gram language modeling (Kneser & Ney) [Spell-Checkers]
• 1996: Maximum Entropy Models (Berger et al.) [Text-Classification]
• 1999: An empirical study of smoothing techniques for language modeling (Chen & Goodman) [Improved-Language-Models]
• 2002: Latent Dirichlet Allocation (LDA) (Blei et al.) [Document-Clustering]
• 2006: Hierarchical Pitman-Yor language model (Teh) [Text-Generation]