Today’s research landscape showcases significant advancements in agentic systems and cross-lingual modeling, with a strong emphasis on memory architectures and optimization techniques. Several papers introduce novel frameworks for enhancing Large Language Model (LLM) capabilities: CAM (Constructivist Agentic Memory) draws from cognitive theory to build hierarchical memory structures for long-document comprehension, while AgentFlow introduces “in-the-flow” optimization using Flow-GRPO (Flow-based Group Refined Policy Optimization) to train planners within multi-turn agentic loops. Concurrently, ARM (Agentic Reasoning Modules) presents an evolutionary approach to discover specialized reasoning components, and Parallel Tokenizers proposes a new vocabulary alignment method to improve cross-lingual transfer in low-resource settings. These works collectively highlight a trend toward more modular, trainable, and cognitively-inspired agent architectures that demonstrate strong generalization and efficiency gains across diverse reasoning and multilingual tasks.

TL;DR

TL;DR: Recent Advances in Agentic and Multilingual AI Systems

This collection highlights significant progress in agentic reasoning systems and cross-lingual representation learning, with a focus on improving efficiency, generalization, and scalability.

🔍 Key Themes & Insights

1. Agentic Memory & Reasoning Systems

• CAM (https://arxiv.org/abs/2510.05520) introduces a constructivist memory framework inspired by Piaget’s theory, enabling hierarchical information organization for long-document comprehension with 3.0% performance gains and 4× faster processing
• ARM (https://arxiv.org/abs/2510.05746) evolves specialized reasoning modules through evolutionary search, replacing simple CoT steps with agentic modules that generalize across domains without re-optimization
• AgentFlow (https://arxiv.org/abs/2510.05592) optimizes planner modules “in-the-flow” using novel RL (Flow-GRPO), achieving 14.9% accuracy gains on search tasks and outperforming GPT-4o with 7B parameters

2. Cross-Lingual Representation Learning

• Parallel Tokenizers (https://arxiv.org/abs/2510.06128) align vocabulary indices across languages using translation dictionaries, enabling shared semantic spaces and improved cross-lingual transfer, especially for low-resource languages

💡 Common Insights

• Modular specialization outperforms monolithic approaches in complex reasoning tasks
• Dynamic, trainable systems significantly beat static, training-free agent frameworks
• Structural alignment (in memory or vocabulary) enables better generalization and efficiency
• Smaller, optimized systems can surpass much larger general-purpose models on specific tasks

These works collectively push toward more efficient, specialized, and generalizable AI systems that better handle long-context reasoning, multilingual processing, and complex tool integration.

CAM: A Constructivist View of Agentic Memory for LLM-Based Reading Comprehension

Authors: Rui Li, Zeyu Zhang, Xiaohe Bo, Zihang Tian, Xu Chen, Quanyu Dai, Zhenhua Dong, Ruiming Tang

Keywords: Agentic Memory, Constructivist Theory, Reading Comprehension, Large Language Models, Memory Development, Hierarchical Memory, Multi-document Processing

Comments: Accepted by NeurIPS 2025

Paper link: http://arxiv.org/abs/2510.05520v1

Abstract

Current Large Language Models (LLMs) are confronted with overwhelming information volume when comprehending long-form documents. This challenge raises the imperative of a cohesive memory module, which can elevate vanilla LLMs into autonomous reading agents. Despite the emergence of some heuristic approaches, a systematic design principle remains absent. To fill this void, we draw inspiration from Jean Piaget’s Constructivist Theory, illuminating three traits of the agentic memory – structured schemata, flexible assimilation, and dynamic accommodation. This blueprint forges a clear path toward a more robust and efficient memory system for LLM-based reading comprehension. To this end, we develop CAM, a prototype implementation of Constructivist Agentic Memory that simultaneously embodies the structurality, flexibility, and dynamicity. At its core, CAM is endowed with an incremental overlapping clustering algorithm for structured memory development, supporting both coherent hierarchical summarization and online batch integration. During inference, CAM adaptively explores the memory structure to activate query-relevant information for contextual response, akin to the human associative process. Compared to existing approaches, our design demonstrates dual advantages in both performance and efficiency across diverse long-text reading comprehension tasks, including question answering, query-based summarization, and claim verification.

Summary

This paper presents CAM (Constructivist Agentic Memory), a novel memory framework for LLM-based reading comprehension of long-form documents. Drawing inspiration from Jean Piaget’s Constructivist Theory, the authors propose a design blueprint centered on three key memory traits: structured schemata (hierarchical organization of information), flexible assimilation (allowing information units to contribute to multiple abstractions), and dynamic accommodation (efficient adaptation to new inputs).

The core technical contribution is an incremental overlapping clustering algorithm that builds a hierarchical memory structure from text chunks. This involves: (1) Foundational network expansion using semantic and positional similarity, (2) Ego-centric disentanglement through node replication to enable overlapping cluster assignments, and (3) Online clustering updates via label propagation for dynamic memory evolution. For inference, CAM employs a “Prune-and-Grow” retrieval strategy that combines global semantic matching with local structural exploration along memory hierarchies.

Experiments across six reading comprehension benchmarks (question answering, summarization, claim verification) demonstrate CAM’s superiority. It consistently outperforms strong baselines including RAPTOR, GraphRAG, and MemTree, achieving average gains of 3.0% across all metrics. Crucially, CAM shows 4× faster processing in online settings while maintaining stable performance, making it the first framework to simultaneously embody structurality, flexibility, and dynamicity. Ablation studies confirm the importance of both hierarchical organization and flexible assimilation in the design.

The work establishes a principled foundation for agentic memory design and provides an efficient solution for real-world long-text comprehension scenarios where documents arrive incrementally.

Critique

Of course. Here is a critique of the paper “CAM: A Constructivist View of Agentic Memory for LLM-Based Reading Comprehension,” focusing on its strengths and weaknesses.

Strengths

1. Novel and Well-Motivated Conceptual Framework: The paper’s primary strength is its foundation in Jean Piaget’s Constructivist Theory. Grounding the design of an AI memory system in a well-established theory of human cognitive development is a significant and novel contribution. The translation of core concepts—structured schemata, flexible assimilation, and dynamic accommodation—into technical design principles is elegant and provides a coherent, principled blueprint that most prior heuristic approaches lack.

2. Clear Identification of a Research Gap: The paper effectively critiques existing agentic memory systems (e.g., MemGPT, RAPTOR, GraphRAG, MemTree) by mapping them against the proposed constructivist traits (Table 1). This clearly establishes the research gap: no existing system simultaneously embodies all three desired properties, particularly the combination of flexibility and dynamicity.

3. Strong and Comprehensive Empirical Evaluation: The experimental design is rigorous. The authors evaluate CAM across six diverse, long-context benchmarks covering single- and multi-document tasks (QA, summarization, claim verification). The consistent and often substantial performance gains over a wide range of strong baselines provide compelling evidence for the effectiveness of the proposed approach. The dual focus on both performance and efficiency (especially in the online setting) is a major practical strength.

4. Practical Contribution of “Online Batch” Processing: The ability of CAM to integrate new information in batches efficiently, while maintaining performance stability, is a key practical advancement. The demonstration that it is over 4x faster than offline methods and scales sub-linearly compared to sequential online methods (MemTree) addresses a critical limitation for real-world applications like processing serialized content or live data streams.

5. Thorough Ablation and Analysis: The paper goes beyond a simple comparison by including a detailed ablation study (Table 3). Testing different LLM backbones, embedding models, retrieval strategies, and architectural variants (e.g., w/o hierarchy, w/o flexibility) provides strong evidence that the core contributions of the framework are responsible for the performance gains, not just the choice of a powerful underlying LLM.

Weaknesses

1. Complexity and Potential Computational Overhead: While the online batch processing is efficient relative to competitors, the overall framework is complex. The three-step memory development process—involving foundational network expansion, ego-centric disentanglement (with node replication), and incremental clustering—likely incurs significant computational and memory overhead, especially for massive documents. The paper acknowledges that LLM operations are the main bottleneck, but a more detailed analysis of the system’s resource footprint would be valuable.

2. Risk of Hallucination Propagation: A significant, though acknowledged, weakness is the reliance on LLM-generated summaries for building the memory hierarchy. Errors or “hallucinations” in a lower-level summary can propagate and become cemented in higher-level abstractions, potentially corrupting the entire memory structure for a document. The paper identifies this as a limitation but does not propose or evaluate any mitigation strategies.

3. Limited Exploration of Inconsistency Handling: The framework, like its predecessors, assumes internally consistent source texts. It lacks explicit mechanisms to detect, represent, or reconcile contradictory information from different parts of a document or across multiple documents. This is a major challenge in real-world open-domain scenarios, and its absence limits the system’s robustness.

4. Clarity of the “Prune-and-Grow” Retrieval: While the associative retrieval strategy is a key part of the inference process, its description is somewhat high-level. The process of using an LLM to iteratively select nodes from a candidate set could be computationally expensive and non-deterministic. A more detailed algorithmic description or a discussion of the average number of iterations required would improve clarity and help assess its practical cost.

5. Presentation of the Ego-Centric Disentanglement: The core technical innovation of “ego-centric disentanglement” and the creation of a “replica network” is conceptually dense. While Figure 2 helps, this section could benefit from a more gradual, intuitive explanation or a concrete, step-by-step mini-example to make the process more accessible to readers.

Overall Assessment

This is a high-quality paper that makes a significant contribution to the field of LLM-based agents and long-context understanding. Its main strength lies in its novel, principled approach derived from cognitive science, which is convincingly validated through extensive and well-designed experiments. The proposed CAM framework demonstrates clear and practical advantages over existing state-of-the-art methods.

The main weaknesses relate to practical implementation concerns (complexity, hallucination, inconsistency handling) rather than flaws in the core idea. The paper is generally well-written and clearly structured, though some technical sections could be further clarified. The limitations are honestly discussed, providing a solid roadmap for future work.

Parallel Tokenizers: Rethinking Vocabulary Design for Cross-Lingual Transfer

Authors: Muhammad Dehan Al Kautsar, Fajri Koto

Keywords: Parallel Tokenizers, Cross-Lingual Transfer, Vocabulary Design, Multilingual Language Models, Low-Resource Languages, Semantic Alignment

Comments: 18 pages, 25 tables, 7 figures

Paper link: http://arxiv.org/abs/2510.06128v1

Abstract

Tokenization defines the foundation of multilingual language models by determining how words are represented and shared across languages. However, existing methods often fail to support effective cross-lingual transfer because semantically equivalent words are assigned distinct embeddings. For example, “I eat rice” in English and “Ina cin shinkafa” in Hausa are typically mapped to different vocabulary indices, preventing shared representations and limiting cross-lingual generalization. We introduce parallel tokenizers. This new framework trains tokenizers monolingually and then aligns their vocabularies exhaustively using bilingual dictionaries or word-to-word translation, ensuring consistent indices for semantically equivalent words. This alignment enforces a shared semantic space across languages while naturally improving fertility balance. To assess their effectiveness, we pretrain a transformer encoder from scratch on thirteen low-resource languages and evaluate it on sentiment analysis, hate speech detection, emotion classification, and sentence embedding similarity. Across all tasks, models trained with parallel tokenizers outperform conventional multilingual baselines, confirming that rethinking tokenization is essential for advancing multilingual representation learning–especially in low-resource settings.

Summary

Based on the provided paper “Parallel Tokenizers: Rethinking Vocabulary Design for Cross-Lingual Transfer,” here is a concise summary focusing on its key contributions, methods, and results:

Key Contributions: The paper introduces parallel tokenizers, a novel framework designed to improve cross-lingual transfer in multilingual language models, especially for low-resource languages. The core idea is to align token indices for semantically equivalent words across languages, ensuring shared embeddings and addressing inefficiencies in traditional multilingual tokenization. This approach reduces fertility imbalance (token length disparities) and semantic redundancy, enabling more effective cross-lingual generalization.

Methods:

1. Parallel Vocabulary Construction:
   • A monolingual English tokenizer serves as the pivot, and its word-type vocabulary is translated into target languages using machine translation (e.g., Google Translate).
   • Monolingual tokenizers are trained for each target language, and their vocabularies are concatenated with the translated English tokens, prioritizing semantic alignment. Duplicates are removed, and the final vocabulary size is fixed (30,522 tokens).
2. Model Input Representation:
   • Language identity tokens (e.g., [JV] for Javanese) are used to select the appropriate tokenizer dynamically. Language identity embeddings are added to token embeddings to disambiguate unaligned tokens and preserve language-specific signals.

Results:

• Tokenization Quality: Parallel tokenizers achieve lower fertility (fewer tokens per word) and parity (cross-lingual token count consistency) scores compared to multilingual baselines (e.g., mBERT’s Single-102L and a custom Single-13L). This indicates more efficient and balanced tokenization.
• Downstream Tasks: The method outperforms baselines in sentiment analysis, hate speech detection, and emotion classification across varying data availability levels (1%–100% of training data), with consistent F1 score improvements.
• Cross-Lingual Representation: PCA visualizations and bitext mining show that parallel tokenizers produce semantically aligned representations across languages, clustering by meaning rather than language family. This enhances cross-lingual transfer, particularly in low-resource or zero-shot settings.
• Continual Pretraining: When applied to continual pretraining from mBERT, parallel tokenizers maintain competitive task performance while improving cross-lingual coherence.

Conclusion: The parallel tokenizer framework effectively addresses foundational challenges in multilingual modeling by aligning semantic representations across languages. It demonstrates significant gains in tokenization efficiency, downstream task performance, and cross-lingual generalization, offering a scalable solution for low-resource language inclusion.

Critique

Of course. Here is a critique of the paper “Parallel Tokenizers: Rethinking Vocabulary Design for Cross-Lingual Transfer,” focusing on its strengths, weaknesses, novelty, significance, and clarity.

Overall Summary

This paper presents a novel and well-executed approach to a fundamental problem in multilingual NLP: the misalignment of semantic representations across languages due to standard tokenization methods. The proposed “Parallel Tokenizer” framework demonstrates clear and consistent improvements across multiple low-resource language tasks, making a strong case for its significance.

Strengths

1. High Novelty: The core idea—constructing vocabularies by aligning word-type tokens across languages via translation to ensure semantically equivalent words share the same embedding index—is genuinely innovative. It directly addresses a known but often overlooked bottleneck in cross-lingual transfer. This is a more principled approach to semantic alignment compared to previous methods that focused primarily on balancing vocabulary allocation.

2. Comprehensive and Rigorous Evaluation: The paper’s experimental design is a major strength. The authors go beyond standard benchmarks by evaluating:
   • Tokenization Quality: Using fertility and parity scores provides intrinsic motivation for the method.
   • Downstream Task Performance: Testing on four different tasks (sentiment, hate speech, emotion, sentence similarity) across multiple data regimes (1% to 100%) robustly demonstrates the method’s utility, especially in low-resource settings.
   • Representation Analysis: Using PCA visualization and bitext mining to show improved cross-lingual semantic clustering provides compelling evidence for the method’s mechanism of action.
   • Ablation Studies: The experiments on limited target-language data (0-shot, 50-shot) and continual pre-training show the method’s versatility and practicality.

3. Significant and Consistent Results: The results are not just statistically significant but also practically meaningful. The parallel tokenizer consistently outperforms strong baselines (including a re-trained Single-13L tokenizer) across nearly all tasks and data conditions. The improvement in cross-lingual representation similarity (Table 3, Figure 4) is particularly convincing.

4. Clear Presentation: The paper is well-structured and easy to follow. The problem is motivated effectively with a concrete example in the abstract. Figures 1 and 2 provide an excellent high-level overview of the method. The writing is generally precise, and the experimental setup is described in sufficient detail for reproducibility.

Weaknesses

1. Scalability and Practical Overhead: The paper’s primary weakness is a lack of deep discussion on the scalability of the approach. Creating a parallel tokenizer for a new set of N languages requires:
   • Training N monolingual tokenizers.
   • Performing word-by-word translation for the pivot language’s vocabulary into N-1 target languages.
   • Manually filtering translation outputs.
   • The vocabulary size grows linearly with the number of languages, which could become a bottleneck for models covering hundreds or thousands of languages. While the method is shown to be effective, its cost-benefit analysis for very large-scale models is an open question.

2. Dependence on Translation Quality: The method’s success is inherently tied to the quality of the word-level machine translation used for vocabulary alignment. The authors acknowledge this in the limitations, noting issues with multi-word and malformed translations. Errors introduced at this stage are baked into the model’s foundation and could propagate, potentially aligning non-equivalent words. The 61% alignment rate, while good, suggests a non-trivial portion of the vocabulary remains unaligned or misaligned.

3. Limited Task Scope: While the chosen tasks are relevant, they are all sequence-level classification tasks. It remains to be seen if the benefits translate to generation tasks (e.g., machine translation, summarization) or more complex reasoning tasks, where the interaction between the aligned embeddings and the decoder/encoder might be different.

4. Comparison to More Recent Baselines: The primary multilingual baseline is mBERT, a model that is several years old. While the inclusion of a re-trained Single-13L tokenizer is a strong and fair baseline, comparing against a more modern multilingual baseline like XLM-R or a smaller version of a multilingual T5 model could have strengthened the paper further.

Conclusion

This is a high-quality paper that introduces a novel and impactful idea. The Parallel Tokenizer framework presents a compelling solution to a core problem in multilingual NLP. Its strengths in novelty, rigorous evaluation, and clear presentation far outweigh its weaknesses. The demonstrated improvements in cross-lingual transfer, particularly for low-resource languages, are significant.

The main caveats lie in the practical scalability of the method and its dependence on external translation systems. However, the paper successfully establishes a new and promising direction for research in multilingual vocabulary design, and the proposed method is likely to inspire future work in this area.

ARM: Discovering Agentic Reasoning Modules for Generalizable Multi-Agent Systems

Authors: Bohan Yao, Shiva Krishna Reddy Malay, Vikas Yadav

Keywords: Agentic Reasoning Modules, Multi-Agent Systems, Chain-of-Thought, Reasoning Enhancement, Automated System Design, Evolutionary Search, Generalizable Reasoning

Comments: 29 pages, 2 figures

Paper link: http://arxiv.org/abs/2510.05746v1

Abstract

Large Language Model (LLM)-powered Multi-agent systems (MAS) have achieved state-of-the-art results on various complex reasoning tasks. Recent works have proposed techniques to automate the design of MASes, eliminating the need for manual engineering. However, these techniques perform poorly, often achieving similar or inferior performance to simple baselines. Furthermore, they require computationally expensive re-discovery of architectures for each new task domain and expensive data annotation on domains without existing labeled validation sets. A critical insight is that simple Chain of Thought (CoT) reasoning often performs competitively with these complex systems, suggesting that the fundamental reasoning unit of MASes, CoT, warrants further investigation. To this end, we present a new paradigm for automatic MAS design that pivots the focus to optimizing CoT reasoning. We introduce the Agentic Reasoning Module (ARM), an agentic generalization of CoT where each granular reasoning step is executed by a specialized reasoning module. This module is discovered through a tree search over the code space, starting from a simple CoT module and evolved using mutations informed by reflection on execution traces. The resulting ARM acts as a versatile reasoning building block which can be utilized as a direct recursive loop or as a subroutine in a learned meta-orchestrator. Our approach significantly outperforms both manually designed MASes and state-of-the-art automatic MAS design methods. Crucially, MASes built with ARM exhibit superb generalization, maintaining high performance across different foundation models and task domains without further optimization.

Summary

Based on the paper “ARM: Discovering Agentic Reasoning Modules for Generalizable Multi-Agent Systems,” here is a summary of its key contributions, methods, and results:

Key Contributions:
The paper introduces the Agentic Reasoning Module (ARM), a novel framework that enhances Chain-of-Thought (CoT) reasoning by replacing simple textual steps with specialized, agentic modules. ARM is discovered automatically through an evolutionary process and serves as a versatile, domain-agnostic building block for multi-agent systems (MAS). The authors demonstrate that ARM significantly outperforms both manually designed and automatically generated MAS approaches while maintaining strong generalization across tasks and foundation models without requiring re-optimization.

Methods:
ARM decomposes reasoning into two components: a Step-Generator Module (m∗), which executes granular reasoning steps using an internal MAS, and a Meta-Policy (π∗), which orchestrates these steps into a complete solution. The framework employs a Reflection-Guided Evolutionary Search to iteratively refine these components. Starting from a baseline CoT module, a Reviewer Agent (comprising a Critic and Designer) analyzes execution traces and proposes targeted code mutations. A scaffolded surrogate objective enables efficient optimization by evaluating candidate modules within stable CoT contexts, avoiding the need for full rollouts. The meta-policy is similarly discovered using a fast surrogate (mCoT) and transfers zero-shot to the optimized ARM.

Results:
Experiments on complex reasoning benchmarks (AIME, HMMT, GPQA, LiveBench) using models like GPT-4.1-nano, GPT-4o, and LLaMA-3.3-70B show that ARM consistently outperforms strong baselines, including CoT, Self-Refine, LLM-Debate, and state-of-the-art automated MAS methods like ADAS and AFlow. ARM achieved the highest average performance (e.g., 47.8% with GPT-4.1-nano) and demonstrated superior generalization across domains and models. Analyses confirmed that ARM reduces per-step error rates and that the meta-policy effectively transfers from surrogate to optimized modules, validating the decoupled discovery strategy.

Critique

Of course. Here is a critique of the paper “ARM: Discovering Agentic Reasoning Modules for Generalizable Multi-Agent Systems,” focusing on its strengths, weaknesses, novelty, significance, and clarity.

Overall Summary

This paper presents ARM, a novel method for automatically discovering a powerful, general-purpose “Agentic Reasoning Module” to replace the simple step-generation in Chain-of-Thought (CoT) prompting. The approach is compelling and the results are significant, demonstrating strong performance and generalization across multiple models and benchmarks. However, the paper has notable weaknesses in its empirical validation and clarity of certain methodological details.

Strengths

1. High Novelty and Insightful Pivot: The core idea is highly novel. Instead of designing increasingly complex multi-agent systems (MAS) with heterogeneous agents, the paper pivots to a more fundamental problem: improving the basic “reasoning unit” itself. The insight that simple CoT often outperforms complex MAS is used to motivate a focus on evolving a superior, drop-in replacement for a single CoT step. This is a refreshing and potentially more scalable direction for the field.

2. Elegant and Principled Methodology: The proposed framework is elegantly decomposed into a Step-Generator (m*) and a Meta-Policy (π*). The use of a “scaffolded surrogate objective” for discovering the Step-Generator is a clever solution to the credit assignment problem in long reasoning chains. Similarly, using a cheap surrogate (m_CoT) to discover the Meta-Policy before transferring it to the powerful m* is a computationally efficient and theoretically justified strategy.

3. Strong and Compelling Results: The empirical results are a major strength. ARM consistently outperforms a wide range of strong baselines, including handcrafted methods (CoT-SC, Self-Refine) and state-of-the-art automated MAS design systems (ADAS, AFlow). Crucially, it demonstrates impressive generalization:
   • It performs well across three different backbone LLMs (GPT-4.1-nano, GPT-4o, Llama-3.3-70B).
   • It achieves this with a single, domain-agnostic training run, unlike competing methods that often require expensive per-domain re-optimization.
4. Effective Ablations and Analysis: The paper includes meaningful analyses that validate its core claims. Figure 3 empirically shows that better-ranked ARMs have a lower per-step error rate, validating the search objective. Figure 2 successfully disentangles the performance gains from the better module (m*) versus the full system (π* + m*), providing strong evidence for the efficacy of the transfer learning strategy.

Weaknesses

1. Lack of Transparency on Discovered Modules: A significant weakness is the lack of detailed description and analysis of the actual ARM and Meta-Policy that were discovered. Appendices C and D are referenced but not included in the provided text. Understanding what the evolutionary process actually created (e.g., “CriticChainOfThoughtV7”) is critical for assessing the method’s interpretability and for the community to build upon. Without this, the “discovery” process feels like a black box.

2. Computational Cost Omission: The paper rightly criticizes other automated MAS methods for being computationally expensive but provides no details on the cost of its own ARM discovery process. The evolutionary tree search with LLM-based reflection and evaluation is likely extremely expensive. A discussion of the computational budget (e.g., number of LLM calls, total cost) is necessary for a fair comparison.

3. Insufficient Baseline Comparison:
   • Tree-of-Thoughts (ToT): A major omission is a comparison with Tree-of-Thoughts and its variants. Given that the discovered Meta-Policy likely involves branching and searching over reasoning paths, a comparison with ToT is essential to contextualize the performance gains.
   • Advanced CoT Variants: The paper compares against standard CoT and Self-Consistency, but not against more recent, sophisticated reasoning methods like Graph-of-Thoughts or other advanced single-agent reasoning frameworks.
4. Clarity of the “Agentic” Nature: The paper positions ARM as an “agentic” block, but its description often makes it sound more like an optimized, complex function or a sophisticated prompt. The distinction between a “module” that internally might use multiple LLM calls and a traditional “multi-agent system” with distinct, role-based agents could be clearer. The claim of building a “MAS from homogeneous building blocks” is interesting, but the provided example (Figure 1) of “Self-refine” is a standard single-agent loop, not a clear multi-agent interaction.

Assessment of Significance and Clarity

• Significance: The results are highly significant. The paper makes a strong case that the prevailing trend of building ever-more-complex heterogeneous MAS may be suboptimal, and that a focus on improving foundational reasoning primitives can yield better, more generalizable systems. If the method is as computationally feasible as it is effective, it could represent a major shift in how we approach reasoning with LLMs.

• Clarity: The presentation is generally clear. The writing is good, the decomposition of the methodology is logical, and the figures support the text well. The main issues with clarity are the omission of critical details (the actual discovered code, computational cost) and the lack of comparison with key baselines (ToT). The theoretical appendix, while commendable, is dense and may be challenging for some readers, but its inclusion is a positive aspect.

Conclusion

This paper introduces a novel, powerful, and well-motivated method that challenges the current paradigm in multi-agent reasoning systems. Its strengths in novelty, empirical performance, and generalization are substantial. However, its impact is currently limited by a lack of transparency regarding the discovered modules and their computational cost, as well as by incomplete empirical comparisons. Addressing these weaknesses in a future version would make this an exceptionally strong contribution.

In-the-Flow Agentic System Optimization for Effective Planning and Tool Use

Authors: Zhuofeng Li, Haoxiang Zhang, Seungju Han, Sheng Liu, Jianwen Xie, Yu Zhang, Yejin Choi, James Zou, Pan Lu

Keywords: Agentic Systems, Tool-Integrated Reasoning, In-the-Flow Optimization, Reinforcement Learning, Multi-Turn Planning, Long-Horizon Credit Assignment, Flow-GRPO

Comments: 45 pages, 12 figures. Project website: https://agentflow.stanford.edu/

Paper link: http://arxiv.org/abs/2510.05592v1

Abstract

Outcome-driven reinforcement learning has advanced reasoning in large language models (LLMs), but prevailing tool-augmented approaches train a single, monolithic policy that interleaves thoughts and tool calls under full context; this scales poorly with long horizons and diverse tools and generalizes weakly to new scenarios. Agentic systems offer a promising alternative by decomposing work across specialized modules, yet most remain training-free or rely on offline training decoupled from the live dynamics of multi-turn interaction. We introduce AgentFlow, a trainable, in-the-flow agentic framework that coordinates four modules (planner, executor, verifier, generator) through an evolving memory and directly optimizes its planner inside the multi-turn loop. To train on-policy in live environments, we propose Flow-based Group Refined Policy Optimization (Flow-GRPO), which tackles long-horizon, sparse-reward credit assignment by converting multi-turn optimization into a sequence of tractable single-turn policy updates. It broadcasts a single, verifiable trajectory-level outcome to every turn to align local planner decisions with global success and stabilizes learning with group-normalized advantages. Across ten benchmarks, AgentFlow with a 7B-scale backbone outperforms top-performing baselines with average accuracy gains of 14.9% on search, 14.0% on agentic, 14.5% on mathematical, and 4.1% on scientific tasks, even surpassing larger proprietary models like GPT-4o. Further analyses confirm the benefits of in-the-flow optimization, showing improved planning, enhanced tool-calling reliability, and positive scaling with model size and reasoning turns.

Summary

Based on the provided paper, here is a summary focusing on its key contributions, methods, and results:

Key Contributions: This paper introduces AgentFlow, a novel, trainable agentic system designed for effective planning and tool use in complex reasoning tasks. Its primary contributions are twofold: first, the AgentFlow framework itself, which coordinates four specialized modules (Planner, Executor, Verifier, and Generator) via a shared, evolving memory; and second, the Flow-based Group Refined Policy Optimization (Flow-GRPO) algorithm, an on-policy reinforcement learning method that enables stable training of the planner directly within the multi-turn execution loop of the agentic system.

Methods: The core innovation of AgentFlow lies in its “in-the-flow” optimization. Unlike monolithic tool-integrated models that train a single policy or training-free agentic systems that rely on static orchestration, AgentFlow actively trains its Planner module. The Flow-GRPO algorithm is central to this. It tackles the long-horizon credit assignment problem by “broadcasting” a single, verifiable final-outcome reward (e.g., from an LLM-as-judge) to every turn in a multi-turn trajectory. This effectively converts the complex multi-turn RL problem into a series of simpler, single-turn policy updates. The method uses group-normalized advantages and KL regularization to stabilize training, allowing the Planner to learn which tools to use and how to use them effectively over long reasoning chains.

Results: The authors conducted a comprehensive evaluation across ten benchmarks spanning search-intensive, agentic, mathematical, and scientific reasoning tasks. Using a Qwen2.5-7B-Instruct model for all modules, AgentFlow with Flow-GRPO fine-tuning significantly outperformed a wide range of strong baselines. It achieved average accuracy gains of 14.9% on search tasks, 14.0% on agentic tasks, 14.5% on mathematical reasoning, and 4.1% on scientific reasoning over the best-performing specialized baselines. Notably, this 7B-parameter system even surpassed the much larger, proprietary GPT-4o (~200B parameters) across all evaluated domains. Further analysis showed that the trained planner learns to optimize tool usage, reduce tool-calling errors, and autonomously discover effective solution pathways, with performance scaling positively with both model size and the number of allowed reasoning turns.

Critique

Of course. Here is a commentary on the strengths and weaknesses of the paper “In-the-Flow Agentic System Optimization for Effective Planning and Tool Use.”

Summary of Strengths

1. High Novelty in the Core Idea: The paper’s central concept—optimizing a single module (the planner) within the live, multi-turn loop of an agentic system—is genuinely novel and addresses a significant gap in the literature. It successfully bridges the gap between monolithic, trainable tool-integrated models (which scale poorly) and flexible, multi-agent systems (which are typically static and training-free). The idea of “in-the-flow” optimization is a powerful and well-motivated contribution.

2. Well-Designed Algorithm (Flow-GRPO): The proposed Flow-GRPO algorithm is clever and well-suited to the problem. The key insight of “broadcasting” a single, sparse, final-outcome reward to every turn in the trajectory effectively converts a complex multi-turn credit assignment problem into a series of more tractable single-turn updates. This is a simple yet powerful solution to a notoriously difficult problem in RL. The use of group-normalized advantages is a sensible technique to stabilize training.

3. Extensive and Impressive Empirical Results: The experimental section is a major strength. The paper demonstrates state-of-the-art performance across a wide range of ten benchmarks (search, agentic, math, science) using only a 7B parameter model, even outperforming much larger proprietary models like GPT-4o. This is a significant result that strongly validates the proposed method. The inclusion of scaling laws (model size, turn budget) and in-depth analysis (tool usage, error rates) provides a comprehensive picture of the system’s capabilities.

4. Clear Ablations and Analysis: The ablation study in Section 4.4 is particularly compelling. It clearly shows that:

   • Simply using a more powerful, frozen planner (GPT-4o) provides only modest gains.
   • Offline Supervised Fine-Tuning (SFT) on expert trajectories leads to catastrophic performance collapse, highlighting the necessity of on-policy, outcome-driven learning.
   • Flow-GRPO is the crucial component that drives the substantial performance improvements.

Summary of Weaknesses

1. Clarity of the “Evolving Memory”: While the memory module M is central to the architecture, its exact structure and the f_mem update function remain somewhat abstract. The paper states it’s a “deterministic, structured record,” but a more concrete example or a detailed schema in the appendix would significantly improve clarity and reproducibility. Understanding what information is stored and how it evolves is key to understanding the planner’s state.

2. Limited Discussion of Limitations: The paper would be strengthened by a more explicit discussion of its limitations. For instance:
   • Computational Cost: Running full multi-turn rollouts for on-policy RL is inherently more expensive than offline training. The cost of data collection and training, especially with tools like web search, is non-trivial and should be acknowledged.
   • Generalization to New Tools: The experiments are conducted with a fixed, known set of tools. It’s unclear how well the trained planner would generalize to entirely new, unseen tools not present during training. The system’s reliance on tool metadata prompts suggests this could be a challenge.
   • Module Interdependence: The paper focuses on training only the planner, keeping the executor, verifier, and generator fixed. This is a practical choice, but it raises the question of whether co-training or fine-tuning other modules could lead to further gains, or if a poorly performing verifier/generator could become a bottleneck.
3. Presentation of the Core Objective: The mathematical formulation of the Flow-GRPO objective (Eq. 5) is dense and could be more accessible. While the intuition of broadcasting the reward is explained well, the equation itself, with its four nested summations (over groups, turns, trajectories, and tokens), is intimidating. A more gradual derivation or a higher-level pseudocode in the main text could improve readability for a broader audience.

Overall Assessment

This is a high-quality paper with a strong, novel contribution. The “in-the-flow” optimization paradigm and the Flow-GRPO algorithm are significant advancements for training collaborative AI agents. The empirical results are robust, extensive, and convincingly demonstrate the superiority of the approach over a wide range of strong baselines.

The main weaknesses are primarily related to exposition—specifically, making the memory mechanism and the full algorithmic cost more transparent—and a more thorough discussion of the method’s boundaries and limitations. Nonetheless, the core ideas and results are compelling and likely to influence future research in agentic systems and tool-integrated reasoning.