Can Textual Gradient Work in Federated Learning?

Authors: Minghui Chen, Ruinan Jin, Wenlong Deng, Yuanyuan Chen, Zhi Huang, Han Yu, Xiaoxiao Li

Published in: The Thirteenth International Conference on Learning Representations (ICLR 2025)

Abstract

Recent studies highlight the promise of LLM-based prompt optimization, especially with TextGrad, which automates “differentiation” via texts and backpropagates textual feedback provided by LLMs. This approach facilitates training in various real-world applications that do not support numerical gradient propagation or loss calculation. It opens new avenues for optimization in decentralized, resource-constrained environments, suggesting that users of black-box LLMs (e.g., ChatGPT) could enhance components of LLM agentic systems (such as prompt optimization) through collaborative paradigms like federated learning (FL).

In this paper, we systematically explore the potential and challenges of incorporating textual gradient into FL. Our contributions are fourfold:

1. We introduce a novel FL paradigm, Federated Textual Gradient (FedTextGrad), that allows FL clients to upload their locally optimized prompts derived from textual gradients, while the FL server aggregates the received prompts through text summarization.

2. We conduct extensive experiments to explore the feasibility of federated textual gradients, highlighting the importance of properly tuning key factors (e.g., local steps) in FL training.

3. We identify a major challenge in federated textual gradient aggregation - retaining essential information from distributed prompt updates.

4. We improve the vanilla variant of FedTextGrad by providing actionable guidance to the LLM when summarizing client prompts by leveraging the Uniform Information Density principle.

Through this principled study, we enable the adoption of textual gradients in FL for optimizing LLMs, identify important issues, and pinpoint future directions, thereby opening up a new research area that warrants further investigation.