Research on Chain-of-Thought PII Leakage in LLMs

A recent study on Chain-of-Thought (CoT) prompting reveals significant privacy risks linked to language models (LLMs). The research demonstrates that CoT can inadvertently reintroduce personally identifiable information (PII) from inputs into reasoning outputs, potentially violating privacy policies. Using a model-agnostic approach, the analysis evaluates leakage across 11 PII types, assessing how leakage varies with CoT budgets and model types. Key findings include evidence that CoT raises the risk of PII re-emergence, particularly for high-risk categories, necessitating enhanced oversight during model inference.

The implications of this research are substantial for both model developers and policymakers. The findings advocate for a nuanced understanding of risk management in LLM deployments, suggesting that increased reasoning budgets can have varying impacts on leakage risk depending on the base model's characteristics. By benchmarking several detection methods, the study encourages the adoption of hybrid policies that balance efficacy with privacy protection, marking a critical step toward safer AI deployments.