Empirical Inference Conference Paper 2024

Redesigning Information Markets in the Era of Language Models

This work addresses the buyer's inspection paradox for information markets. The paradox is that buyers need to access information to determine its value, while sellers need to limit access to prevent theft. To study this, we introduce an open-source simulated digital marketplace where intelligent agents, powered by language models, buy and sell information on behalf of external participants. The central mechanism enabling this marketplace is the agents' dual capabilities: they not only have the capacity to assess the quality of privileged information but also come equipped with the ability to forget. This ability to induce amnesia allows vendors to grant temporary access to proprietary information, significantly reducing the risk of unauthorized retention while enabling agents to accurately gauge the information's relevance to specific queries or tasks. To perform well, agents must make rational decisions, strategically explore the marketplace through generated sub-queries, and synthesize answers from purchased information. Concretely, our experiments (a) uncover biases in language models leading to irrational behavior and evaluate techniques to mitigate these biases, (b) investigate how price affects demand in the context of informational goods, and (c) show that inspection and higher budgets both lead to higher quality outcomes.

Author(s): Weiss, M. and Rahaman, N. and Wüthrich, M. and Bengio, Y. and Li, L. E. and Schölkopf, B. and Pal, C.
Book Title: First Conference on Language Modeling (COLM)
Year: 2024
Month: October
Bibtex Type: Conference Paper (conference)
Event Place: Philadelphia, Pennsylvania
State: Published
URL: https://openreview.net/forum?id=Zq9Dfj4nBo
Electronic Archiving: grant_archive
How Published: arXiv:2403.14443
Links:

BibTex

@conference{WeiRahetal24,
  title = {Redesigning Information Markets in the Era of Language Models},
  booktitle = {First Conference on Language Modeling (COLM)},
  abstract = {This work addresses the buyer's inspection paradox for information markets. The paradox is that buyers need to access information to determine its value, while sellers need to limit access to prevent theft. To study this, we introduce an open-source simulated digital marketplace where intelligent agents, powered by language models, buy and sell information on behalf of external participants. The central mechanism enabling this marketplace is the agents' dual capabilities: they not only have the capacity to assess the quality of privileged information but also come equipped with the ability to forget. This ability to induce amnesia allows vendors to grant temporary access to proprietary information, significantly reducing the risk of unauthorized retention while enabling agents to accurately gauge the information's relevance to specific queries or tasks. To perform well, agents must make rational decisions, strategically explore the marketplace through generated sub-queries, and synthesize answers from purchased information. Concretely, our experiments (a) uncover biases in language models leading to irrational behavior and evaluate techniques to mitigate these biases, (b) investigate how price affects demand in the context of informational goods, and (c) show that inspection and higher budgets both lead to higher quality outcomes. },
  howpublished = {arXiv:2403.14443},
  month = oct,
  year = {2024},
  slug = {weirahetal24},
  author = {Weiss, M. and Rahaman, N. and W{\"u}thrich, M. and Bengio, Y. and Li, L. E. and Sch{\"o}lkopf, B. and Pal, C.},
  url = {https://openreview.net/forum?id=Zq9Dfj4nBo},
  month_numeric = {10}
}