Surveying the literature with LLMs
One of our primary usecases for using LLM-based workflows is to survey the literature. LLM-based agents are very useful for quickly obtaining an overview of what has been done in a field.
One still has to keep a few caveats in mind:
- Even when using LLM-based agents there is no guarantee that there are no hallucinations. While the hallucination rate is much lower than with pure LLMs, there can still be hallucinations.
- It is very difficult for those systems to be comprehensive. This has to do with the fact that some resources are very hard to access and that, in general, it is very difficult in machine learning systems to have a high precision and recall at the same time.
Answering general questions
For answering general questions we often default to Perplexity. This is based on web search and can compile answers to questions based on the search results. It is fast, but will only give superficial answers to scientific questions.
We most often use conventional search engines such as Google if we know what we search for (e.g. we want to look up opening hours of something). LLM-assisted search tools (Perplexity, but also ChatGPT with web search) are best in cases where we do not even exactly know what we search for.
Answering scientific questions
FutureHouse Platform
For surveying scientific papers, we often use the FutureHouse Plaform. On there, they have different agents.
Asta
An alternative tool, Asta, built by Ai2 can perform similar search and we often use it in combination with the FutureHouse Plaform.
OpenAI Deep Research (đź’µ)
Especially with GPT-5 Pro, Deep Research can sometimes deliver impressive results.
“Consuming papers”
For more easily digesting papers, we use different tools:
NotebookLM is excellent for converting paper(s) into forms that are easier to consume, such as podcasts, mindmaps, reports, videos. We mostly use it to generate podcasts and study guides. The podcasts are particularly useful for getting key ideas (or even differences) between papers in a form that can be consumed on the go. Sometimes the results are also useful to inspire a narrative.
Speechify can be useful to listen to papers (instead of reading them).