Your LLM Is Only One Part of the Stack

How that stack is designed can matter as much as model choice for both output quality and cost.

In AI-assisted research, preserving the model is not enough. The research record increasingly includes the environment, tools, prompts and infrastructure around it.

Science that can't be re-run is testimony, not evidence.

That principle becomes harder to uphold as AI enters research workflows.

Suppose a researcher wants to process 1,000 documents and extract specific information into a structured database. At first glance, this looks like a model-selection problem: which LLM gives the best extraction quality? But that is only the visible part of the system.

The model has to run somewhere. It needs access to the documents. Results need to be stored. Code or another application has to orchestrate the requests. The workflow may need retrieval, an embedding model or a vector database. Compute has to be allocated. Prompts and dependencies change. And if the result matters scientifically, someone may need to understand and re-run the work years later.

The moment an LLM moves from a chat window into a research workflow, the model stops being the reproducible unit. The workflow becomes the reproducible unit.

The 1,000-document example also exposes another problem with model-first thinking.

Imagine the extraction job does not need to return results interactively. It can run for several hours or overnight.

That changes the engineering question.

A smaller model running on more modest infrastructure may be entirely sufficient. A larger model may finish sooner, but speed may not be the constraint that matters. The relevant question becomes:

That is why benchmark rankings only tell part of the story.

Two models can perform similarly enough for a particular research task while requiring very different infrastructure. Conversely, a model that leads a benchmark may be unnecessary for a repetitive part of a workflow.

The sensible unit of evaluation is therefore not just the model.

For interactive work, latency may dominate the decision. For a large batch process, throughput and cost may matter more. For sensitive research data, where the model runs may be decisive.

The model and the compute should match the work.

The same pattern appears with retrieval-augmented generation.

A researcher may need an LLM to work with publications, internal documentation, datasets or material newer than the model's training data. Retrieval can supply relevant information to the model as additional context.

But the moment that happens, the research object becomes larger.

Now the result may depend on the language model, the source material, the embedding process, retrieval logic, the vector database and the application connecting those components.

RAG does not make an LLM deterministic, nor does it guarantee that an answer is correct. It changes the information available to the model and can help keep responses closer to material relevant to the task.

From a reproducibility perspective, however, the important observation is elsewhere: the answer is no longer produced by one model alone. It is produced by a system.

If that system changes, the conditions under which the result was produced change as well.

Agentic workflows extend this further. Instead of receiving one prompt and returning one response, an agent may inspect files, call tools, evaluate intermediate outputs, execute code or continue through several steps toward a larger objective.

Different steps may even use different models.

A stronger model might plan or review a task while a smaller model handles repetitive operations. That can make the workflow more efficient and keep model usage closer to the requirements of each step.

But it also increases the number of components that can change.

What used to be a researcher working with an editor, code and Git can become a researcher interacting with an AI system that interacts with software, services, models and infrastructure underneath.

The interface may become simpler, but the underlying research record does not.

Research software already has a durability problem.

Dependencies disappear. Libraries change. environments rot. Infrastructure moves. Code that ran reliably several years ago may no longer run at all.

AI adds another source of instability.

An external model can change. An API alias can point to a newer version. A provider can retire a model. Prompts can be edited. Agent frameworks evolve. The software surrounding the model can disappear.

In other words, computational drift and model drift both break the line.

Saving the final response from an LLM does not solve this problem, and knowing that a researcher used “Model X” does not solve it either.

To understand an AI-assisted result later, we increasingly need to know not only what model was used, but also under what computational conditions it was used.

That is where reproducibility has to become more precise.

LLMs are probabilistic systems. Re-running the same workflow does not imply that every token must be reproduced byte for byte.

That is the wrong standard.

The relevant question is whether the computational conditions of the work can be reconstructed well enough to inspect, re-run and build on the research.

Nuvolos Snapshots freeze the whole AI toolchain, local model weights, the environment, and every logged prompt, so an AI-assisted result can be re-run years later. Because we run sovereign local inference, we can pin the exact model, not a moving API alias that vendors change or retire without notice. The reproducible unit is the frozen model + environment + logged prompts — not an identical token stream.

This matters because even apparently stable model settings should not be mistaken for deterministic scientific artefacts.

The goal is not to pretend stochastic systems are deterministic.

The goal is to preserve the conditions under which the work happened.

There will always be another model.

Another benchmark leader.

Another agent framework.

Another API.

Another way of connecting AI to scientific software.

That is normal.

Research infrastructure should not require teams to rebuild the environment every time that happens.

Nuvolos treats the model as one component inside a governed workspace. Researchers can run local models where that is appropriate, connect other model providers when needed, work with code and data, and add agent tooling without making the model itself the permanent center of the environment.

For Research IT, this also changes the governance problem.

AI can enter the research environment without turning into a collection of disconnected tools around which each team has to build its own infrastructure.

Model choice can remain flexible while the research environment remains governed.

And when local inference is required, sovereignty becomes concrete in the architecture rather than simply a label attached to the service.

This is the larger shift.

AI is not merely another tool researchers use to produce results.

The model, the environment and the interactions with that model increasingly become part of the record explaining how the result came to exist.

That record needs to survive model deprecation, changing dependencies and the next generation of AI tooling.

Nuvolos keeps the line of science unbroken from first data through analysis to a finding others can build on. And now that line includes AI.

The useful questions are therefore no longer only: Which model performs best?

They become:

Which model is appropriate for this part of the workflow?

What tools and data does it depend on?

Where should each part run?

What needs to be recorded?

And can someone still re-run the work when today's model is no longer the model everyone is talking about?

The model may get the attention, but the research record is what has to survive.

The tooling around the model matters just as much as the model itself, both for output and cost.