Your LLM Is Only One Part of the Stack

How that stack is designed can matter as much as model choice for both output quality and cost.

A smaller model running on more modest infrastructure may be entirely sufficient. A larger model may finish sooner, but speed may not be the constraint that matters. The engineering choice is not just model size and type. It is the schema harness enforcing structural integrity and instruction compliance before data hits the database.

Suppose a researcher wants to process 1,000 documents and extract specific information into a structured database. At first glance, this looks like a model-selection problem: which LLM gives the best extraction quality? But that is only the visible part of the system.

The model has to run somewhere. It needs access to the documents. Results need to be stored. Code or another application has to orchestrate the requests. The workflow may need retrieval, an embedding model or a vector database. Compute has to be allocated. Prompts and dependencies change. And if the result matters scientifically, someone may need to understand and re-run the work years later.

The moment an LLM moves from a chat window into a research workflow, the model stops being the reproducible unit. The workflow becomes the reproducible unit.

The 1,000-document example also exposes another problem with model-first thinking.

Imagine the extraction job does not need to return results interactively. It can run for several hours or overnight.

That changes the engineering question.

A smaller model running on more modest infrastructure may be entirely sufficient. A larger model may finish sooner, but speed may not be the constraint that matters. The relevant question becomes: Which model is appropriate for this step, on which infrastructure, with which tools, and at what cost?

LLMs are probabilistic systems. Re-running the same workflow does not imply reproducing every token byte for byte. If outputs vary, how do you verify reproducibility? Through an evaluation harness that tests task success and semantic correctness rather than identical token streams.

A model should be evaluated in context, against the task, infrastructure, tools, data and constraints it will actually operate within.

For interactive work, latency may dominate the decision, whereas for a large batch process, throughput and cost may matter more. For sensitive research data, where the model runs may be decisive. The model and the compute should match the work.

The same pattern appears with retrieval-augmented generation.

A researcher may need an LLM to work with publications, internal documentation, datasets or material newer than the model's training data. Retrieval can supply relevant information to the model as additional context.

But the moment that happens, the research object becomes larger. Now the result may depend on the language model, the source material, the embedding process, retrieval logic, the vector database and the application connecting those components.

RAG can steer a model toward more relevant information, but it does not make the model deterministic or guarantee correctness. More importantly for reproducibility, the answer is no longer produced by one model alone. It is produced by a system.

Agentic workflows extend this further. Instead of receiving one prompt and returning one response, an agent may inspect files, call tools, evaluate intermediate outputs, execute code or continue through several steps toward a larger objective.

Different steps may even use different models.

A stronger model might plan or review a task while a smaller model handles repetitive operations. That can make the workflow more efficient and keep model usage closer to the requirements of each step.

What used to be a researcher working with an editor, code and Git can become a researcher interacting with an AI system that interacts with software, services, models and infrastructure underneath.

The interface may become simpler, but the underlying research record does not.

Research software already has a durability problem.

Dependencies disappear. Libraries change. Environments rot. Infrastructure moves. Code that ran reliably several years ago may no longer run at all.

AI adds another source of instability.

An external model can change. An API alias can point to a newer version. A provider can retire a model. Prompts can be edited. Agent frameworks evolve. The software surrounding the model can disappear.

In other words, computational drift and model drift both break the line.

Saving the final output, or simply recording that “Model X” was used, is not enough. Re-running the work requires knowing the computational conditions under which the result was produced.

That is where reproducibility has to become more precise.

LLMs are probabilistic systems. Re-running the same workflow does not imply reproducing every token byte for byte. The relevant question is whether the computational conditions can be reconstructed well enough to inspect, re-run and build on the work.

Nuvolos Snapshots freeze the whole AI toolchain, local model weights, the environment, and every logged prompt, so an AI-assisted result can be re-run years later. Because we run sovereign local inference, we can pin the exact model, not a moving API alias that vendors change or retire without notice.

The reproducible unit = the frozen model + environment + logged prompts, not an identical token stream.

The goal is not to make a stochastic system deterministic. It is to preserve the conditions under which the work happened.

✔ There will always be another model.

✔ Another agent framework.

✔ Another API.

That is normal. Research infrastructure should not require teams to rebuild the surrounding environment every time one of them changes.

Nuvolos treats the model as one component inside a governed workspace. Researchers can run local models where that is appropriate, connect other model providers when needed, work with code and data, and add agent tooling without making the model itself the permanent center of the environment.

For Research IT, this creates a governance problem as well: AI should not enter the institution as a growing collection of disconnected tools, each with its own infrastructure. Model choice can remain flexible while the surrounding research environment remains governed.

And when local inference is required, sovereignty becomes concrete in the architecture rather than simply a label attached to the service.

AI is becoming part of the research record itself.

Nuvolos treats the model as one component inside a governed workspace. Researchers can run local models where appropriate, connect external model providers when needed, work with code and data, and use emerging AI research environments such as Claude Science alongside AI coding agents such as OMP. New models and tools can be tested against the task at hand without rebuilding the stack or turning every experiment into a separate infrastructure project.

The model may get the attention, but the research record is what has to survive.

Want to test Claude Science, OMP or different models in your research workflow? Try them in Nuvolos without rebuilding the environment each time. Contact us!