UpTrajectory Review

A developer named Prabanta has published Janas, an open-source project on GitHub that claims to let you run large language models on hardware where the model weights exceed available RAM. The technique is called expert streaming: instead of loading the entire model into memory at startup, Janas keeps most of the model on SSD and pulls in only the 'experts' (specialized sub-networks inside mixture-of-experts architectures like Mixtral or DeepSeek) that are actually needed for each token. The name is a nod to MoE — Mixture of Experts — and the pitch is that a workstation with a modest amount of RAM plus a fast NVMe drive can now serve models that previously demanded datacenter-class memory footprints.

For a small-business operator, the immediate appeal is cost. If you have wanted to self-host a capable LLM for internal tooling — drafting customer emails, summarizing documents, triaging support tickets — but balked at renting a GPU instance or buying a machine with 128 GB of RAM, Janas suggests a middle path: the hardware you already have, plus a good SSD. That matters because the economics of AI right now push small teams toward API subscriptions, which means per-token fees, data leaving your premises, and a dependency on a vendor's pricing whims. A credible local option changes the calculus, especially for businesses handling sensitive client data.

What is genuinely new here is not the underlying idea — academic work on offloading model weights to storage has existed for years, and projects like llama.cpp already support partial CPU-and-disk inference — but the focus on MoE architectures specifically. Because mixture-of-experts models activate only a fraction of their parameters per token, they are unusually well suited to this trick: you might only need a few gigabytes of experts in RAM at any moment even if the full model is 100 GB or more. Whether Janas delivers usable speed in practice is the open question. Disk streaming introduces latency per token, and the Hacker News thread (only eight comments at the time of posting) will likely surface real-world benchmarks quickly.

The second-order effects cut in two directions. On one hand, tools like this compress the hardware gap between hobbyists and well-funded AI labs, which is good for competition and for businesses that value sovereignty over their data. On the other hand, 'larger than RAM' is not the same as 'fast enough to be useful.' If inference is too slow for interactive work, Janas becomes a curiosity rather than a production tool, and operators who skip the benchmarking step may waste an afternoon. There is also a maintenance risk inherent in any small open-source project: if the author moves on, you own the codebase.

Our advice: treat this as a signal, not a solution. If you are already comfortable with command-line tooling and have an NVMe drive sitting idle, clone the repo and try it with a model you care about — the GitHub page should have setup instructions and a list of supported architectures. Measure tokens per second on your actual workload before drawing conclusions. If you are not technical enough to self-host, bookmark the project and revisit it in a few months; the MoE-offloading space is moving fast, and a more polished tool will likely follow. Either way, the direction is worth watching: the cost of running capable models locally is falling, and small businesses that pay attention now will have more leverage over their AI vendors later.

Takeaway: MoE expert streaming could let small businesses self-host large LLMs on ordinary hardware, but benchmark speed on your workload before betting on it.