On-premise LLM deploymentinference on your own hardware, with documentation and support in Russian or English
We deploy open language models inside your infrastructure. Requests never leave for an external service, model weights and logs stay with you, and you get the diagrams, test records and operating manual in hand.
When an external API stops fitting
If none of these is your case, on-premise deployment is probably unnecessary — the section on when a perimeter is not needed covers it.
The requests contain personal data
A customer enquiry, a CV, a medical record, a message thread — all personal data. Sending it to an external service means sending it across a border.
Internal rules prohibit external services
An information security policy, critical infrastructure requirements, industry regulations. The prohibition is often explicit: data may not leave the perimeter.
Vendor dependency is too high
A key is revoked, prices change, the service is unavailable in your region — and your feature stops with it. That is a commercial risk, not a technical one.
Volume and latency no longer work
Thousands of requests a day, a hard limit on response time, operation in an isolated network with no internet access.
What stays with you after the project
A project is not delivered when the model answered in a demo, but when your team can operate it without us. That is why the list includes documents, not just accesses.
The stages and what each delivers
Assessment
We work through the scenario: which requests, who uses them, what volume per day and at peak, what counts as a correct answer. We review the hardware you already have.
Design and work plan
We fix the deployment diagram and a plan with milestones. You can still walk away at this point — and that is fine: better before the contract than after.
Deployment and pilot
We install and tune inference, run your scenarios and capture metrics under load. If quality misses the criteria, we change the model or the approach.
Handover to operation
We hand over documentation, train the team and agree the support procedure.
What we need from you for an accurate estimate
Answering these points is enough — the rest comes out in assessment.
When we will talk you out of the project
On-premise deployment is not universally better. It costs more up front and needs maintenance. This comes out in assessment, before the contract.
- Fewer than a hundred requests a day and no personal data. An external API will be cheaper than a node idling 23 hours a day.
- You need the largest closed model, not an open one. Open-weight models cannot be deployed locally in the same form — if quality rests on a closed model, a local substitute will not deliver it.
- No GPU and no hardware budget. Start with dedicated capacity on subscription or by the hour — that way you validate the scenario without buying hardware.
What people ask about local inference
Which models can you deploy?
Open models with open weights that permit deployment on customer infrastructure. We select the set for your task and quality requirements. The interface is compatible with the OpenAI standard, so a model can be replaced without rewriting the application.
How much GPU memory is required?
It depends on the model, context length and concurrency. We size it during assessment: under-sizing gives you a system that does not work, over-sizing makes you pay for idle capacity. We deliberately do not quote figures before that.
Can you deploy on hardware we already have?
Yes. If you already run GPU nodes, we deploy on them and no new hardware is required. We check whether memory and bandwidth suffice for the target model and propose options: quantisation, a different model, or extra capacity only for peak load.
Does the environment need internet access?
After installation, no. The model runs autonomously and requests do not go out. Model updates and patches follow an agreed procedure, including offline media where external networks are prohibited.
What happens when answer quality degrades?
That is a normal condition rather than an incident: the request stream shifts or the data drifts from the training distribution. The operating manual defines monitoring points, and support defines the response: run the control set, compare metrics, decide on fine-tuning or model replacement.
Who is responsible for incidents after go-live?
The incident owner is your operations team, because the system sits in your environment. Our role is second line: request procedure, response times and escalation are fixed in the support agreement. We do not leave a system without a description of what to do on failure.
Can we start with a pilot without buying hardware?
Yes. The pilot can run on dedicated capacity and the purchase decision can be made from the test record. That costs more at pilot stage and less than buying the wrong hardware.
Describe the scenario — we come back with an assessment
Tell us what the model should do, what data the requests carry and what volume you expect. You get an assessment of which delivery format fits and a work plan. If the task does not suit on-premise deployment, we will say so in the reply.
batony@agent.qq.com