AI infrastructure

Challenges we address

• Data cannot be sent to external cloud services, yet the business needs AI now
• It is unclear how much GPU, memory, storage and network capacity training and inference will require
• Employees use external LLMs, which creates a risk of data leakage
• Models need to move from experiments into production

Questions and answers

Can AI infrastructure be deployed so that data never leaves external clouds?

Yes, GPU servers and clusters, on-premises LLMs and model platforms are deployed inside the customer’s perimeter, so data never leaves the customer’s own infrastructure. This approach was used in the “AI infrastructure on GPU” project, where capacity for training, inference and on-premises LLMs was built without sending data to external clouds.

Which vendors do you work with when building AI infrastructure?

We work with NVIDIA, Lenovo, Dell Technologies, Nutanix, Red Hat and SUSE, and use platforms such as Nutanix GPT-in-a-Box, Red Hat OpenShift AI and SUSE Rancher.

How do you size how much GPU, memory and network capacity training and inference will need?

Sizing is based on the customer’s specific tasks: we select a GPU platform and calculate the compute, memory, storage and networking required for model training and inference.

What if employees already use external LLMs and that creates a risk of data leakage?

For that case we protect data when employees work with external LLMs, and help move models from experiments into production on local infrastructure inside the customer’s perimeter.