#Model Storage

A Model Catalog or Model Registry stores model metadata and the location of a model artifact. The model files themselves remain in a storage backend that the inference service can access. Supported storage backends are:

• S3 object storage: a Storage Initializer downloads the model before the main container starts.
• Persistent Volume Claim (PVC): a Storage Initializer loads the model from a mounted persistent volume.
• OCI container: also called a modelcar in KServe; the container runtime uses image layers and caching to load the model efficiently.

The artifact location is entered as a Model URI when registering a model version. For project-level S3, URI, or OCI endpoints managed in the UI, see Using Connections.

#Choose a storage backend

After placing the model in storage, open AI Hub > Model Registry > Register Model and enter the URI. You can then deploy the active registered version from the Model Registry. A catalog entry may also provide an artifact URI and can be deployed directly from AI Hub > Model Catalog.

#Using S3 object storage

S3 is commonly used for model storage. The inference service needs the endpoint and credentials through a Secret attached to a ServiceAccount, or through an S3 Connection selected in the console.

#Authentication configuration

It is recommended to create a separate ServiceAccount and Secret for each project. The following example shows the KServe S3 Secret format:

apiVersion: v1
stringData:
  AWS_ACCESS_KEY_ID: YOUR_ACCESS_KEY
  AWS_SECRET_ACCESS_KEY: YOUR_SECRET_KEY
kind: Secret
metadata:
  annotations:
    serving.kserve.io/s3-endpoint: your_s3_service_ip:your_s3_port
    serving.kserve.io/s3-usehttps: '0'
  name: s3-creds
  namespace: demo-space
type: Opaque
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: sa-models
  namespace: demo-space
secrets:
  - name: s3-creds

#Deploy an S3-backed model

The following is a direct KServe example. In the console, the recommended workflow is to register storageUri as the model artifact URI and create the service from the active Model Registry version.

kind: InferenceService
apiVersion: serving.kserve.io/v1beta1
metadata:
  annotations:
    serving.kserve.io/deploymentMode: Standard
  labels:
    aml.cpaas.io/runtime-type: vllm
  name: s3-demo
  namespace: demo-space
spec:
  predictor:
    maxReplicas: 1
    minReplicas: 1
    model:
      modelFormat:
        name: transformers
      protocolVersion: v2
      resources:
        limits:
          cpu: '2'
          ephemeral-storage: 10Gi
          memory: 8Gi
        requests:
          cpu: '2'
          memory: 4Gi
      runtime: aml-vllm-0.11.2-cpu
      storageUri: s3://models/Qwen2.5-0.5B-Instruct
    securityContext:
      seccompProfile:
        type: RuntimeDefault
    serviceAccountName: sa-models

Replace the runtime with an installed ClusterServingRuntime. The storageUri must point to the model directory in the S3 bucket, and the ServiceAccount must be able to read the Secret.

#Using OCI containers for model storage

You can package a model as an OCI image and push it to an OCI-compatible registry such as Harbor or Quay. Modelcar storage is useful for offline installations and for fast startup from a registry with node-side image caching.

Record the image reference as the Model URI, for example:

oci://registry.example.com/models/qwen:1.0

The inference cluster must be able to pull the image. Configure an OCI Connection when the registry is private. For packaging and deployment steps, see Using KServe Modelcar for Model Storage.

#Using a PVC for model storage

#Upload model files to a PVC

You can serve a model from an existing PVC. Upload the files from a Workbench that has the PVC attached:

#Prerequisites

• You have access to the Alauda AI dashboard and target namespace.
• You have a running Workbench and a PVC.
• The Workbench is attached to the PVC.
• The model files are available on your local machine or in the Workbench.

For Workbench creation and PVC attachment, see Create Workbench.

#Procedure

1. In the Alauda AI dashboard, select Workbench.
2. Find a running Workbench and select Connect.
3. Open the IDE file browser. In JupyterLab, use Files; in code-server, use Explorer.
4. Navigate to the root of the attached PVC, usually the Workbench home directory. Files created there persist in the PVC.
5. Create a directory for the model, such as models/Qwen2.5-0.5B-Instruct.
6. Upload the model files. In JupyterLab, select Upload; in code-server, drag the files into the target directory.
7. Confirm that the model files appear in the target directory.

The Model URI is:

pvc://<pvc-name>/models/Qwen2.5-0.5B-Instruct

Register this URI as the artifact location for a Model Registry version. The PVC and the inference service must be in a namespace where the service can mount the claim.

#Deploy a PVC-backed model

kind: InferenceService
apiVersion: serving.kserve.io/v1beta1
metadata:
  annotations:
    serving.kserve.io/deploymentMode: Standard
  labels:
    aml.cpaas.io/runtime-type: vllm
  name: pvc-demo-1
  namespace: demo-space
spec:
  predictor:
    maxReplicas: 1
    minReplicas: 1
    model:
      modelFormat:
        name: transformers
      protocolVersion: v2
      resources:
        limits:
          cpu: '2'
          ephemeral-storage: 10Gi
          memory: 8Gi
        requests:
          cpu: '2'
          memory: 4Gi
      runtime: aml-vllm-0.11.2-cpu
      storageUri: pvc://model-pvc/models/Qwen2.5-0.5B-Instruct
    securityContext:
      seccompProfile:
        type: RuntimeDefault

Replace model-pvc and the path with the PVC and directory containing the model.

#Model URI requirements

The console validates that a Model URI has a scheme and a non-empty location. For example, the following forms are valid when supported by the deployment configuration:

• s3://<bucket>/<path>
• pvc://<pvc-name>/<optional-path>
• oci://<registry>/<repository>:<tag>
• another configured URI scheme supported by the storage initializer or serving runtime

A local path such as /models/qwen or a bare repository name is not a complete Model URI. Confirm that the chosen serving runtime supports the model format and that the runtime can access the referenced storage.