Overview
Create GPUaaS pools to manage GPU resources and offer GPU-as-a-Service to your users.
Creating a New Pool
To add a new GPU pool, log in to the admin panel and click the "GPUaaS" tab on the sidebar. This will open the "GPU as a Service" page. On this page, click the "Add Pools" button to open the Add Pool modal.
In the Add Pool modal, enter the pool name in the Pool Name textbox.
Then, select the desired region and the accelerator (GPU) type from the Select Accelerator (GPU) dropdown menu.
Choose the specific GPUs to include in the pool by selecting the checkboxes next to them.
The Sharing Ratio slider controls how many vGPU can be created from each physical card.
Choose between performance and security modes to meet your needs. Performance mode allows multiple tasks to run simultaneously, enhancing speed. Security mode provides higher protections by isolating tasks.
Note that Performance mode with Spatial scheduling brings a great user experience only when the overall VRAM usage of all concurrent users of a GPU card does not exceed the physical VRAM capacity of the card so is not recommended where small VRAM capacity cards are used or the simultaneous use of large models is expected.
Adjust the Time Quantum slider, which determines how long each vGPU gets full use of the GPU resources when there are multiple active workloads. Click Enable.
If the sharing ratio is set to 1 this will not be shown.
Enabling the Pool
A modal displays a progress indicator, confirming that the GPUaaS service is being enabled.
The GPU as a service page refreshes, and the active pool's status will appear as ACTIVE.