2.4.1 Release Notes

Recipe Execution Logs in User Panel — (HAI-6814)

Enhanced GPUaaS pod instance observability by introducing role-based recipe execution logs in the User Panel. Users can view high-level service configuration events (Started, Completed, and Failed), while administrators have access to detailed Ansible execution logs for troubleshooting. Execution logs are retained before Kubernetes TTL cleanup or instance decommission to ensure a permanent audit trail. Additionally, users can report failures to their system administrator with a single click via a pre-filled email containing the failure context and a direct link to the relevant log. Instance configuration details, including GPU, CPU, RAM, and resource pool, are also captured alongside logs to support faster hardware correlation and root cause analysis.

Provides a permanent audit trail of recipe activity across the platform, visible at the correct depth by role.

Multi-VG Support per Region with Automatic Storage Allocation — (HAI-6288)

Removes the single-VG-per-region constraint, allowing administrators to register multiple storage nodes in a region. Platform automatically selects the optimal VG for each storage request based on absolute free space — no user or admin input required. Includes best-effort instance-storage collocation to minimise latency. Admins gain region- and node-level storage utilization dashboards with team/volume drill-down and a standardised shared volume creation flow consistent with other platform creation workflows.

Enhanced regional storage scalability by removing the fixed storage capacity limitation, allowing administrators to expand storage online without disrupting existing infrastructure or requiring resource replacement. The update also reduces single points of storage failure while maintaining full backward compatibility, ensuring that existing single-volume group (VG) regions continue to operate without migration or reconfiguration.

Floating IP Management for GPUaaS Nodes from Admin Panel — (HAI-7087)

Introduces a dedicated Floating IPs sub-tab under Infrastructure > Nodes > Accelerator in the admin panel, replacing the previous SSH + Haidra CLI workflow. Administrators can view, create, edit, and delete floating IP mappings per region from a graphical interface with input validation (well-formed IPv4, 1:1 mapping enforcement). IP addresses are auto-discovered from per-node Rescan results or entered manually. Haidra remains the authoritative source, the admin panel reconciles its local state with Haidra on each sync.

Introduced floating IP management through the Admin Panel, enabling administrators to assign and manage floating IPs without requiring SSH access. This enhancement reduces the time required to assign a floating IP from minutes to seconds, eliminates dependency on manual CLI operations for routine IP management and minimizes the risk of configuration errors caused by unvalidated command-line entries.

High Availability for KVM Cluster Panel Nodes — (HAI-6347)

Implements automatic primary/secondary failover for HAI KVM cluster panel nodes using etcd for distributed leader election. A periodic HA agent aligns controller services and SQL Database replication to the elected leader without manual intervention. VMs can be individually set to auto-restart or not depending on whether the VM should restart automatically on failure. HA is enabled via a cluster UI toggle, enforces a 3-node minimum, and supports full disable/revert.

Eliminates single points of failure in KVM cluster management, reducing unplanned downtime for VM workloads and providing automated recovery without operator intervention on primary node failure — improving SLA reliability for hosted VM infrastructure.

Context-aware Navigation from Teams View — (HAI-4828)

Adds a "Go back" button to the instance detail page in the User Panel. Breadcrumb and back navigation now adapt based on the user's entry point: navigating from the Instances tab preserves the existing Instances > [instance name] breadcrumb; navigating from the Teams tab now correctly shows Teams > [team's instances] > [instance name] with step-by-step back navigation. Routing fix included as part of the same work.

Improved navigation within the Teams view by eliminating a dead-end user experience when accessing an instance. Users can now seamlessly navigate back, enhancing workflow continuity, reducing friction for team administrators and providing a more intuitive and consistent user experience.

Historical Instance Metrics Display Accuracy — (HAI-3707)

Fixes display of historical GPUaaS pool metrics after re-subscription. Metrics are now correctly scoped per instance creation timestamp so that re-subscribing to a pool starts from a clean baseline. Adds UI/UX enhancements for discoverability of historical instance usage. The historical view now honours the same GPU type / pool name display setting (configured by the service provider) that was in effect when the instance was active, providing display parity between active and deleted instance views.

Improved historical metrics retention for deleted instances by preserving metric values as they appeared while the instance was active. This ensures consistency between live and historical views, reducing user confusion and support requests related to billing accuracy while providing a more reliable audit trail for usage verification.

HAI-8133

[FE] Deploy Instance button is a no-op on /instances/create — click fires no request

When completing the instance creation wizard and clicking Deploy Instance, the button produced no response — no request was sent, no loading state appeared, and the page remained unchanged. The button's click handler is now correctly wired to the provisioning request. Users can self-serve VM provisioning from the console without workarounds.

HAI-8589

Bare metal instance creation doesn't filter out unavailable service types

When launching a bare metal instance, users were presented with regions and instance types that had no available hardware, leading to selections that would silently fail at deployment. The platform now filters out unsupported regions and instance types during bare metal instance creation, showing only genuinely deployable options.

HAI-8577

[FE] Frontend sends restricted instead of no access when Scheduler is configured during Resource Policy creation and edit

When creating or editing resource policies with a scheduler configured, resources set to "No Access" were silently saved as "Restricted", causing users to receive access they were explicitly meant to be denied. The platform now correctly preserves the "No Access" designation. Admins can trust that access restrictions are enforced exactly as configured.

HAI-8388

Activity progress does not show up in instance details page [REGRESSION]

When performing operations such as resizing or disk expansion on a VM instance, users were unable to see real-time progress updates on the Instance Details page. The platform now correctly retrieves and displays the activity status, completion percentage, and progress message during any long-running instance operation.

HAI-8009

Network not visible on freshly created VM cluster without restarting clusterpanel-api

When setting up a new VM cluster, configured networks did not appear on the Networks Setup tab immediately after creation, requiring a manual service restart. A data synchronisation issue during cluster initialisation has been resolved. Operators can now proceed directly from cluster creation to network configuration without any manual intervention.

HAI-7884

KVM Node Deinitialise Action Returns 404 and is non-functional

Administrators attempting to deinitialise a KVM node via the context menu received a silent failure with no error shown in the UI. The backend API endpoint for the deinitialise action was missing and has now been correctly registered. Operators can now successfully deinitialise KVM nodes, enabling clean node removal and redeployment workflows.

HAI-7574

[BE/FE] Suspending a team does not stop its running instances; suspension dialog lacks warning about active instances

When admins suspended a team, its running instances continued consuming compute resources unchecked, silently leaking costs. The platform now automatically stops all running instances when a team is suspended, and the confirmation dialog explicitly lists the number of active instances that will be terminated.

HAI-8597

[Security] Baremetal subscription SSH password stored and returned in plaintext

SSH passwords generated during baremetal node provisioning were being stored unencrypted in the database and returned in plaintext through subscription API responses. The platform now encrypts SSH passwords at rest, clears credentials when a subscription ends, and no longer includes them in standard subscription responses.

HAI-8643

[BE] Standardise and complete logout

After logging out, users' browser sessions retained a stale authentication cookie because the server did not explicitly clear it. The logout flow has been corrected so the server now deletes the session cookie and both the Admin and User panels use a consistent logout endpoint. Users are now fully signed out on every logout path.

HAI-8011

User Panel tracks one process-global Keycloak session — any /api/logout revokes the most recent login, not the caller's

In multi-user environments, signing out of the User Panel could silently revoke a different user's active session. The logout flow has been corrected to resolve and revoke the caller's own session independently. Only the requesting user's session is terminated on logout.

HAI-8339

GPUaaS in-pod nvidia-smi/nvtop report 0% usage for docker-run workloads

On GPUaaS instances running Docker-based workloads, in-pod nvidia-smi and nvtop incorrectly reported 0% GPU utilisation even under heavy load. The GPU monitoring component has been updated to detect workloads running outside its interception layer. Operators and users can now rely on standard in-pod GPU monitoring tools for accurate utilisation readings.

HAI-8187

Soft-deleted service_incompatible_models rows permanently exclude GPU pools — Custom GPUaaS service won't list in instance-create flow

When admins removed a GPU model from a Custom GPUaaS service's incompatible models list, the service would silently disappear from the instance creation flow. The pool eligibility check now correctly honours those removals. Admins and tenants can immediately create instances after updating incompatible model settings.

HAI-8166

GET /api/regions ~5 s — region LIST endpoint does per-node SSH/K8s storage-utilisation work on every request

Administrators experienced ~5-second load times every time the Regions list was opened in the admin panel, caused by live storage checks across all nodes on each page load. Storage utilisation data is now collected asynchronously. Admins can open the Regions list in under one second.

HAI-7948

[Billing] Incorrect hourly cost displayed in UI — total is overstated

Admins and tenants viewing the instance usage timeline were seeing inflated hourly costs — the billing page was mistakenly adding service charges on top of an already-combined total. The cost aggregation has been corrected to accurately sum resource and service charges without double-counting.

HAI-7885

KVM Node reboot is non-functional though node is in 'online' state

Administrators saw nodes displayed as "Online" when the backend state was "in service", causing Reboot actions to fail. The platform now correctly synchronises the displayed node status with the true backend state. Administrators can now reboot KVM nodes reliably with no misleading status labels.

HAI-7270

Accelerator Pool — Incorrect team listing and subscription status when time filter applied

On the Accelerator Pool detail page, the "Teams using this pool" tab was showing all historical subscribers with an incorrect "subscribed" status even when a time filter was applied. The platform now correctly filters the team list to active subscribers within the selected window.

HAI-8458

Shared Storage Name Uniqueness Validation Incorrect Across Teams

Teams were incorrectly blocked from using a volume name already taken by a different team. The uniqueness check has been corrected to enforce uniqueness only within each team. Teams can now independently name their Shared Storage volumes without being affected by other teams' naming choices.

HAI-8048

[BE] Region compatibility check fails for Accelerator-Workload-only regions — 'no compatible instance types found'

Admins configuring GPU-only regions encountered a false "no compatible instance types found" error during region compatibility validation, preventing those regions from being used for GPU deployments. The compatibility check now correctly recognises Accelerator-Workload instance types without requiring CPU-Workload types to also be enabled.

HAI-8042

GPU Index Mismatch After System Reboot

After a node reboot, GPU index positions could shift at the hardware level, leaving the platform's inventory out of sync and causing provisioning failures with 'GPU index already in use' errors. The platform now tracks GPUs by unique hardware identifier rather than position index. GPU pools are automatically reconciled and ready to accept workloads after node restarts.