Skip to main content

Service Exposure (2.0.1)

Service Exposure

Use custom service ports to provide external access to services running within GPUaaS Instances. Traffic routes through an assigned Service Gateway node, making your services accessible externally on a port within the 20000-29999 range.

Service Gateway Node Setup

List Service Gateway Nodes

To confirm a GPUaaS node is recognized as a potential service gateway, run haiDra list nodes. This command displays registered nodes. If your intended service gateway node appears in the list, it is ready for configuration so jump to the Service Port Management section.

Register Service Gateway Node

To configure a GPUaaS node as a service gateway and enable custom service port exposure, use the haiDra update-node-info command. Provide the node's hostname, its internal IP address, the external gateway IP, and --is-controller (if it holds the controller role) to designate it as a service gateway. This action registers or updates the node, allowing it to manage external connections for GPUaaS instances.

bash
haiDra update-node-info --hostname server55 --node-ip 192.168.1.29 --gateway-ip 101.136.12.113 --is-controller

Service Port Management

Expose Service Port

Port Mapping Range

Your exposed service port will map to an external port within the 20000-29999 range, not the original internal port.

To make a service port within a GPUaaS instance accessible externally, use the haiDra expose-service command. Specify the service name, the internal port to expose, the pod name of the GPUaaS instance, the team ID, and the protocol (TCP or UDP). The command exposes the service port externally on a dynamically assigned port within the 20000-29999 range. View this external port information in the 'Connection Info' tab of your GPUaaS instance.

bash
haiDra expose-service --service-name ollama  --port 11434 --pod-name vanilla-worker1 --team-id a6b61ce7-ea98-4d61-a160-105f278e5e33 --protocol TCP

Once exposed the user will see the service after clicking Refresh connection

Delete Service Port

To remove an externally exposed service port that is no longer needed, use the haiDra delete-service command. Provide the service name of the exposed service and its team ID. This action de-provisions the service port, making it inaccessible from outside the HostedAI environment.

haiDra delete-service --service-name ollama --team-id  a6b61ce7-ea98-4d61-a160-105f278e5e33

Current limitations:

  • The --service-name must be unique within a team. If you setup a second service with the same name the exposure will move to the pod where it was most recently set.

  • The --port value is not shown in the UI, so set a indicative service name.

  • If the service gateway IP for a Node is behind a NAT, the internal node IP will be displayed in the UI, but will be available via the service gateway IP.