Skip to content

Add Ryzen AI NPU/Hybrid model selection for Hy-MT2-7B #8

Description

@HoangVung

Title: Add Ryzen AI NPU/Hybrid support for Hy-MT2-7B model testing

Problem

The app currently works with the "AMD Lemonade / TurnkeyML" backend and can detect the available Lemonade model:

Hy-MT2-7B-GGUF-Q4_K_M-VulkanClean

However, this model appears to be a GGUF/VulkanClean variant, which likely runs on Vulkan/iGPU rather than the Ryzen AI NPU.

I want to test and customize the Hy-MT2-7B model path so the app can use Ryzen AI NPU acceleration when available.

Current Behavior

The app shows:

Backend: AMD Lemonade / TurnkeyML
Model: Hy-MT2-7B-GGUF-Q4_K_M-VulkanClean (Lemonade)
Base URL: http://127.0.0.1:13305/api/v1

Translation can be started through the Local LLM Translation page, but there is no clear way to confirm or select whether the model is running on:

  • CPU
  • Vulkan/iGPU
  • Ryzen AI NPU
  • Hybrid NPU + iGPU

Expected Behavior

Add support for testing Hy-MT2-7B with Ryzen AI NPU/Hybrid execution.

The app should allow the user to:

  1. Detect available Lemonade models from:

http://127.0.0.1:13305/api/v1/models

  1. Distinguish model/runtime variants, for example:

GGUF / Vulkan / iGPU
ONNX / OGA / NPU
Hybrid / Ryzen AI

  1. Select a Hy-MT2-7B NPU or Hybrid model if available.

  2. Save the selected model into the runtime config.

  3. Show clearly on the UI which runtime is being used:

AMD Lemonade - Vulkan/iGPU
AMD Lemonade - Ryzen AI NPU
AMD Lemonade - Hybrid NPU+iGPU

  1. Provide a small test button to verify the selected model with a short Chinese-to-Vietnamese translation, for example:

今天我们学习导数。

Expected output:

Hôm nay chúng ta học đạo hàm.

Suggested Implementation

Please keep the change minimal and avoid refactoring unrelated parts of the app.

Suggested steps:

  1. Only inspect the files related to runtime model configuration, Lemonade backend selection, and the UI model settings page.
  2. Add model variant detection based on model name keywords such as:

NPU
Hybrid
OGA
ONNX
RyzenAI
Vulkan
GGUF

  1. Add or improve the model selector so the user can select the exact Lemonade model returned by "/models".
  2. Persist the selected model in the existing runtime config.
  3. Update the Local LLM Translation page header to display the selected backend/model/runtime clearly.
  4. Add a quick test action that sends one short request to the selected backend/model.
  5. Do not change the translation pipeline, parser, subtitle logic, or file handling unless strictly necessary.

Validation

Run only the relevant checks.

Manual test:

  1. Start Lemonade server at:

http://127.0.0.1:13305/api/v1

  1. Open the app:

http://localhost:3000

  1. Go to the model configuration page.
  2. Refresh or load the Lemonade model list.
  3. Select a Hy-MT2-7B NPU/Hybrid model if available.
  4. Save configuration.
  5. Return to the Local LLM Translation page.
  6. Confirm the header displays the selected runtime/model.
  7. Run a short translation test.
  8. Check Windows Task Manager Performance tab to see whether NPU usage increases when using an NPU/Hybrid model.

Notes

If only the following model is available:

Hy-MT2-7B-GGUF-Q4_K_M-VulkanClean

then the app should clearly indicate that this is likely a GGUF/Vulkan/iGPU path, not an NPU path.

The app should not assume that any Hy-MT2-7B model automatically uses NPU. It should display the detected runtime type based on model name or backend metadata when available.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions