API reference

Embed model quality testing in your CI pipeline. On Enterprise and above you can create an application and get a key from the console.

Conventions

  • Request method: all endpoints use POST with parameters in the JSON body.
  • Authentication: send the header Authorization: Bearer <api_key>。
  • Response format：{ code, msg, data }，code of 0 means success.
  • Rate limits: governed by the QPS and daily call limits on your application; exceeding them returns 1606 / 1607.
  • Callbacks: pushed to the callback URL configured on your application once the task finishes; the header X-Sign is an HMAC-SHA256 signature.

Evaluation

Create an evaluation task

POST
/api/v1/open/eval/create

Creates an evaluation task and queues it for asynchronous execution. Authentication: the header Authorization: Bearer <api_key>. If a callback URL is configured, the platform pushes the result to it once the task finishes (the X-Sign header carries an HMAC-SHA256 signature).

Example request

{
  "model_id": 1,
  "tier_id": 3,
  "strength_id": 2,
  "dim_ids": [1, 2, 3],
  "name": "Pre-launch regression"
}

Example response

{
  "code": 0,
  "msg": "ok",
  "data": {
    "task_no": "a1b2c3d4e5f6a7b8",
    "status": 0,
    "cost_quota": 6
  }
}

Query evaluation results

POST
/api/v1/open/eval/result

Looks up progress and results by task ID. While the task is running only progress is returned; once complete, per-dimension scores are included.

Example request

{
  "task_no": "a1b2c3d4e5f6a7b8"
}

Example response

{
  "code": 0,
  "msg": "ok",
  "data": {
    "task_no": "a1b2c3d4e5f6a7b8",
    "status": 2,
    "score": 86.5,
    "grade": "A",
    "dimensions": [
      {"dimension": "Accuracy", "score": 92.0, "grade": "S", "pass_num": 5, "fail_num": 1}
    ]
  }
}

Metadata

Query evaluation options

POST
/api/v1/open/eval/options

Returns every currently available dimension, intensity level and tier, for building the parameters of a create-task call.

Example request

{}

Example response

{
  "code": 0,
  "msg": "ok",
  "data": {
    "dimensions": [{"id": 1, "name": "Accuracy", "slug": "accuracy"}],
    "strengths": [{"id": 2, "name": "Standard", "level": 2}],
    "tiers": [{"id": 3, "name": "Enterprise", "level": 3}]
  }
}

Account

Query remaining quota

POST
/api/v1/open/account/quota

Returns the evaluation and API quota balances for the account that owns the current application.

Example request

{}

Example response

{
  "code": 0,
  "msg": "ok",
  "data": {
    "eval": {"total": 2000, "used": 137, "remain": 1863},
    "api": {"total": 50000, "used": 892, "remain": 49108}
  }
}