API Endpoints

API Endpoints

This document is aimed to be used by HM-API users. It explains each of the available API endpoints, what they do, and how to use them.

Pro Tip: A lot of endpoints return data in json format, the jq utility is great to format json in human readbable way, for example:

~$ echo '{"status":"OK"}' | jq '.'
{
  "status": "OK"
}

Table of contents

HM-API operations

Get health status

GET /healthz

Gets the health status of HM-API and returns OK if everything is OK.

Example

~$ curl http://localhost:8080/healthz 
OK

Reload the HM-API config

POST /reload

Reads the according configmap from kubernetes (by default hm-api-config) and applies it. Note that often HM-API also reloads the config automatically, without this endpoint being called, for example when a new instance gets deployed.

Example

~$ curl \
  -X POST \
  http://localhost:8080/reload \
  | jq '.'
{
  "success": true
}

Job Operations

Get the currently running jobs

GET /jobs

Gets a list of all jobs that are currently running. Those may include deployments, standby-on actions, delete jobs.

Example

~$ curl \
  -u user:pass \
  http://localhost:8080/jobs \
  | jq '.'
[
  {
    "id": "hm-deploy-1-theinstance1-9",
    "status": "ACTIVE",
    "reason": ""
  }
]

Get the running jobs for one organization

GET /jobs/<orgId>

Same like the above call to /jobs, but it only returns the jobs for the specified organization id.

Example

~$ curl \
  -u user:pass \
  http://localhost:8080/jobs/1 \
  | jq '.'
[
  {
    "id": "hm-deploy-1-theinstance1-9",
    "status": "ACTIVE",
    "reason": ""
  }
]

Get details about a specific job

GET /jobs/<orgId>/<jobId>

Same like the above two calls, but only returns the status for one specific job.

Example

~$ curl \
  -u user:pass \
  http://localhost:8080/jobs/1/hm-deploy-1-theinstance1-9 \
  | jq '.'
{
  "id": "hm-deploy-1-theinstance1-9",
  "status": "ACTIVE",
  "reason": ""
}

Delete a running job, aborting it

DELETE /jobs/<orgId>/<jobId>

Takes the organization id and the job id for a running job and deletes it. This will interrupt the job in the middle of its execution. To obtain the job ids the above explained calls to list the jobs can be used. HM-API jobs are running in kubernetes jobs, this is the same like deleting the job via the kubectl delete job command.

Example

~$ curl \
  -X DELETE \
  -u user:pass \
  http://localhost:8080/jobs/1/hm-deploy-1-theinstance1-9 \
  | jq '.'
{
  "success": true
}

Instance Operations

List the existing instances

GET /instance

Retrieves all the known instances from Kubernetes and returns them as a list in json. To obtain the list of instances HM-API relies on the instance configmaps which are stored in Kubernetes, those can also be looked at via

kubectl -n metrictank get configmap -l app=instance-config

Example

~$ curl \
  -u user:pass \
  http://localhost:8080/instance \
  | jq '.'
[
  {
    "name": "instance-name",
    "org": 1
  },
  {
    "name": "another-instance",
    "org": 1
  }
]

Get the details of one specific instance

GET /instance/<orgId>/<instanceId>

This endpoint will return the configuration of the specified instance. The returned json output can be rather long in some cases. To find out what each of the configuration parameters mean, please look at the document about instance configuration.

Example

~$ curl \
  -u user:pass \
  http://localhost:8080/instance/1/theinstance1 \
  | jq '.'
{
  "org": 1,
  "name": "theinstance1",
  "plan": {
    "Name": "Medium",
    "ScaleFactor": 1,
    "Partitions": 32,
    "CPU": 100,
    "Mem": 1000
  },
  "storage": {
    "schemas": "[default]\npattern = .*\nretentions = 1s:8d:1h:2,1m:60d:6h:2,30m:2y:6h:2\n",
    "aggregations": "[default]\npattern = .*\nxFilesFactor = 0.1\naggregationMethod = avg,sum\n"
  },
  "indexRules": "[default]\npattern =\nmax-stale = 10d\n",
  "overrides": null,
  "overridesA": null,
  "overridesB": null,
  "adminKey": "keykeykeykeykey",
  "color": "a",
  "tsdbAuth": {
    "keykeykeykeykey": {
      "isAdmin": "true",
      "orgId": "1"
    },
    "anotherkey": {
      "isAdmin": "false",
      "orgId": "1"
    }
  },
  "revision": 9,
  "state": "deploying",
  "importer": null,
  "jobData": null,
  "rolloutConcurrency": 1
}

Updating/Deploying an instance

POST /instance

This endpoint is used to create new instances or update existing instances. It accepts an instance configuration and generates all the instance related Kubernetes resources based on it. To learn more about instance configurations, please refer to this.

Then it will update all the existing Kubernetes accordingly, or create them if they don’t exist yet. If a resource configuration changes then Kubernetes will update everything accordingly to reflect the latest config. f.e. an update to a Metrictank deployment causes the recreation of its pods with the latest pod definition.

Example, creating a new instance

~$ curl \
  -X POST \
  -u user:pass \
  -H 'Content-Type: application/json' \
  -d '{"org": 1, "name": "testinstance1", "plan": {"Name": "Small", "ScaleFactor": 1}}'  \
  http://localhost:8080/instance
"hm-deploy-1-testinstance1-1"

Example, updating an existing instance

To not have to write an entire instance config every time something needs to be modified it is a good idea to first obtain the config of an existing instance from HM-API, modify it, and then post it back to HM-API.

The easiest way to dothat is to store its config into a file, edit the config there, and then post the file back into HM-API.

~$ curl \
  -u user:pass \
  http://localhost:8080/instance/1/testinstance1 \
  | jq '.' > /tmp/testinstance1.json
~$ edit /tmp/testinstance1.json
~$ curl -s -X POST -u user:pass -H 'Content-Type: application/json' -d@/tmp/testinstance1.json

Compare the stored instance config to the existing resources

GET /instance/<orgId>/<instanceId>/diff

There might be situations when it’s necessary to verify if the existing instance config really reflects the current status on Kubernetes or not. There are various reasons why they could differ:

  • A resource in Kubernetes has been modified directly, bypassing HM-API
  • HM-API got updated and some default values have changed, or new defaults have been added
  • A Kubernetes version update could have added new properties to resources The diff endpoint reads the instance config from the according configmap and internally generates all the Kubernetes resource definitions for that instance based on the config, then it fetches the resources from Kubernetes and compares them against the internally generated ones. If there is a difference it gets returned.

Example

~$ kubectl \
  -n metrictank \
  edit deployment mt-read00-1-small-testinstance1-a
# editing the image to an alternative version
~$ ~$ curl \
  -u user:pass \
  http://localhost:8080/instance/1/testinstance1/diff
--- deployments: mt-read00-1-small-testinstance1-a ---
--- Old
+++ New
       "name": "metrictank",
-      "image": "us.gcr.io/metrictank-gcr/metrictank:v0.11.0",
+      "image": "us.gcr.io/metrictank-gcr/metrictank:manually_entered_image",
       "ports": [

~$

Compare a custom instance config to the existing resources

POST /instance/diff

The functionality of this endpoint is very similar to the above described diff endpoint. The only difference is that it doesn’t read the stored instance config from the configmap, but instead it takes the instance config from the post request’s body and then does the same comparisons.

Example

curl \
 -u user:pass \
 -X POST \
 -H 'Content-Type: application/json' \
 -d '{"name": "testinstance1", "org": 1, "color": "b", "plan": {"Name": "Small", "ScaleFactor": 1}}' \
 http://localhost:8080/instance/diff
--- services: mt-read-1-small-testinstance1 ---
--- Old
+++ New
    "app": "metrictank",
-   "color": "a",
+   "color": "b",
    "instance": "testinstance1",

--- services: graphite-1-small-testinstance1 ---
--- Old
+++ New
    "app": "graphite",
-   "color": "a",
+   "color": "b",
    "instance": "testinstance1",
...

List all instances with a difference between the config and the actual resources

GET /instance/statuslist

The above two sections explain how it is possible that there is a difference between the configuration of an instance and the deployed resources of an instance. The statuslist endpoint can be used to get a list of all instances where such a difference is present (any difference). This can be useful in situations where for example the default version of a component has been changed, and now the user needs to get a list of all instances where this version change has not been applied yet.

Example

~$ curl \
  -u user:pass \
  http://localhost:8080/instance/statuslist'
X   qa-shared
X   testinstance1
OK  testinstance2

This output indicates that the instances qa-shared and testinstance1 are not up to date. To get the details about what’s not up to date, the diff endpoint can be used.

Deleting an instance

DELETE /instance

This endpoint deletes an existing instance. It allows the user to choose whether the stored data should be deleted from the backend store and Kafka or not, by default it will be deleted.

Example

This example shows how to keep the data in the backend store and in Kafka while deleting an instance.

~$ curl \
  -u user:pass \
  -X DELETE \
  -H 'Content-Type: application/json' \
  -d '{"name": "testinstance1", "org": 1, "keepData": true}' \
  http://localhost:8080/instance
"hm-delete-1-testinstance1"

Note: Even if the data doesn’t get deleted from Kafka and the backend stored by HM-API, over time it gets expired and purged automatically.

Applying a modified instance configuration

PUT /instance/<orgId>/<instanceId>/update

When generating an instance config, the default settings from the HM-API config get merged with the instance specific settings. If a modification to the HM-API config has been done, either by editing its config map or by updating HM-API to a version which included a change to a default value, it is possible that existing instances don’t match the configuration anymore.

The update endpoint is useful to make an instance match the stored configuration without modifying any configuration settings. This allows a user who manages many instances to update some settings only once in the global hm-api configuration, and then call the update endpoint once for each instance to make them reflect the change.

Example

  • Updating the metrictank version from v0.11.0 to v0.11.0-152-ga7084bf
~$ kubectl -n metrictank edit configmap hm-api-config
/configmap/hm-api-config edited
  • Checking if the instance diff looks as expected. (this funcationality is described here)
~$ curl -u user:pass http://localhost:8080/instance/1/testinstance1/diff'
--- deployments: mt-write00-1-small-testinstance1, mt-read00-1-small-testinstance1-a ---
--- Old
+++ New
       "name": "metrictank",
-      "image": "us.gcr.io/metrictank-gcr/metrictank:v0.11.0",
+      "image": "us.gcr.io/metrictank-gcr/metrictank:v0.11.0-152-ga7084bf",
       "ports": [
  • Applying the change to the instance
~$ curl \
  -u user:pass \
  -X PUT \
  http://localhost:8080/instance/1/testinstance1/update
"hm-deploy-1-testinstance1-2"
  • The change is now getting applied to the instance via the normal deployment mechanism

Turning standby on

POST /instance/<orgId>/<instanceId>/standby-on

Turning standby off

POST /instance/<orgId>/<instanceId>/standby-off

Switching colors

POST /instance/<orgId>/<instanceId>/switchcolor

Starting the importer

POST /instance/<orgId>/<instanceId>/importer

Stopping the importer

DELETE /instance/<orgId>/<instanceId>/importer