Viewing Host Level Metrics with SigNoz Docker App

TLDR Priya wanted to view host level metrics in SigNoz's standalone docker app. Ashu guided them to enable a dashboard and import a JSON to view metrics. After more queries, Prashant clarified how to display metrics correctly for different hosts.

Photo of Priya
Priya
Tue, 10 Oct 2023 09:27:32 UTC

Hello <#C01HWUTP4HH|general> I am using signoz standalone docker app, I can see section metrics like explained in the official website. Basically I want to see the host level metrics like cpu, memory etc. How to see that ?

Photo of Ashu
Ashu
Tue, 10 Oct 2023 09:35:57 UTC

Hi Priya That will come under `Dashboards` tab. You would need to create a hostmetrics dashboard. Check out these docs:

Photo of Priya
Priya
Tue, 10 Oct 2023 11:01:45 UTC

Hi Ashu fo the response so I have installed SigNoz on Docker Standalone and in above doc SIGNOZ_INGESTION_KEY is needed what is that how to get it ?

Photo of Ashu
Ashu
Tue, 10 Oct 2023 11:05:51 UTC

Can you check the docs under self-host section?

Photo of Priya
Priya
Tue, 10 Oct 2023 11:26:03 UTC

so I already have signoz up and running and its all all the containers up. I have instrumented my go application as well using this doc . enabled the tracer on endpoints as well so I am able to get the metrics for that endpoint. What I need is system level metrics as well where the application is running. I am sorry for these naive questions and Thank you attending these Ashu

Photo of Ashu
Ashu
Tue, 10 Oct 2023 11:40:27 UTC

Is SigNoz and your application running on the same host?

Photo of Priya
Priya
Tue, 10 Oct 2023 11:49:25 UTC

yes

Photo of Ashu
Ashu
Tue, 10 Oct 2023 12:12:25 UTC

Can you try these steps: • Go to Dashboards • Click on `+New Dashboard` • Click on `Import json` • Paste JSON from this in the window of `Import JSON` This should work. Prashant just wanted to confirm once, if the application and signoz are on the same machine, we don't need to install otel-collector separately, right?

Photo of Priya
Priya
Tue, 10 Oct 2023 12:23:24 UTC

Thanks Ashu but just to understand how exactly it will collect the host metrics via the json ?

Photo of Priya
Priya
Tue, 10 Oct 2023 12:24:47 UTC

Like for controllers i am starting a span using otel tracer and adding the attributes i need and thats coming under traces, dont i need to add cpde for host metrivs as well?

Photo of Ashu
Ashu
Tue, 10 Oct 2023 12:29:03 UTC

By defualt, when you install SigNoz, the hostmetrics receiver is enabled. You can read more about the receiver here: SigNoz installation comes with an opentelemetry collector which has the hostmetrics receiver enabled. So the data is collected and stored in signoz database. The json just enables creating the dashboard automatically. You should see a dashboard like this once it works.

Photo of Priya
Priya
Tue, 10 Oct 2023 12:59:56 UTC

Thanks Ashu it worked.

Photo of Priya
Priya
Tue, 10 Oct 2023 13:02:15 UTC

just one last question this dashboard is showing hostname as signoz_host where to change this name ?

Photo of Priya
Priya
Tue, 10 Oct 2023 13:11:32 UTC

Also is there any way to see what exactly data it is storing which is getting used in the dashboard .. just to verify if getting the stats from the host where the application is running. Also what if the application is running on different host and signoz is on different host. where is this metrics coming from how to ensure that ?..

Photo of Priya
Priya
Tue, 10 Oct 2023 13:12:01 UTC

By defualt, when you install SigNoz, the hostmetrics receiver is enabled then its collecting the metrics from the machine where signoz is running right ?

Photo of Ashu
Ashu
Tue, 10 Oct 2023 13:20:52 UTC

> By defualt, when you install SigNoz, the hostmetrics receiver is enabled then its collecting the metrics from the machine where signoz is running right ? Yes > Also what if the application is running on different host and signoz is on different host. where is this metrics coming from how to ensure that ?.. If the application is running on a different host, you would need to install otel-collector binary there and configure it to send data to the machine where SigNoz is installed. > just one last question this dashboard is showing hostname as signoz_host where to change this name ? Prashant would be able to help you on this. > Also is there any way to see what exactly data it is storing which is getting used in the dashboard .. just to verify if getting the stats from the host where the application is running. <@4K1456> is there any way to do this? I am guessing this is a clickhouse table where the data is stored.

Photo of Priya
Priya
Tue, 10 Oct 2023 13:39:03 UTC

Thank you Ashu <@4K1555> <@4K1456> if you xan help understanding above questions

Photo of Prashant
Prashant
Tue, 10 Oct 2023 13:52:37 UTC

Priya In the hostmetrics dashboard shared above by Ashu , hostname is a variable. More sources i.e. VM/K8s nodes you send data from, more hosts will be available in that option

Photo of Priya
Priya
Tue, 10 Oct 2023 13:56:47 UTC

```"variables": { "hostname": { "customValue": "", "description": "Hostname of the instance", "modificationUUID": "f8ed0d75-f0e7-4c82-978e-22bfe98a5c2b", "multiSelect": false, "name": "hostname", "queryValue": "SELECT DISTINCT(JSONExtractString(labels, 'host_name'))\nFROM signoz_metrics.time_series_v2\nWHERE not has(JSONExtractKeys(labels), 'k8s_node_name')", "selectedValue": "signoz-host", "showALLOption": false, "sort": "DISABLED", "textboxValue": "", "type": "QUERY" }``` I believe its coming from above json that we imported in dashboard, can we change it to actual host name ? how to see the data stored in click house

Photo of Priya
Priya
Tue, 10 Oct 2023 18:40:15 UTC

Lets say my go application is running in a cluster of 5 machines and need to send the machine metrics to signoz ui.. now how to change the hostname so that dashboard shows the metrics for each of the hosts

Photo of Priya
Priya
Wed, 11 Oct 2023 05:52:23 UTC

Ashu I have few more questions: 1. do we support sampling here ? (like collect traces for 1 out of 10 api calls) 2. process level metrics ( how much of cpu, memory etc is getting consumed by my application) 3. if my application is making db connection (redis connection) how to #db connections etc. Thank you for helping and answering all questions

Photo of Prashant
Prashant
Wed, 11 Oct 2023 20:46:04 UTC

> I believe its coming from above json that we imported in dashboard, can we change it to actual host name ? how to see the data stored in click house Did you import `hostmetrics-with-variable.json`? If yes, that `signoz-host` default value that is only shown once. But when hostname variable dropdown is clicked, you should see list of your VMs.

Photo of Prashant
Prashant
Wed, 11 Oct 2023 20:46:50 UTC

> Lets say my go application is running in a cluster of 5 machines and need to send the machine metrics to signoz ui.. now how to change the hostname so that dashboard shows the metrics for each of the hosts Also, in case of K8s nodes, you will need to import instead.