SigNoz Community

SigNoz Community

#support

Issues with Monitoring HTTP Endpoints Configuration

TLDR Andre faced issues setting up monitoring for HTTP endpoints. Prashant and Srikanth provided guidance and troubleshooting. The issue was resolved after restarting Docker containers and configuring the dashboard panel.

Powered by Struct AI

1

60
10mo
Solved
Join the chat
Feb 16, 2023 (10 months ago)
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
03:18 PM
I followed the instructions, added two endpoints but nothing is showing up
https://signoz.io/docs/monitor-http-endpoints/
Ankit
Photo of md5-dbe7088320fe1d922707613e02f3420d
Ankit
04:35 PM
Prashant can you verify this?

1

04:36
Ankit
04:36 PM
Srikanth should we add httpcheck receivers at otel-collector config or otel-collector-metrics config? https://signoz.io/docs/monitor-http-endpoints/#steps-to-monitor-the-health-of-the-http-endpoints
Srikanth
Photo of md5-ce04a9988e2fd758a659dc55be6f2543
Srikanth
04:40 PM
This should ideally go to otel-collector-metrics but it also works with otel-collector config if there is only one main collector.
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
05:05 PM
moved it to otel-collector-metrics, still nothing showing up
Srikanth
Photo of md5-ce04a9988e2fd758a659dc55be6f2543
Srikanth
05:40 PM
Just to make sure we are on the same page, where are you verifying if this is working or not?
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
05:40 PM
this does not work
05:40
Andre
05:40 PM
I tried in both configuration files
Srikanth
Photo of md5-ce04a9988e2fd758a659dc55be6f2543
Srikanth
05:41 PM
How are you verifying?
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
05:41 PM
I updated the otel-collector-metrics and restarted the containter
05:41
Andre
05:41 PM
went to the interface, and nowthing shows up
Srikanth
Photo of md5-ce04a9988e2fd758a659dc55be6f2543
Srikanth
05:42 PM
> went to the interface, and nowthing shows up
Which interface are you referring to?
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
05:42 PM
Image 1 for
Srikanth
Photo of md5-ce04a9988e2fd758a659dc55be6f2543
Srikanth
05:44 PM
You are looking at the wrong place. I am not sure if you understand the functionality of the monitoring HTTP endpoint. It checks whether the HTTP call to the endpoint returns success with the status code. It doesn’t create any traces. There will be a metric name with httpcheck_status which you can verify in the dashboard panel and can set alerts on but the traces is different altogether.
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
05:46 PM
I think I understand
05:46
Andre
05:46 PM
it should query that endpoint every X seconds
05:46
Andre
05:46 PM
where can I see the statistics of that?
Srikanth
Photo of md5-ce04a9988e2fd758a659dc55be6f2543
Srikanth
05:48 PM
By statistics if you mean the metric value (either 1 or 0), you go to dashboards -> create new dashboard -> add panel -> time series -> and type httpcheck_status in the metric name text box.
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
05:55 PM
its not showing there
05:56
Andre
05:56 PM
Image 1 for
06:03
Andre
06:03 PM
Image 1 for
Prashant
Photo of md5-1899629483c7ab1dccfbee6cc2f637b9
Prashant
06:03 PM
works as expected
Image 1 for works as expected
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
06:04 PM
it does not work
Prashant
Photo of md5-1899629483c7ab1dccfbee6cc2f637b9
Prashant
06:04 PM
Andre can you share how you have overridden the config?
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
06:04 PM
yes
06:05
Andre
06:05 PM
Image 1 for
06:05
Andre
06:05 PM
Image 1 for
Prashant
Photo of md5-1899629483c7ab1dccfbee6cc2f637b9
Prashant
06:06 PM
okay, looks good.
06:06
Prashant
06:06 PM
Have you restarted the container with the updated config?
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
06:07 PM
yes
06:07
Andre
06:07 PM
Image 1 for
Prashant
Photo of md5-1899629483c7ab1dccfbee6cc2f637b9
Prashant
06:08 PM
In case of Docker, often times config changes do not reflect when restarted directly.
06:08
Prashant
06:08 PM
Try stopping => removing the container, and start again.
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
06:08 PM
what?
Prashant
Photo of md5-1899629483c7ab1dccfbee6cc2f637b9
Prashant
06:09 PM
docker stop clickhouse-setup-otel-collector-metrics-1
docker rm clickhouse-setup-otel-collector-metrics-1

./install.sh
06:10
Prashant
06:10 PM
I verified that it works in K8s.
Let me check that in Docker as well.
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
06:10 PM
restarted
06:10
Andre
06:10 PM
lets lee
06:10
Andre
06:10 PM
see
Prashant
Photo of md5-1899629483c7ab1dccfbee6cc2f637b9
Prashant
06:11 PM
wait for the interval duration.. and retry creating metrics in dashboard
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
06:12 PM
at least the option now shows up in the dropdown
06:13
Andre
06:13 PM
but still no data
06:13
Andre
06:13 PM
Image 1 for
Prashant
Photo of md5-1899629483c7ab1dccfbee6cc2f637b9
Prashant
06:15 PM
hover over the widget
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
06:17 PM
this should be generating datapoints every 5 seconds on the 1st API and every 15 seconds on the second API
06:20
Andre
06:20 PM
Image 1 for
06:20
Andre
06:20 PM
Image 1 for
06:21
Andre
06:21 PM
why are there so many?
06:21
Andre
06:21 PM
this is super hard to read
06:23
Andre
06:23 PM
how can I filter the statuses?
Prashant
Photo of md5-1899629483c7ab1dccfbee6cc2f637b9
Prashant
06:23 PM
use where condition to filter URLs, and use Legends to format the labels.
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
06:25 PM
Image 1 for
06:25
Andre
06:25 PM
does not work
06:27
Andre
06:27 PM
grouping also does not work
06:27
Andre
06:27 PM
Image 1 for
Prashant
Photo of md5-1899629483c7ab1dccfbee6cc2f637b9
Prashant
06:39 PM
grouping only works in aggregator function..
it does not work for NOOP
Andre
Photo of md5-0bac02e55c8e4418dbe908965cee18bb
Andre
06:40 PM
Image 1 for
06:40
Andre
06:40 PM
is this correct with NOOP?
06:46
Andre
06:46 PM
also is it possible that the condition for testing an endpoint is an expected json response instead of just 200?
Feb 17, 2023 (10 months ago)
Srikanth
Photo of md5-ce04a9988e2fd758a659dc55be6f2543
Srikanth
06:37 AM
No, that’s not possible as of now.

SigNoz Community

Built with ClickHouse as datastore, SigNoz is an open-source APM to help you find issues in your deployed applications & solve them quickly | Knowledge Base powered by Struct.AI

Indexed 1023 threads (61% resolved)

Join Our Community

Similar Threads

Monitoring HTTP Endpoints Health in Kubernetes Deployment

Narasimhamurthi faced issues with monitoring HTTP endpoints health and collector-metrics pod in a Kubernetes deployment. Srikanth provided guidance, and the issue was resolved after correcting the config.

44
10mo
Solved

Integrating TimescaleDB Prometheus Data with SigNoz

Milan was unable to view TimescaleDB Prometheus data in SigNoz. With suggestions from Srikanth and Prashant, they identified and fixed a configuration error by removing the unnecessary evaluation_interval setting.

1

29
6mo
Solved

Headers in Monitoring HTTP Endpoints & Otel-Collector Error

Rohit asks about providing headers in monitoring HTTP endpoints and reports errors in otel-collector logs. Srikanth mentions headers are unsupported and requires more investigation for errors.

7
9mo

Issues with SigNoz Setup and Data Persistence in AKS

Vaibhavi experienced issues setting up SigNoz in AKS, and faced data persistence issues after installation. Srikanth provided guidance on ClickHouse version compatibility and resource requirements, helping Vaibhavi troubleshoot and resolve the issue.

59
8mo
Solved

Removing Default Logs and Collecting Application Logs

Srikanth wanted to remove default logs and collect their application logs. nitya-signoz provided two solutions, and Srikanth chose the second one, which involves editing the ote-collector-config.yaml file.

2

43
6mo
Solved
Powered By