#support

Issues with Azure Prometheus Metrics Scraping

TLDR Alex is encountering problems scraping metrics from Azure Prometheus and shared their config details. Srikanth suggested a possible issue with the target's accessibility, although Alex affirmed its accessibility despite the lack of a /metrics endpoint in Azure. The issue remains unresolved.

Join the chat

Nov 06, 2023 (3 weeks ago)

Photo of md5-94c93df7325e8fde185c76c659656ee9

Alex

06:05 PM

hi all,
I didn't see any question related to it and maybe someone else is facing the same issue, I'm using the Azure Prometheus Managed instance and I'm trying to scrape the metrics from there, I have the config below in the signoz-otel-collector-metrics configmap.

prometheus:    
    config:
      scrape_configs:
      - job_name: "prometheus-xxx"
        scrape_interval: 90s
        oauth2:
          client_id: 'xxxx'
          client_secret: 'xxxx'
          token_url: 'https://login.microsoftonline.com/xxxxx/oauth2/token'
        static_configs:
          - targets: [""]

in the pod log i can see the error below:

message 2023-11-06T17:59:34.651Z  warn  internal/transaction.go:114 Failed to scrape Prometheus endpoint  {"kind": "receiver", "name": "prometheus", "data_type": "metrics", "scrape_timestamp": 1699293574641, "target_labels": "{__name__=\"up\", instance=\"", job=\"prometheus-xxx\"}"}

any tip here?

Nov 08, 2023 (3 weeks ago)

Photo of md5-ce04a9988e2fd758a659dc55be6f2543

Srikanth

02:04 PM

This error means that signoz-otel-collector is not able to access the configured target. Is this target accessible from the signoz instance?

Nov 09, 2023 (2 weeks ago)

Alex

10:57 AM

yes, it's accessible but Azure dont have the /metrics endpoint i asked them if we can have a query with the /metrics and then i try to do it

Nov 11, 2023 (2 weeks ago)

Alex

05:07 PM

https://prometheus.io/docs/prometheus/latest/querying/api/#otlp-receiver
seems the azure cannot support the otlp receiver, no idea how can i scrap the metrics, if someone knows, please let me know

SigNoz Community

Built with ClickHouse as datastore, SigNoz is an open-source APM to help you find issues in your deployed applications & solve them quickly | Knowledge Base powered by Struct.AI

Indexed 1023 threads (61% resolved)

Join Our Community

Similar Threads

Issue with Helm Query-Service Installation

Juan experiences issues where helm query-service installation fails. After discussions and diagnosis with Srikanth and Prashant, the issue gets solved with a pull request from Github.

2mo

Issue Sending Telemetry Data to Signoz Otel Collector

Sudhansu has problems sending telemetry data from a spring-petclinic sidecar app using the provided configuration. Prashant suggested checking the logs and generating traces but the issue is unresolved.

2mo

Troubleshooting Clickhouse Exporter Error

ELLIOTTCABLE encountered an error with Clickhouse exporters, likely due to a misconfiguration hinted by Srikanth. This was confirmed when ELLIOTTCABLE realized that Clickhouse's HTTP protocol was not implemented. The user subsequently inquired about HTTP-protocol support or alternative configurations.

1mo