Issues Querying Metrics in Signoz with Otel and PromQL Syntax
TLDR Alex is having trouble querying metrics sent to Signoz from Otel. Despite getting some metrics, Alex is unable to query certain labels. Srikanth suggested using PromQL syntax, but Alex still faces issues.
Srikanth
Tue, 14 Nov 2023 16:50:36 UTCWhat do you mean you can't query any of the labels? You just need to follow PromQL syntax.
Alex
Tue, 14 Nov 2023 22:27:26 UTCidk, i was not able to use the proper syntax, now i can use some, i'm quite frustrated because i'm not able to query something like that ```{__name__!=""}``` to list all the metrics in signoz
SigNoz Community
Indexed 1060 threads
Built with ClickHouse as datastore, SigNoz is an open-source APM to help you find issues in your deployed applications & solve them quickly | Knowledge Base powered by Struct.AI
View in App
Alex
Tue, 14 Nov 2023 11:37:45 UTChi all i'm scraping my metrics using the otel and sending them to the signoz in another cluster, but I'm facing a really hard time querying the metrics that is my otel config ```receivers: fluentforward: endpoint: 0.0.0.0:24224 hostmetrics: collection_interval: 30s scrapers: cpu: {} disk: {} filesystem: {} load: {} memory: {} network: {} prometheus: config: scrape_configs: - job_name: kubernetes-service-endpoints-dev kubernetes_sd_configs: - role: endpoints relabel_configs: - action: labelmap regex: __meta_kubernetes_service_label_(.+) - action: replace source_labels: - __meta_kubernetes_namespace target_label: kubernetes_namespace - action: replace source_labels: - __meta_kubernetes_service_name target_label: kubernetes_name - action: keep regex: .+ source_labels: - __address__ - action: keep regex: Pod;metrics separator: ; source_labels: - __meta_kubernetes_endpointslice_address_target_kind - __meta_kubernetes_endpointslice_port_name service: pipelines: metrics: exporters: - logging - otlp/metrics processors: - memory_limiter - batch receivers: - prometheus``` when i try to build some dashboard using the promql, if i type up for example i can receive something like that ```up{__temporality__="Unspecified",job_name="generic-collector",k8s_container_name="clickhouse",k8s_namespace_name="monitoring",k8s_node_name="xxxx-36537629-vmss00001w",k8s_pod_name="chi-xxxx-signoz-clickhouse-cluster-0-0-0",k8s_pod_phase="Running",k8s_pod_ready="true",k8s_pod_uid="971b1982-99df-4bb4-a5b4-3a9b51b81d8f",signoz_k8s_component="clickhouse",signoz_k8s_instance="bmcl-signoz",signoz_k8s_name="clickhouse"}``` but i cannot query for any of this labels, can someone point me to some good documentation?