Headers in Monitoring HTTP Endpoints & Otel-Collector Error
TLDR Rohit asks about providing headers in monitoring HTTP endpoints and reports errors in otel-collector logs. Srikanth mentions headers are unsupported and requires more investigation for errors.
Srikanth
Thu, 02 Mar 2023 00:28:06 UTC> 1. Is there a way to provide headers in monitoring http endpoints? One of the endpoints that I want to monitor requires an authorisation header. Not currently supported now. > 1. I am constantly seeing these errors in my otel-collector logs. Is this concerning? Can these be disabled somehow so I don’t miss the important logs? What language SDKs are you using? This happens when the metric temporality is delta. This shouldn’t happen unless you explicitly configure the SDKs to prefer delta as the spec recommended default is cumulative. If you didn’t change any SDK configuration let me know what SDK are you using otherwise you can use the cumulative temporality to not see this error.
Rohit
Thu, 02 Mar 2023 09:42:39 UTCSrikanth
1. Okay, thank you. I’ll explore other solutions.
2. I am using FastAPI (Python) - To be more specific -> opentelemetry-instrumentation-fastapi = “^0.36b0”
I wanted to understand if this error is coming from metrics or from traces? Since I’m seeing these errors in my otel-collector logs and not otel-collector-metrics logs. For metrics, I have implemented
Srikanth
Thu, 02 Mar 2023 09:57:47 UTCDid you configure the temporality through env or constructor params?
Rohit
Thu, 02 Mar 2023 10:44:51 UTCSorry for being a bit dumb here. But I didn’t really understand what you mean. This is my configuration -> In my start application code ->
```FastAPIInstrumentor.instrument_app(app)
prometheus_instrumentor = Instrumentator(
should_group_status_codes=True,
should_ignore_untemplated=True,
should_respect_env_var=True,
should_instrument_requests_inprogress=True,
excluded_handlers=["/health", "/metrics"],
env_var_name="ENABLE_METRICS",
inprogress_name="http_requests_inprogress",
inprogress_labels=True,
)
prometheus_instrumentor.instrument(app).expose(app, include_in_schema=False)```
In my environment variables ->
```OTEL_RESOURCE_ATTRIBUTES=service.name=inventory,deployment.environment=stage,version=1.0.0
OTEL_EXPORTER_OTLP_ENDPOINT=
Rohit
Fri, 03 Mar 2023 13:01:54 UTCSrikanth anything that you could maybe help me with here please?
Srikanth
Fri, 03 Mar 2023 17:26:22 UTCI am not sure what is the issue here. I will have to spend some time digging this deeper.
SigNoz Community
Indexed 1061 threads
Built with ClickHouse as datastore, SigNoz is an open-source APM to help you find issues in your deployed applications & solve them quickly | Knowledge Base powered by Struct.AI
View in App
Rohit
Wed, 01 Mar 2023 17:24:52 UTCHi Team, I’ve been experimenting with Signoz for quite a while, and hats-off to an amazing product. I have 2 things that i needed help with - 1. Is there a way to provide headers in monitoring http endpoints? One of the endpoints that I want to monitor requires an authorisation header. 2. I am constantly seeing these errors in my otel-collector logs. Is this concerning? Can these be disabled somehow so I don’t miss the important logs? ```error exporterhelper/queued_retry.go:394 Exporting failed. The error is not retryable. Dropping data. {"kind": "exporter", "data_type": "metrics", "name": "clickhousemetricswrite", "error": "Permanent error: invalid temporality and type combination; Permanent error: invalid temporality and type combination; Permanent error: invalid temporality and type combination; Permanent error: invalid temporality and type combination; Permanent error: invalid temporality and type combination; Permanent error: invalid temporality and type combination; Permanent error: invalid temporality and type combination; Permanent error: invalid temporality and type combination; Permanent error: invalid temporality and type combination; Permanent error: invalid temporality and type combination", "errorCauses": [{"error": "Permanent error: invalid temporality and type combination"}, {"error": "Permanent error: invalid temporality and type combination"}, {"error": "Permanent error: invalid temporality and type combination"}, {"error": "Permanent error: invalid temporality and type combination"}, {"error": "Permanent error: invalid temporality and type combination"}, {"error": "Permanent error: invalid temporality and type combination"}, {"error": "Permanent error: invalid temporality and type combination"}, {"error": "Permanent error: invalid temporality and type combination"}, {"error": "Permanent error: invalid temporality and type combination"}, {"error": "Permanent error: invalid temporality and type combination"}], "dropped_items": 143}
/go/pkg/mod/go.opentelemetry.io/[email protected]/exporter/exporterhelper/queued_retry.go:394
/go/pkg/mod/go.opentelemetry.io/[email protected]/exporter/exporterhelper/metrics.go:135
/go/pkg/mod/go.opentelemetry.io/[email protected]/exporter/exporterhelper/queued_retry.go:205
/go/pkg/mod/go.opentelemetry.io/[email protected]/exporter/exporterhelper/internal/bounded_memory_queue.go:61```