Issue Resolved: Unable to Update Metrics Retention Period

TLDR Asif was having trouble updating metrics retention period. Srikanth suggested a fresh install after deletion. Asif successfully resolved his issue after removing all docker resources and recreating everything.

Photo of Asif
Asif
Mon, 28 Aug 2023 09:12:40 UTC

<#C01HWQ1R0BC|support> Hello All, I have been facing an issue related to retention period. I am trying to change Metrics retention period but it got stuck forever. I tried clearing all traces, logs, restarting docker container, restarting server even. But after all of this it still stuck at updating state. Not even failing so that i can try again Can anyone please help?

Photo of Srikanth
Srikanth
Tue, 29 Aug 2023 03:28:14 UTC

How much do you have already stored for metrics? What are your system resources limits?

Photo of Asif
Asif
Tue, 29 Aug 2023 06:28:29 UTC

Signoz default value. May be 30 days initially. Then updated to 3. But later cleared all logs, traces, metrics just to get this settings applied but still showing same System resources 6 cpu, 12 G mem. we keep getting below logs in frontend docker container when i visit the settings page ```[29/Aug/2023:04:46:34 +0000] "GET /api/v1/settings/ttl?type=metrics HTTP/1.0" 200 101 "http://<my domain>/settings" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36" "10.145.0.20"```

Photo of Srikanth
Srikanth
Tue, 29 Aug 2023 06:44:23 UTC

TTL in ClickHouse is not very good. Since you mentioned you can delete the data, you can clean up the current installation with `rm -rf` and do a fresh install. It should succeed when you try to set ttl initially because there won’t be any data. But if you try to change it later it will probably run into the same issues again.

Photo of Asif
Asif
Tue, 29 Aug 2023 10:25:56 UTC

stopped the clickhouse container, removed all data with `rm -rf data/clickhouse/store/*` and started container again...but still showing same

Photo of Asif
Asif
Tue, 29 Aug 2023 11:20:44 UTC

this got resolved after removing all docker containers, volumes, all available docker resources & recreate everything