#support

Resolving Full Memory Issue in Pods

TLDR Itay had a full memory issue with their pods, and asked for help. Prashant advised increasing the PVC size and correctly setting TTL, and provided a helpful link.

4mo

Join the chat

Jul 21, 2023 (4 months ago)

Photo of md5-70a65c96b293c1aafa37eaaefeaefa64

Itay

04:28 PM

^^^^ adding to this the pods are in not ready/running state, and the logs suggest that the memory is full. no space available
so who to recover from this? and we use s3 so why pvc is so full, any way to clear space.

Please does anyone know anyting about this

Jul 24, 2023 (4 months ago)

Photo of md5-1899629483c7ab1dccfbee6cc2f637b9

Prashant

05:20 AM

It's easiest to increase the PVC size by few GBs.
Followed by setting TTL to appropriate values.

05:23

Prashant

05:23 AM

Trying to update TTL when disk is already full often results in issues causing downtime.

Itay

03:53 PM

Prashant Where do i set the PVC size?

Prashant

06:01 PM

Itay Docs: https://signoz.io/docs/operate/clickhouse/increase-clickhouse-pv/

SigNoz Community

Built with ClickHouse as datastore, SigNoz is an open-source APM to help you find issues in your deployed applications & solve them quickly | Knowledge Base powered by Struct.AI

Indexed 1023 threads (61% resolved)

Join Our Community

Similar Threads

Troubleshooting Memory Space Issue in Kubernetes with Signoz

Abel had trouble running signoz on Kubernetes due to 'not enough space'. Pranay provided steps to increase PV. Eventually, Abel confirmed solution after changing PV size to '50Gi'.

Issue with Helm Installation in GKE Autopilot Cluster

Kalman faced issues with helm installation with pods stuck in init state, and some crashing in a GKE autopilot cluster. Mayur provided suggestions to diagnose the issue, including checking IAM permissions and storage classes, and adjusting resource limits in the helm values. The thread is unresolved.

1mo

Resolving Log Retention and Login Issues in Clickhouse

duuZ faced issues with log retention and login in Clickhouse. nitya-signoz provided solutions for log cleanup, PVC increase, and running one replica of the query service.

9mo