Resolving PodDisruptionBudgets Conflicts in Clickhouse Restart
TLDR Al encountered PodDisruptionBudgets conflicts on ClickHouse restart, which interfered with Kubernetes upgrade. Prashant assured that only one PDB should remain in the new chart. Al later confirmed successful remediation.
Jul 18, 2023 (2 months ago)
I notice when restarting clickhouse, the following events occur:
Pod "chi-signoz-clickhouse-cluster-0-0-0" matches multiple PodDisruptionBudgets. Chose "signoz-clickhouse-cluster" arbitrarily. Pod "chi-signoz-clickhouse-cluster-0-0-0" matches multiple PodDisruptionBudgets. Chose "signoz-clickhouse" arbitrarily.
This also seemed to interfere with upgrading kubernetes:
(UpgradeFailed) Drain of aks-node did not complete: Evict blocked by conflicting disruption budgets on pod chi-signoz-clickhouse-cluster-0-0-0 on node aks-node: chi-signoz-clickho use-cluster-0-0-0. See http://aka.ms/aks/debugdrainfailures Code: UpgradeFailed Message: Drain of aks-node did not complete: Evict blocked by conflicting disruption budgets on pod chi-signoz-clickhouse-cluster-0-0-0 on node aks-node: chi-signoz-clickhouse-cluster-0-0-0.
Jul 19, 2023 (2 months ago)
There should be only one PDB
Have you followed all steps from v0.23 migration docs before upgrading?
Indexed 826 threads (62% resolved)
SigNoz crashing in k8s due to ClickHouse OOM
Travis reported SigNoz crashing in k8s due to ClickHouse OOM. The team suggested increasing resources for ClickHouse, and other troubleshooting steps, but the issue remains unresolved.
Issues with SigNoz Setup and Data Persistence in AKS
Vaibhavi experienced issues setting up SigNoz in AKS, and faced data persistence issues after installation. Srikanth provided guidance on ClickHouse version compatibility and resource requirements, helping Vaibhavi troubleshoot and resolve the issue.
Error Upgrading Chart Version in Kubernetes Environment
Manikandan encountered an error when upgrading to a different chart version. Srikanth helps correct the mistake by suggesting to pull the new releases and upgrade.