docs: record the collector batch size defect
This commit is contained in:
parent
dc3115a82d
commit
1289f40532
1 changed files with 18 additions and 0 deletions
|
|
@ -99,6 +99,24 @@ ConfigMap, which the kube-prometheus-stack Grafana sidecar loads as
|
|||
`nereus.yaml`. The correlation requires the API at `LOG_LEVEL=info`, because the
|
||||
per-request log line is the only one that carries `trace_id`.
|
||||
|
||||
Two collector defects were found and fixed on the way to this result, and both
|
||||
belong in the report as corrective actions. The first is the capability problem
|
||||
above. The second only appears after a restart: `start_at: beginning` makes the
|
||||
receiver replay whole log files, the batch processor had no size cap, and Loki
|
||||
refused the resulting push.
|
||||
|
||||
```text
|
||||
HTTP 503, ResourceExhausted: grpc: received message larger than max
|
||||
(4378796 vs. 4194304)
|
||||
```
|
||||
|
||||
The exporter retries that request unchanged, so the backlog never drains.
|
||||
Capping the batch with `send_batch_size: 1000` and `send_batch_max_size: 2000`
|
||||
keeps every push under Loki's 4 MiB gRPC limit. What remains after a restart is
|
||||
a `Permanent` rejection of replayed lines older than Loki's acceptance window,
|
||||
which drops those lines rather than blocking, and leaves live traffic
|
||||
unaffected.
|
||||
|
||||
## Presentation commands
|
||||
|
||||
```bash
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue