Fix RabbitMQ/PostgreSQL/Zabbix meter source miswires

docs/en/changes/changes.md

@@ -65,6 +65,9 @@
 * Fix pprof task log records using seconds for a duration that is in minutes. `PprofServiceHandler.recordPprofTaskLog` used `TimeUnit.SECONDS.toMillis(task.getDuration())` for the record timestamp and time bucket, while the GraphQL API, `PprofTaskCommand` and overlap checks treat duration as minutes. The log now uses `TimeUnit.MINUTES`, the same as profile task logs, so the record TTL stays aligned with the task.
 * Fix Kafka broker `request_queue_time_ms`, `remote_time_ms`, `response_queue_time_ms` and `response_send_time_ms` reporting observation rate instead of latency. Those rules used histogram `*_count.rate`, so under load the chart showed requests per second while the metric names and Kafka's `*TimeMs` histograms are millisecond latency. The Prometheus JMX exporter `kafka-2_0_0.yml` does not expose `*_sum` for these histograms, so the rules now use the exported `quantile="0.50"` gauge.
 * Fix Kafka broker throughput and topic metrics. `meter_kafka_broker_topic_bytesin_per_second` read `kafka_server_brokertopicmetrics_bytesout_total`, so per-topic inbound traffic matched outbound; it now reads `bytesin_total`. The Prometheus JMX exporter exports Kafka's `BrokerTopicMetrics` message, byte and request meters per topic and again as the all-topics total without a `topic` label, so `meter_kafka_broker_messages_per_second`, `meter_kafka_broker_bytes_in_per_second` and `meter_kafka_broker_bytes_out_per_second` summed both and reported twice the real rate, and every `meter_kafka_broker_topic_*` metric carried an extra `topic=""` series holding the broker total, which the Topic Rates chart drew as an extra line. Broker rules now read only the all-topics total and topic rules only the per-topic series. `meter_kafka_broker_network_processor_avg_idle_percent` applied `rate` to Kafka's idle-ratio gauge, so it reported how fast the idle percentage changed; it now reports the idle percentage. Values stored before the upgrade keep the old numbers, so these charts step at the upgrade.
+* Fix RabbitMQ node singleblock allocator meters reading multiblock carriers. `meter_rabbitmq_node_allocated_singleblock_used` / `_unused` copied `kind=mbcs`, so the Singleblock charts matched Multiblock; they now use `kind=sbcs`, and `_unused` subtracts the used blocks size from the carriers size, as RabbitMQ's Erlang memory allocators dashboard does, instead of reporting the whole carriers size. Values stored before the upgrade keep the old numbers, so these charts step at the upgrade.
+* Fix PostgreSQL temporary-files rate reading temp bytes. `meter_pg_temporary_files_rate` / `meter_pg_instance_temporary_files_rate` used `pg_stat_database_temp_bytes`, so the Temporary Files / s charts showed byte throughput; they now use `pg_stat_database_temp_files`, and no chart shows the temporary-file byte throughput. Values stored before the upgrade keep the old numbers, so these charts step at the upgrade.
+* Fix Zabbix agent filesystem usage percentage reading inode usage. `meter_vm_filesystem_percentage` used `vfs.fs.inode` `pused`, so the File System Mountpoint Usage chart showed inode occupancy; it now uses the agent's `vfs.fs.size[/,pused]`, the space usage `df` and the Telegraf VM rule report, which leaves the space reserved for root out (the OTEL VM rule counts that space as used, so it reads higher on the same disk). Add `vfs.fs.size[/,pused]` to the required item keys, and drop unused `vfs.fs.inode[/,pused]` and the duplicate `vm.memory.size[pavailable]`. Values stored before the upgrade keep the old numbers, so this chart steps at the upgrade.
 
 #### UI
 * Add a Virtual GenAI evaluation-record page and evaluation-score chart in Horizon UI, so operators can inspect evaluation result, level, reason, judge model, timestamp, trace linkage, and the `gen_ai_model_evaluation_score_ppm` trend for evaluated records.

docs/en/setup/backend/backend-postgresql-monitoring.md

@@ -38,7 +38,7 @@ PostgreSQL cluster is cataloged as a `Layer: PostgreSQL` `Service` in OAP. Each
 | Conflicts Trend |      | meter_pg_conflicts_rate | The trend of the number of queries canceled due to conflicts with recovery in this database | postgres-exporter|
 | Deadlock Trend |      | meter_pg_deadlocks_rate | The trend of the number of deadlocks detected in this database | postgres-exporter|
 | Cache Hit Rate |   %   | meter_pg_cache_hit_rate | The rate of cache hit | postgres-exporter|
-| Temporary Files Trend |      | meter_pg_temporary_files_rate | The rate of total amount of data written to temporary files by queries in this database. All temporary files are counted, regardless of why the temporary file was created, and regardless of the log_temp_files setting | postgres-exporter|
+| Temporary Files Trend |      | meter_pg_temporary_files_rate | The rate of temporary files created by queries in this database. All temporary files are counted, regardless of why the temporary file was created, and regardless of the `log_temp_files` setting | postgres-exporter|
 | Checkpoint Stat Trend |      | meter_pg_checkpoint_write_time_rate </br> meter_pg_checkpoint_sync_time_rate </br> meter_pg_checkpoint_req_rate </br>meter_pg_checkpoint_timed_rate | The trend of checkpoint stat | postgres-exporter|
 | Active Sessions |      | meter_pg_active_sessions | The number of connections which state is active  | postgres-exporter|
 | Idle Sessions |      | meter_pg_idle_sessions | The number of connections which state is idle,idle in transaction or idle in transaction (aborted)  | postgres-exporter|

oap-server/analyzer/meter-analyzer-scripts-test/src/test/resources/scripts/mal/test-otel-rules/postgresql/postgresql-instance.data.yaml

@@ -154,7 +154,7 @@ input:
         service_instance_id: test-instance
         mode: user
       value: 100.0
-  pg_stat_database_temp_bytes:
+  pg_stat_database_temp_files:
     - labels:
         datname: test-value
         host_name: test-host

oap-server/analyzer/meter-analyzer-scripts-test/src/test/resources/scripts/mal/test-otel-rules/postgresql/postgresql-service.data.yaml

@@ -76,7 +76,7 @@ input:
         service_instance_id: test-instance
         host_name: test-host
       value: 100.0
-  pg_stat_database_temp_bytes:
+  pg_stat_database_temp_files:
     - labels:
         service_instance_id: test-instance
         host_name: test-host

oap-server/analyzer/meter-analyzer-scripts-test/src/test/resources/scripts/mal/test-otel-rules/rabbitmq/rabbitmq-node.data.yaml

@@ -121,6 +121,21 @@ input:
         usage: carriers_size
         kind: mbcs_pool
       value: 100.0
+    # Distinct from mbcs to catch copy-paste.
+    - labels:
+        cluster: test-cluster
+        node: test-node
+        alloc: test-value
+        usage: blocks_size
+        kind: sbcs
+      value: 150.0
+    - labels:
+        cluster: test-cluster
+        node: test-node
+        alloc: test-value
+        usage: carriers_size
+        kind: sbcs
+      value: 200.0
   rabbitmq_process_resident_memory_bytes:
     - labels:
         cluster: test-cluster
@@ -236,7 +251,7 @@ expected:
       - labels:
           cluster: 'rabbitmq::test-cluster'
           node: test-node
-        value: 10000.0
+        value: 9000.0
   meter_rabbitmq_node_allocated_unused_percent:
     entities:
       - scope: SERVICE_INSTANCE
@@ -247,7 +262,7 @@ expected:
       - labels:
           cluster: 'rabbitmq::test-cluster'
           node: test-node
-        value: 0.0
+        value: 1000.0
   meter_rabbitmq_node_allocated_used_bytes:
     entities:
       - scope: SERVICE_INSTANCE
@@ -258,7 +273,7 @@ expected:
       - labels:
           cluster: 'rabbitmq::test-cluster'
           node: test-node
-        value: 300.0
+        value: 450.0
   meter_rabbitmq_node_allocated_unused_bytes:
     entities:
       - scope: SERVICE_INSTANCE
@@ -269,7 +284,7 @@ expected:
       - labels:
           cluster: 'rabbitmq::test-cluster'
           node: test-node
-        value: 0.0
+        value: 50.0
   meter_rabbitmq_node_allocated_total_bytes:
     entities:
       - scope: SERVICE_INSTANCE
@@ -280,7 +295,7 @@ expected:
       - labels:
           cluster: 'rabbitmq::test-cluster'
           node: test-node
-        value: 300.0
+        value: 500.0
   meter_rabbitmq_node_process_resident_memory_bytes:
     entities:
       - scope: SERVICE_INSTANCE
@@ -303,7 +318,7 @@ expected:
           alloc: test-value
           cluster: 'rabbitmq::test-cluster'
           node: test-node
-        value: 300.0
+        value: 500.0
   meter_rabbitmq_node_allocated_multiblock_used:
     entities:
       - scope: SERVICE_INSTANCE
@@ -363,7 +378,7 @@ expected:
           alloc: test-value
           cluster: 'rabbitmq::test-cluster'
           node: test-node
-        value: 100.0
+        value: 150.0
   meter_rabbitmq_node_allocated_singleblock_unused:
     entities:
       - scope: SERVICE_INSTANCE
@@ -375,4 +390,4 @@ expected:
           alloc: test-value
           cluster: 'rabbitmq::test-cluster'
           node: test-node
-        value: 100.0
+        value: 50.0

oap-server/analyzer/meter-analyzer-scripts-test/src/test/resources/scripts/mal/test-zabbix-rules/agent.data.yaml

@@ -55,18 +55,23 @@ input:
         host: test-value
         2: pused
       value: 100.0
-  vfs_fs_inode:
+  vfs_fs_size:
     - labels:
         1: test-value
+        2: used
         host: test-value
-        2: pused
-      value: 100.0
-  vfs_fs_size:
+      value: 40.0
     - labels:
         1: test-value
-        2: test-value
+        2: total
         host: test-value
       value: 100.0
+    # Distinct from used / total * 100: pused excludes the root-reserved space.
+    - labels:
+        1: test-value
+        2: pused
+        host: test-value
+      value: 42.0
   vfs_dev_read:
     - labels:
       value: 100.0
@@ -184,7 +189,7 @@ expected:
       - labels:
           1: test-value
           host: test-value
-        value: 100.0
+        value: 42.0
   meter_vm_vfs_fs_size:
     entities:
       - scope: SERVICE
@@ -193,7 +198,12 @@ expected:
     samples:
       - labels:
           1: test-value
-          2: test-value
+          2: used
+          host: test-value
+        value: 40.0
+      - labels:
+          1: test-value
+          2: total
           host: test-value
         value: 100.0
   meter_vm_disk_read:

oap-server/server-starter/src/main/resources/otel-rules/postgresql/postgresql-instance.yaml

@@ -84,7 +84,7 @@ metricsRules:
   - name: instance_cache_hit_rate
     exp: (pg_stat_database_blks_hit*100 / (pg_stat_database_blks_read + pg_stat_database_blks_hit)).sum(['datname','host_name','service_instance_id'])
   - name: instance_temporary_files_rate
-    exp: pg_stat_database_temp_bytes.sum(['datname','host_name','service_instance_id']).rate('PT1M')
+    exp: pg_stat_database_temp_files.sum(['datname','host_name','service_instance_id']).rate('PT1M')
 
   ## checkpoint
   - name: instance_checkpoint_write_time_rate

oap-server/server-starter/src/main/resources/otel-rules/postgresql/postgresql-service.yaml

@@ -64,7 +64,7 @@ metricsRules:
   - name: cache_hit_rate
     exp: (pg_stat_database_blks_hit*100 / (pg_stat_database_blks_read + pg_stat_database_blks_hit)).avg(['service_instance_id','host_name'])
   - name: temporary_files_rate
-    exp: pg_stat_database_temp_bytes.sum(['service_instance_id','host_name']).rate('PT1M')
+    exp: pg_stat_database_temp_files.sum(['service_instance_id','host_name']).rate('PT1M')
 
   ## checkpoint
   - name: checkpoint_write_time_rate

oap-server/server-starter/src/main/resources/otel-rules/rabbitmq/rabbitmq-node.yaml

@@ -75,6 +75,6 @@ metricsRules:
   - name: allocated_multiblock_pool_unused
     exp: erlang_vm_allocators.tagEqual('usage' , 'carriers_size' , 'kind', 'mbcs_pool').sum(['cluster', 'node', 'alloc'])
   - name: allocated_singleblock_used
-    exp: erlang_vm_allocators.tagEqual('usage' , 'blocks_size' , 'kind', 'mbcs').sum(['cluster', 'node', 'alloc'])
+    exp: erlang_vm_allocators.tagEqual('usage' , 'blocks_size' , 'kind', 'sbcs').sum(['cluster', 'node', 'alloc'])
   - name: allocated_singleblock_unused
-    exp: erlang_vm_allocators.tagEqual('usage' , 'carriers_size' , 'kind', 'mbcs').sum(['cluster', 'node', 'alloc'])
+    exp: erlang_vm_allocators.tagEqual('usage' , 'carriers_size' , 'kind', 'sbcs').sum(['cluster', 'node', 'alloc']) - erlang_vm_allocators.tagEqual('usage' , 'blocks_size' , 'kind', 'sbcs').sum(['cluster', 'node', 'alloc'])

oap-server/server-starter/src/main/resources/zabbix-rules/agent.yaml

@@ -36,17 +36,15 @@ requiredZabbixItemKeys:
   - system.cpu.util[,user]
   # memory
   - vm.memory.size[available]
-  - vm.memory.size[pavailable]
   - vm.memory.size[total]
-  - vm.memory.size[pavailable]
   # swap
   - system.swap.size[,free]
   - system.swap.size[,total]
   - system.swap.size[,pused]
   # file
-  - vfs.fs.inode[/,pused]
   - vfs.fs.size[/,total]
   - vfs.fs.size[/,used]
+  - vfs.fs.size[/,pused]
   - vfs.dev.read[,ops,avg1]
   - vfs.dev.write[,ops,avg1]
 
@@ -78,9 +76,9 @@ metrics:
     exp: system_swap_size.tagEqual('2', 'pused')
   # file
   - name: filesystem_percentage
-    exp: vfs_fs_inode.tagEqual('2', 'pused').avg(['1', 'host'])
+    exp: vfs_fs_size.tagEqual('2', 'pused').avg(['1', 'host'])
   - name: vfs_fs_size
-    exp: vfs_fs_size.avg(['1', '2', 'host'])
+    exp: vfs_fs_size.tagNotEqual('2', 'pused').avg(['1', '2', 'host'])
   - name: disk_read
     # `* 1024` is for adapting to `divide 1024` by the VM UI template configuration, which converts `byte`->`kb` because of OTEL/Prometheus Node exporter.
     exp: vfs_dev_read * 1024