【与达梦同行】监控系列（四）Prometheus的自定义配置DM的告警监控项（短信/邮件）

一、概括

1. 整体的搭建以及配置前面都已经写了详细的步骤，现在对这个图应该有初步了解了。

2. Prometheus告警是通过自身的Alertmanager组件实现的，所以需要搭建Alertmanager以及配置相关的告警规则

二、安装告警alertmanager

1. alertmanager下载

Prometheus是一个划分平台，metrics的收集和存储与警报是分开的，警报是由Alertmanager负责，这是监控环境的独立部分。警报的规则是在Prometheus server上进行定义的，这些规则可以触发时间，然后将其传到alertmanager，alertmanager随后决定如何处理各自的警报，处理复制之类的问题，并决定在发送警报时使用什么机制：实时消息、电子邮件或者是其他钉钉、微信等工具。

prometheus --> 触发阈值 --> 超出持续时间 --> alertmanager --> 分组|抑制|静默 --> 媒介类型 --> 邮件|钉钉|微信等

功能

• 分组(group)：将类似性质的告警合并为单个通知，比如网络通知、主机通知、服务通知。
• 静默(silences)：是一种简单的特定时间静默的机制，例如：服务器要升级维护可以先设置这个时间段告警静默
• 抑制(inhibition)：当告警发出后，停止重复发送由此告警引发的其他告警即合并一个故障引起多个报警时间，可以消除冗余告警

下载地址：https://prometheus.io/download/ ，下载相应版本，安装到服务器上官网提供的是二进制版，解压就能用，不需要编译。【版本也可使用文章末尾的资料包】
根据环境下载对应的版本，x86环境的话用这个alertmanager-0.24.0.linux-amd64.tar.gz

2. alertmanager安装

## 1. 新建目录
[root@VM-24-17-centos opt]# mkdir /opt/jiankong/jiankongSoft -p
[root@VM-24-17-centos opt]# cd /opt/jiankong/jiankongSoft
## 2. 上传压缩包 并解压
[root@server ~]# tar -zxvf  alertmanager-0.24.0.linux-amd64.tar.gz -C /opt/jiankong
## 3. 重命名alertmanager文件名
[root@localhost jiankongSoft]# cd /opt/jiankong
[root@localhost jiankong]# mv alertmanager-0.24.0.linux-amd64 alertmanager-0.24.0

3. alertmanager启动

## 5. alertmanager
[root@localhost alertmanager-0.24.0]# cd /opt/jiankong/alertmanager-0.24.0
## 6. 启动服务
[root@localhost alertmanager-0.24.0]# nohup ./alertmanager > ./alertmanager.log 2>&1 &
## 查看端口默认是9093
[root@server ~]# lsof -i:9093##  通过浏览器访问http://被监控端IP:9093 查看地址
http://192.168.145.161:9093

4. alertmanager注册服务（可选）

## 1.编辑服务自启动文件
[root@master1 ~]#vim /lib/systemd/system/alertmanager.service
## 提示 这里用的是9091端口
[root@master1 ~]#cat /lib/systemd/system/alertmanager.service[Unit]
Description=Alertmanager
After=network.target
[Service]
Type=simple
User=root
Group=root
WorkingDirectory=/opt/jiankong/alertmanager-0.24.0
ExecStart=/opt/jiankong/alertmanager-0.24.0/alertmanager --web.listen-address="0.0.0.0:9093"
Restart=on-failure
[Install]
WantedBy=multi-user.target## 2. 更新systemd服务信息
[root@master1 ~]# systemctl daemon-reload
## 3. 服务开机自动启动
[root@master1 ~]# systemctl enable --now alertmanager.service 
## 4.启动服务
[root@master1 ~]# systemctl start alertmanager## 其他的备用命令
[root@master1 ~]# systemctl enable alertmanager.service
[root@master1 ~]# systemctl stop alertmanager.service
[root@master1 ~]# systemctl restart alertmanager.service
[root@master1 ~]# systemctl status alertmanager.service
[root@master1 ~]# journalctl -u alertmanager -f -n 500

三、prometheus配置告警配置

## 1. 进入到prometheus目录
[root@localhost prometheus-2.40.5]# cd /opt/jiankong/prometheus-2.40.5
## 2. 编译prometheus.yml配置文件
[root@VM-24-17-centos prometheus-2.35.0]# vim prometheus.yml ##yml格式 注意文件对齐方式 
# my global config         2.1. 仅以将值修改为60s
global:scrape_interval: 60s # Set the scrape interval to every 15 seconds. Default is every 1 minute.evaluation_interval: 60s # Evaluate rules every 15 seconds. The default is every 1 minute.# scrape_timeout is set to the global default (10s).# Alertmanager configuration 2.2. 添加告警平台的地址
alerting:alertmanagers:- static_configs:- targets:- 127.0.0.1:9093# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
# 2.3. 配置读取规则的路径
rule_files: - "rules/*.yml"# - "second_rules.yml"# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:# The job name is added as a label `job=` to any timeseries scraped from this config.- job_name: "prometheus"# metrics_path defaults to '/metrics'# scheme defaults to 'http'.static_configs:- targets: ["localhost:9091"]- job_name: "dm_dem_node"metrics_path: "/dem/metrics"static_configs:- targets: ["192.168.145.162:8080"]- job_name: "dm_node_single"static_configs:- targets: ["192.168.145.162:9100"]labels:origin_prometheus: '单机测试'- job_name: "dm_db_single"metrics_path: "/prometheus"static_configs:- targets: ["192.168.145.162:9200"]labels:cluster_name: '单机测试'

四、配置prometheus的rule文件

## 3。 配置rules规则 
[root@VM-24-17-centos prometheus-2.35.0]# mkdir rules## 4. 新建custom_test.yml规则文件,内容如下，也可去文末资料包中直接导入.（按需调整）
[root@VM-24-17-centos prometheus-2.35.0]# vim custom_test.yml

groups:
- name: 主机状态-监控告警rules:- alert: 主机失联expr: up == 0for: 15slabels:severity: 严重annotations:  description: "主机：{{ $labels.instance }} 服务宕机"summary: "主机：{{ $labels.instance }}:服务器超过1m无法连接"- alert: 集群发生切换expr: dmdbms_switching_occurs == 0for: 15slabels:severity: 严重annotations:description: "主机：{{ $labels.instance }} 数据库模式切换"summary: "主机：{{ $labels.instance }}:数据库发生模式切换，疑似发生集群切换请核实"- alert: 数据库状态转换expr: dmdbms_db_status_occurs == 0for: 15slabels:severity: 警告annotations:description: "主机：{{ $labels.instance }} 数据库状态转换"summary: "主机：{{ $labels.instance }}:数据库OPEN状态变化，疑似发生状态转换请核实"- alert: 数据库连接数过高expr: dmdbms_session_percentage*100 > 80for: 15slabels:severity: 警告annotations:description: "主机：{{ $labels.instance }} 数据库连接数过高"summary: "主机：{{ $labels.instance }}:数据库的连接数超过80%，请确认是否正常"- alert: 数据库存在锁等待expr: dmdbms_dead_lock_num_info >0for: 15slabels:severity: 警告annotations:description: "主机：{{ $labels.instance }} 数据库存在锁等待"summary: "主机：{{ $labels.instance }}:数据库存在锁等待，请确认是否正常"    - alert: cpu使用率过高expr: 100 * (1 - avg(irate(node_cpu_seconds_total{mode="idle"}[2m])) by(instance)) > 80for: 15slabels:severity: 警告annotations:description: "主机：{{ $labels.instance }} cpu使用率过高"summary: "主机：{{ $labels.instance }}:cpu使用率过高，超过80%"- alert: 内存使用率过高expr: (1 - (node_memory_MemAvailable_bytes / (node_memory_MemTotal_bytes)))* 100 > 80for: 15slabels:severity: 警告                        annotations:description: "主机：{{ $labels.instance }} 内存使用率过高"summary: "主机：{{ $labels.instance }}:内存使用率过高，超过80%"- alert: 数据库内存池过高expr: dmdbms_memory_curr_pool_info/dmdbms_memory_total_pool_info*100 > 0.32for: 15slabels:severity: 警告annotations:description: "主机：{{ $labels.instance }} 内存池使用率过高"summary: "主机：{{ $labels.instance }}:内存池使用率过高，超过80%"- alert: 表空间使用率超过80%expr: ((dmdbms_tablespace_size_total_info - dmdbms_tablespace_size_free_info)/dmdbms_tablespace_size_total_info)* 100 > 80for: 15slabels:severity: 警告annotations:description: "主机：{{ $labels.instance }} 存在表空间使用率超过80%"summary: "主机：{{ $labels.instance }}: 存在表空间使用率超过80%，请核实"   - alert: 磁盘分区使用率超过80%expr: (node_filesystem_size_bytes{fstype=~"ext.*|xfs|nfs",mountpoint !~".*(pod|docker).*"}-node_filesystem_free_bytes{fstype=~"ext.*|xfs|nfs",mountpoint !~".*(pod|docker).*"}) *100/(node_filesystem_avail_bytes {fstype=~"ext.*|xfs|nfs",mountpoint !~".*(pod|docker).*"}+(node_filesystem_size_bytes{fstype=~"ext.*|xfs|nfs",mountpoint !~".*(pod|docker).*"}-node_filesystem_free_bytes{fstype=~"ext.*|xfs|nfs",mountpoint !~".*(pod|docker).*"})) >80for: 15slabels:severity: 警告annotations:description: "主机：{{ $labels.instance }} 磁盘分区使用率超过80%"summary: "主机：{{ $labels.instance }}: 磁盘分区使用率超过80%，请核实"- alert: 数据库存在事务等待expr: dmdbms_trx_info >0for: 15slabels:severity: 警告annotations:description: "主机：{{ $labels.instance }} 数据库存在事务等待"summary: "主机：{{ $labels.instance }}:数据库存在事务等待，请确认是否正常"- alert: 定时备份错误数expr: dmdbms_joblog_error_alarm == 0for: 15slabels:severity: 警告annotations:description: "主机：{{ $labels.instance }} 数据库定时备份发生错误"summary: "主机：{{ $labels.instance }}:数据库定时备份发生错误，请确认是否正常"- alert: 会话超时数expr: dmdbms_waiting_session >= 1for: 15slabels:severity: 警告        annotations:description: "主机：{{ $labels.instance }} 数据库会话超时数"summary: "主机：{{ $labels.instance }}:数据库会话超时数，请确认是否正常"    

## 5.修改完成以后进行配置文件校验，避免因为空格等问题导致出错
[root@localhost prometheus-2.40.5]# ./promtool check config prometheus.yml 
Checking prometheus.ymlSUCCESS: 1 rule files foundSUCCESS: prometheus.yml is valid prometheus config file syntaxChecking rules/custom_test.ymlSUCCESS: 13 rules found

五、重启prometheus并进行连接校验

## 6.重启prometheus
[root@localhost prometheus-2.40.5]# ps -ef|grep prometheus
root       30944       1  0 13:09 ?        00:00:06 /opt/jiankong/prometheus-2.40.5/prometheus 
[root@localhost prometheus-2.40.5]# kill -9 30944
[root@localhost prometheus-2.40.5]# nohup ./prometheus --config.file="prometheus.yml" --web.listen-address="0.0.0.0:9091" > ./prometheus.log 2>&1 &## 5.访问prometheus，查看是否数据拉取成功
http://101.42.236.144:9091/

六、（邮件告警）alertmanager配置邮件告警

## 1. alertmanager
[root@localhost alertmanager-0.24.0]# cd /opt/jiankong/alertmanager-0.24.0
## 2. 修改配置文件alertmanager.yml
[root@localhost alertmanager-0.24.0]# vim alertmanager.ymlglobal:  #smtp与163邮箱一致。查看文末其他163邮箱开启POP配置的申请即可smtp_smarthost: 'smtp.163.com:25'smtp_from: '1393592xxxx@163.com'smtp_auth_username: '1393592xxxx@163.com'smtp_auth_password: 'YONUZCAGJHIUYXXX'smtp_require_tls: false 
templates:- './template/alertmanager-*.tmpl'    # 读取自定义通知模板定义的文件。      
route:group_by: ['alertname']group_wait: 1m     # 一个新分组等待发送报警的时间group_interval: 15m  #已经发送成功了报警的组,有新增alert加入组时下一次发送报警的时间repeat_interval: 1h # 报警无变化情况下,重复发送告警时间。默认1h#resolve_timeout: 5m # 该时间内未收到报警则视为问题解决receiver: 'email'
receivers:- name: 'email'email_configs:- to: 'xxx@163.com' #配置邮件发送给谁，多人的话逗号分隔html: '{{ template "email.to.html" . }}' # HTML模板文件正文send_resolved: true
inhibit_rules:  #告警抑制- source_match:severity: '严重'target_match:severity: '警告'equal: ['alertname', 'job', 'instance']

七、（邮件告警）alertmanager配置邮件模板

## 3。 新建template目录
[root@VM-24-17-centos alertmanager-0.24.0# mkdir template## 4. 新建alertmanager-email.tmpl规则文件,内容如下，也可去文末资料包中直接导入.
[root@VM-24-17-centos alertmanager-0.24.0# cd template
[root@VM-24-17-centos template]# vim alertmanager-email.tmpl

{{ define "email.to.html" }}
{{- if gt (len .Alerts.Firing) 0 -}}
{{- range $index, $alert := .Alerts -}}
========= <span style=color:red;font-size:36px;font-weight:bold;> 监控告警 span>=========<br><span style=font-size:20px;font-weight:bold;> 告警程序:span>     Alertmanager <br>
<span style=font-size:20px;font-weight:bold;> 告警类型:span>    {{ $alert.Labels.alertname }} <br>
<span style=font-size:20px;font-weight:bold;> 告警级别:span>    {{ $alert.Labels.severity }} 级 <br>
<span style=font-size:20px;font-weight:bold;> 故障主机:span>    {{ $alert.Labels.instance }} {{ $alert.Labels.device }} <br>
<span style=font-size:20px;font-weight:bold;> 告警主题:span>    {{ .Annotations.summary }} <br>
<span style=font-size:20px;font-weight:bold;> 告警详情:span>    {{ $alert.Annotations.message }}{{ $alert.Annotations.description}} <br>
<span style=font-size:20px;font-weight:bold;> 主机标签:span>    {{ range .Labels.SortedPairs  }} <br> [{{ .Name }}: {{ .Value  | html }} ]{{ end }}<br>
<span style=font-size:20px;font-weight:bold;> 故障时间:span>    {{ ($alert.StartsAt.Add 28800e9).Format "2006-01-02 15:04:05" }}<br>========= = end =  =========<br>
<br>
<br>
<br>
<br><div><div style=margin:40px><p style=font-size:20px>运维部门p><p style=color:red;font-size:14px>(这是一封自动发送的邮件，请勿回复。)p>div><div align=right style="margin:40px;border-top:solid 1px gray" id=bottomTime><p style=margin-right:20px>达梦运维监控平台p><label style=margin-right:20px>{{ ($alert.StartsAt.Add 28800e9).Format "2006-01-02 " }}<br>label>div>
div>{{- end }}
{{- end }}
{{- if gt (len .Alerts.Resolved) 0 -}}
{{- range $index, $alert := .Alerts -}}
========= <span style=color:#00FF00;font-size:24px;font-weight:bold;> 告警恢复 span>=========<br>
<span style=font-size:20px;font-weight:bold;> 告警程序:span>    Alertmanager <br>
<span style=font-size:20px;font-weight:bold;> 告警主题:span>    {{ $alert.Annotations.summary }}<br>
<span style=font-size:20px;font-weight:bold;> 告警主机:span>    {{ .Labels.instance }} <br>
<span style=font-size:20px;font-weight:bold;> 告警类型:span>    {{ .Labels.alertname }}<br>
<span style=font-size:20px;font-weight:bold;> 告警级别:span>    {{ $alert.Labels.severity }} 级 <br>
<span style=font-size:20px;font-weight:bold;> 告警详情:span>    {{ $alert.Annotations.message }}{{ $alert.Annotations.description}}<br>
<span style=font-size:20px;font-weight:bold;> 故障时间:span>    {{ ($alert.StartsAt.Add 28800e9).Format "2006-01-02 15:04:05" }}<br>
<span style=font-size:20px;font-weight:bold;> 恢复时间:span>    {{ ($alert.EndsAt.Add 28800e9).Format "2006-01-02 15:04:05" }}<br>
========= = end =  =========
<br>
<br>
<br>
<br><div><div style=margin:40px><p style=font-size:20px>运维部门p><p style=color:red;font-size:14px>(这是一封自动发送的邮件，请勿回复。)p>div><div align=right style="margin:40px;border-top:solid 1px gray" id=bottomTime><p style=margin-right:20px>达梦运维监控平台p><label style=margin-right:20px>{{ ($alert.StartsAt.Add 28800e9).Format "2006-01-02 " }}<br>label>div>
div>
{{- end }}
{{- end }}
{{- end }}{{ define "__subject" }}
{{- range $index, $alert := .Alerts -}}{{ $test := "DM监控告警通知:"}}{{$test}}{{$alert.Annotations.message }}{{$alert.Annotations.description}}{{($alert.StartsAt.Add 28800e9).Format "2006-01-02 15:04:05" }}
{{- end }}
{{- end }}

八、（邮件告警）重启altermanager

## 5.重启alertmanager
[root@localhost alertmanager-0.24.0]# ps -ef|grep alertmanager
root       32651   29851  0 20:45 pts/1    00:00:02 ./alertmanager
[root@localhost alertmanager-0.24.0]# kill -9 32651
[root@localhost alertmanager-0.24.0]# nohup ./alertmanager > ./alertmanager.log 2>&1 &##  通过浏览器访问http://被监控端IP:9093 查看地址
http://192.168.145.161:9093

九、（邮件告警）测试告警

#@ 1. 手动利用脚本触发测试告警
[root@localhost alertmanager-0.24.0]# vim alerts_message.sh#!/usr/bin/env bash
alerts1='[{"labels": {"alertname": "DiskRunningFull1","dev": "sda11","instance": "example11"},"annotations": {"info": "The disk sda1 is running full","summary": "please check the instance example1"}}]'
curl -XPOST -d"$alerts1" http://localhost:9093/api/v1/alerts## 2.赋予权限
[root@localhost alertmanager-0.24.0]# chmod 755 alerts_message.sh 
## 3.执行脚本
[root@localhost alertmanager-0.24.0]# ./alerts_message.sh 
{"status":"success"}## 4.测试alertmanager的状态

等待一分钟左右就会发送到邮箱

十、（短信告警）alertmanager配置短信告警

短信告警其实就是将信息转发到指定的接口，然后接口进行处理。现场用的短信。具体的接口实现可以参照源码sms_notification.zip（在文末资料包中）

## 1. alertmanager
[root@localhost alertmanager-0.24.0]# cd /opt/jiankong/alertmanager-0.24.0
## 2. 修改配置文件alertmanager.yml
[root@localhost alertmanager-0.24.0]# vim alertmanager.ymlglobal:resolve_timeout: 5m 
route:group_by: ['alertname']group_wait: 1m     # 一个新分组等待发送报警的时间group_interval: 15m  #已经发送成功了报警的组,有新增alert加入组时下一次发送报警的时间repeat_interval: 1h # 报警无变化情况下,重复发送告警时间。默认1hreceiver: 'webhook'
receivers:- name: 'webhook'webhook_configs:- url: 'http://127.0.0.1:9400/alertMessage/receive2' 
inhibit_rules:- source_match:severity: '严重'target_match:severity: '警告'equal: ['alertname', 'job', 'instance']

十一、（短信告警）短信接口的代码实现

package com.dameng.sms_notification.controller;import cn.hutool.core.date.DateUtil;
import cn.hutool.core.util.StrUtil;
import cn.hutool.http.HttpRequest;
import cn.hutool.http.HttpUtil;
import cn.hutool.json.JSONArray;
import cn.hutool.json.JSONObject;
import cn.hutool.json.JSONUtil;
import cn.hutool.log.StaticLog;
import com.dameng.sms_notification.util.CustomUtil;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;
import org.springframework.web.bind.annotation.*;import java.nio.charset.StandardCharsets;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;@RestController
@RequestMapping("alertMessage")
@Component
public class ReceiveAlertMessageController {@Value("${mobile.personnel}")private String personnels;private String smsUrl = "http://10.23.153.11/http/v1/formSend";@PostMapping("receive")public String receiveMsg(@RequestBody byte[] data) {String msg = new String(data, 0, data.length, StandardCharsets.UTF_8);StaticLog.info("接收AlertManager预警消息：" + msg);Map<String, Object> result = new HashMap<>();if(StrUtil.isBlank(msg)){result.put("msg", "报警失败");result.put("code", 0);String warnStr = JSONUtil.toJsonStr(result);result.put("date", msg);StaticLog.warn("数据解析失败，date is {}",warnStr);return warnStr;}try {JSONObject jsonObject = JSONUtil.parseObj(msg);JSONArray alertsJsonArray = jsonObject.getJSONArray("alerts");for (int i = 0; i < alertsJsonArray.size(); i++) {JSONObject alertsjsonObject = alertsJsonArray.getJSONObject(i);//获取告警的类型状态String status = alertsjsonObject.getStr("status"); //firing//获取告警描述JSONObject labelList = alertsjsonObject.getJSONObject("labels");String alertname = labelList.getStr("alertname");//集群发生切换String instance = labelList.getStr("instance");//188.131.177.169:9200String severity = labelList.getStr("severity");//严重//告警具体内容JSONObject annotationsList = alertsjsonObject.getJSONObject("annotations");// String description = annotationsList.getStr("description");String summary = annotationsList.getStr("summary");if(status.equals("firing")){//修改时间Date startsAt = alertsjsonObject.getDate("startsAt",DateUtil.parse( DateUtil.today(), "yyyy-MM-dd"));//拼接字符串String firingSendStr = summary + ",故障时间：" + DateUtil.format(startsAt, "yyyy-MM-dd HH:mm:ss");//获取手机号List<String> mobileList = CustomUtil.convertList(personnels);for (String mobileStr : mobileList) {if(StrUtil.isNotBlank(mobileStr)){StaticLog.info("发送人：{},发送内容：{}",mobileStr,firingSendStr);HashMap<String, Object> paramMap = new HashMap<>();paramMap.put("mobileTo", mobileStr);paramMap.put("sendMsg", firingSendStr);paramMap.put("userName", "ozab");paramMap.put("pwd", "D4F971D22B7E4C30F85036D466B49B5A");paramMap.put("organizationId", "000025");paramMap.put("sysId", "162");paramMap.put("messageType", "162");String result3= HttpUtil.get(smsUrl, paramMap);StaticLog.info("执行结果：{}",result3);if(!result3.contains("成功")){StaticLog.error("send info fail ,because {}",result3);result.put("msg", "报警失败");result.put("code", 0);result.put("because", result3);String warnStr = JSONUtil.toJsonStr(result);return warnStr;}/*                 //调用发送短信接口StaticLog.info("发送人：{},发送内容：{}",mobileStr,firingSendStr);JSONObject entries = new JSONObject();entries.set("PhoneNumberSet", new String[]{mobileStr});entries.set("SmsSdkAppId","1400692305");entries.set("SignName",  "学习分享个人网");entries.set("TemplateId",  "1436191");entries.set("TemplateParamSet",  new String[]{"1234"});// 设置请求头Map heads = new HashMap<>();heads.put("Content-Type", "application/json;charset=UTF-8");String result2 = HttpRequest.post("sms.tencentcloudapi.com").headerMap(heads, false).body(entries.toJSONString(1)).execute().body();System.out.println(result2);*/}}}else{//"status": "resolved",//恢复时间Date startsAt = alertsjsonObject.getDate("endsAt",DateUtil.parse( DateUtil.today(), "yyyy-MM-dd"));//拼接字符串String resolvedSendStr = summary + ",故障恢复时间：" + DateUtil.format(startsAt, "yyyy-MM-dd HH:mm:ss");//获取手机号List<String> mobileList = CustomUtil.convertList(personnels);for (String mobileStr : mobileList) {if(StrUtil.isNotBlank(mobileStr)){//调用发送短信接口//添加到日志StaticLog.info("发送人：{},发送内容：{}",mobileStr,resolvedSendStr);HashMap<String, Object> paramMap = new HashMap<>();paramMap.put("mobileTo", mobileStr);paramMap.put("sendMsg", resolvedSendStr);paramMap.put("userName", "ozab");paramMap.put("pwd", "D4F971D22B7E4C30F85036D466B49B5A");paramMap.put("organizationId", "000025");paramMap.put("sysId", "162");paramMap.put("messageType", "162");String result3= HttpUtil.get(smsUrl, paramMap);StaticLog.info("执行结果：{}",result3);if(!result3.contains("成功")){StaticLog.error("send info fail ,because {}",result3);result.put("msg", "报警失败");result.put("code", 0);result.put("because", result3);return JSONUtil.toJsonStr(result);}}}}}} catch (Exception e) {StaticLog.error("send info fail ,because {}",e.getMessage());result.put("msg", "报警失败");result.put("code", 0);String warnStr = JSONUtil.toJsonStr(result);result.put("because", e.getMessage());return warnStr;}/**接收AlertManager预警消息：{"receiver":"webhook","status":"firing","alerts":[{"status":"firing","labels":{"alertname":"内存使用率过高","instance":"120.53.103.235:9100","job":"dm_node_agent","severity":"警告"},"annotations":{"description":"主机：120.53.103.235:9100 内存使用率过高","summary":"主机：120.53.103.235:9100:内存使用率过高，超过80%"},"startsAt":"2022-06-11T15:33:09.089Z","endsAt":"0001-01-01T00:00:00Z","generatorURL":"http://VM-24-17-centos:9090/graph?g0.expr=%281+-+%28node_memory_MemAvailable_bytes+%2F+%28node_memory_MemTotal_bytes%29%29%29+%2A+100+%3E+80\u0026g0.tab=1","fingerprint":"d159c71a32560544"}],"groupLabels":{"alertname":"内存使用率过高"},"commonLabels":{"alertname":"内存使用率过高","instance":"120.53.103.235:9100","job":"dm_node_agent","severity":"警告"},"commonAnnotations":{"description":"主机：120.53.103.235:9100 内存使用率过高","summary":"主机：120.53.103.235:9100:内存使用率过高，超过80%"},"externalURL":"http://VM-24-17-centos:9093","version":"4","groupKey":"{}:{alertname=\"内存使用率过高\"}","truncatedAlerts":0}接收AlertManager预警消息：{"receiver":"webhook","status":"resolved","alerts":[{"status":"resolved","labels":{"alertname":"内存使用率过高","instance":"120.53.103.235:9100","job":"dm_node_agent","severity":"警告"},"annotations":{"description":"主机：120.53.103.235:9100 内存使用率过高","summary":"主机：120.53.103.235:9100:内存使用率过高，超过80%"},"startsAt":"2022-06-11T15:33:09.089Z","endsAt":"2022-06-11T15:42:54.089Z","generatorURL":"http://VM-24-17-centos:9090/graph?g0.expr=%281+-+%28node_memory_MemAvailable_bytes+%2F+%28node_memory_MemTotal_bytes%29%29%29+%2A+100+%3E+80\u0026g0.tab=1","fingerprint":"d159c71a32560544"}],"groupLabels":{"alertname":"内存使用率过高"},"commonLabels":{"alertname":"内存使用率过高","instance":"120.53.103.235:9100","job":"dm_node_agent","severity":"警告"},"commonAnnotations":{"description":"主机：120.53.103.235:9100 内存使用率过高","summary":"主机：120.53.103.235:9100:内存使用率过高，超过80%"},"externalURL":"http://VM-24-17-centos:9093","version":"4","groupKey":"{}:{alertname=\"内存使用率过高\"}","truncatedAlerts":0}*/result.put("msg", "success");result.put("code", 1);return JSONUtil.toJsonStr(result);}@PostMapping("receive2")public String receive2(@RequestBody byte[] data) {String msg = new String(data, 0, data.length, StandardCharsets.UTF_8);StaticLog.info("接收AlertManager预警消息：" + msg);Map<String, Object> result = new HashMap<>();if(StrUtil.isBlank(msg)){result.put("msg", "报警失败");result.put("code", 0);String warnStr = JSONUtil.toJsonStr(result);result.put("date", msg);StaticLog.warn("数据解析失败，date is {}",warnStr);return warnStr;}try {JSONObject jsonObject = JSONUtil.parseObj(msg);JSONArray alertsJsonArray = jsonObject.getJSONArray("alerts");for (int i = 0; i < alertsJsonArray.size(); i++) {JSONObject alertsjsonObject = alertsJsonArray.getJSONObject(i);//获取告警的类型状态String status = alertsjsonObject.getStr("status"); //firing//获取告警描述JSONObject labelList = alertsjsonObject.getJSONObject("labels");String alertname = labelList.getStr("alertname");//集群发生切换String instance = labelList.getStr("instance");//188.131.177.169:9200String severity = labelList.getStr("severity");//严重//告警具体内容JSONObject annotationsList = alertsjsonObject.getJSONObject("annotations");// String description = annotationsList.getStr("description");String summary = annotationsList.getStr("summary");if(status.equals("firing")){//修改时间Date startsAt = alertsjsonObject.getDate("startsAt",DateUtil.parse( DateUtil.today(), "yyyy-MM-dd"));//拼接字符串String firingSendStr = summary + ",故障时间：" + DateUtil.format(startsAt, "yyyy-MM-dd HH:mm:ss");//获取手机号List<String> mobileList = CustomUtil.convertList(personnels);for (String mobileStr : mobileList) {if(StrUtil.isNotBlank(mobileStr)){StaticLog.info("发送人：{},发送内容：{}",mobileStr,firingSendStr);HashMap<String, Object> paramMap = new HashMap<>();paramMap.put("mobileTo", mobileStr);paramMap.put("sendMsg", firingSendStr);paramMap.put("userName", "ozab");paramMap.put("pwd", "D4F971D22B7E4C30F85036D466B49B5A");paramMap.put("organizationId", "000025");paramMap.put("sysId", "162");paramMap.put("messageType", "162");String result3= HttpUtil.post(smsUrl, paramMap);StaticLog.info("执行结果：{}",result3);if(!result3.contains("成功")){StaticLog.error("send info fail ,because {}",result3);result.put("msg", "报警失败");result.put("code", 0);result.put("because", result3);String warnStr = JSONUtil.toJsonStr(result);return warnStr;}/*                 //调用发送短信接口StaticLog.info("发送人：{},发送内容：{}",mobileStr,firingSendStr);JSONObject entries = new JSONObject();entries.set("PhoneNumberSet", new String[]{mobileStr});entries.set("SmsSdkAppId","1400692305");entries.set("SignName",  "学习分享个人网");entries.set("TemplateId",  "1436191");entries.set("TemplateParamSet",  new String[]{"1234"});// 设置请求头Map heads = new HashMap<>();heads.put("Content-Type", "application/json;charset=UTF-8");String result2 = HttpRequest.post("sms.tencentcloudapi.com").headerMap(heads, false).body(entries.toJSONString(1)).execute().body();System.out.println(result2);*/}}}else{//"status": "resolved",//恢复时间Date startsAt = alertsjsonObject.getDate("endsAt",DateUtil.parse( DateUtil.today(), "yyyy-MM-dd"));//拼接字符串String resolvedSendStr = summary + ",故障恢复时间：" + DateUtil.format(startsAt, "yyyy-MM-dd HH:mm:ss");//获取手机号List<String> mobileList = CustomUtil.convertList(personnels);for (String mobileStr : mobileList) {if(StrUtil.isNotBlank(mobileStr)){//调用发送短信接口//添加到日志StaticLog.info("发送人：{},发送内容：{}",mobileStr,resolvedSendStr);HashMap<String, Object> paramMap = new HashMap<>();paramMap.put("mobileTo", mobileStr);paramMap.put("sendMsg", resolvedSendStr);paramMap.put("userName", "ozab");paramMap.put("pwd", "D4F971D22B7E4C30F85036D466B49B5A");paramMap.put("organizationId", "000025");paramMap.put("sysId", "162");paramMap.put("messageType", "162");String result3= HttpUtil.post(smsUrl, paramMap);StaticLog.info("执行结果：{}",result3);if(!result3.contains("成功")){StaticLog.error("send info fail ,because {}",result3);result.put("msg", "报警失败");result.put("code", 0);result.put("because", result3);String warnStr = JSONUtil.toJsonStr(result);return warnStr;}}}}}} catch (Exception e) {StaticLog.error("send info fail ,because {}",e.getMessage());result.put("msg", "报警失败");result.put("code", 0);String warnStr = JSONUtil.toJsonStr(result);result.put("because", e.getMessage());return warnStr;}/**接收AlertManager预警消息：{"receiver":"webhook","status":"firing","alerts":[{"status":"firing","labels":{"alertname":"内存使用率过高","instance":"120.53.103.235:9100","job":"dm_node_agent","severity":"警告"},"annotations":{"description":"主机：120.53.103.235:9100 内存使用率过高","summary":"主机：120.53.103.235:9100:内存使用率过高，超过80%"},"startsAt":"2022-06-11T15:33:09.089Z","endsAt":"0001-01-01T00:00:00Z","generatorURL":"http://VM-24-17-centos:9090/graph?g0.expr=%281+-+%28node_memory_MemAvailable_bytes+%2F+%28node_memory_MemTotal_bytes%29%29%29+%2A+100+%3E+80\u0026g0.tab=1","fingerprint":"d159c71a32560544"}],"groupLabels":{"alertname":"内存使用率过高"},"commonLabels":{"alertname":"内存使用率过高","instance":"120.53.103.235:9100","job":"dm_node_agent","severity":"警告"},"commonAnnotations":{"description":"主机：120.53.103.235:9100 内存使用率过高","summary":"主机：120.53.103.235:9100:内存使用率过高，超过80%"},"externalURL":"http://VM-24-17-centos:9093","version":"4","groupKey":"{}:{alertname=\"内存使用率过高\"}","truncatedAlerts":0}接收AlertManager预警消息：{"receiver":"webhook","status":"resolved","alerts":[{"status":"resolved","labels":{"alertname":"内存使用率过高","instance":"120.53.103.235:9100","job":"dm_node_agent","severity":"警告"},"annotations":{"description":"主机：120.53.103.235:9100 内存使用率过高","summary":"主机：120.53.103.235:9100:内存使用率过高，超过80%"},"startsAt":"2022-06-11T15:33:09.089Z","endsAt":"2022-06-11T15:42:54.089Z","generatorURL":"http://VM-24-17-centos:9090/graph?g0.expr=%281+-+%28node_memory_MemAvailable_bytes+%2F+%28node_memory_MemTotal_bytes%29%29%29+%2A+100+%3E+80\u0026g0.tab=1","fingerprint":"d159c71a32560544"}],"groupLabels":{"alertname":"内存使用率过高"},"commonLabels":{"alertname":"内存使用率过高","instance":"120.53.103.235:9100","job":"dm_node_agent","severity":"警告"},"commonAnnotations":{"description":"主机：120.53.103.235:9100 内存使用率过高","summary":"主机：120.53.103.235:9100:内存使用率过高，超过80%"},"externalURL":"http://VM-24-17-centos:9093","version":"4","groupKey":"{}:{alertname=\"内存使用率过高\"}","truncatedAlerts":0}*/result.put("msg", "success");result.put("code", 1);String infoStr = JSONUtil.toJsonStr(result);return infoStr;}@GetMapping("receive1")public String receiveMsg1() {System.out.println(personnels);//获取list的数据List<String> personnelsList = CustomUtil.convertList(personnels);Map<String, Object> result = new HashMap<>();result.put("msg", "报警失败");result.put("code", 0);String testStr1 = "{\"receiver\":\"webhook\",\"status\":\"firing\",\"alerts\":[{\"status\":\"firing\",\"labels\":{\"alertname\":\"内存使用率过高\",\"instance\":\"120.53.103.235:9100\",\"job\":\"dm_node_agent\",\"severity\":\"警告\"},\"annotations\":{\"description\":\"主机：120.53.103.235:9100 内存使用率过高\",\"summary\":\"主机：120.53.103.235:9100:内存使用率过高，超过80%\"},\"startsAt\":\"2022-06-11T15:33:09.089Z\",\"endsAt\":\"0001-01-01T00:00:00Z\",\"generatorURL\":\"http://VM-24-17-centos:9090/graph?g0.expr=%281+-+%28node_memory_MemAvailable_bytes+%2F+%28node_memory_MemTotal_bytes%29%29%29+%2A+100+%3E+80\\u0026g0.tab=1\",\"fingerprint\":\"d159c71a32560544\"}],\"groupLabels\":{\"alertname\":\"内存使用率过高\"},\"commonLabels\":{\"alertname\":\"内存使用率过高\",\"instance\":\"120.53.103.235:9100\",\"job\":\"dm_node_agent\",\"severity\":\"警告\"},\"commonAnnotations\":{\"description\":\"主机：120.53.103.235:9100 内存使用率过高\",\"summary\":\"主机：120.53.103.235:9100:内存使用率过高，超过80%\"},\"externalURL\":\"http://VM-24-17-centos:9093\",\"version\":\"4\",\"groupKey\":\"{}:{alertname=\\\"内存使用率过高\\\"}\",\"truncatedAlerts\":0}\n";String testStr = "{\"receiver\":\"webhook\",\"status\":\"firing\",\"alerts\":[{\"status\":\"firing\",\"labels\":{\"alertname\":\"集群发生切换\",\"host_name\":\"VM-0-17-centos\",\"instance\":\"188.131.177.169:9200\",\"job\":\"dmdbms_oa_dw\",\"severity\":\"严重\"},\"annotations\":{\"description\":\"主机：188.131.177.169:9200 数据库模式切换\",\"summary\":\"主机：188.131.177.169:9200:数据库发生模式切换，疑似发生集群切换请核实\"},\"startsAt\":\"2022-06-12T03:44:39.089Z\",\"endsAt\":\"0001-01-01T00:00:00Z\",\"generatorURL\":\"http://VM-24-17-centos:9090/graph?g0.expr=dmdbms_switching_occurs+%3D%3D+0\\u0026g0.tab=1\",\"fingerprint\":\"11ecdd9b1158f914\"},{\"status\":\"firing\",\"labels\":{\"alertname\":\"集群发生切换\",\"host_name\":\"VM-0-4-centos\",\"instance\":\"62.234.115.217:9200\",\"job\":\"dmdbms_oa_dw\",\"severity\":\"严重\"},\"annotations\":{\"description\":\"主机：62.234.115.217:9200 数据库模式切换\",\"summary\":\"主机：62.234.115.217:9200:数据库发生模式切换，疑似发生集群切换请核实\"},\"startsAt\":\"2022-06-12T03:44:39.089Z\",\"endsAt\":\"0001-01-01T00:00:00Z\",\"generatorURL\":\"http://VM-24-17-centos:9090/graph?g0.expr=dmdbms_switching_occurs+%3D%3D+0\\u0026g0.tab=1\",\"fingerprint\":\"3f151f5edafed23b\"}],\"groupLabels\":{\"alertname\":\"集群发生切换\"},\"commonLabels\":{\"alertname\":\"集群发生切换\",\"job\":\"dmdbms_oa_dw\",\"severity\":\"严重\"},\"commonAnnotations\":{},\"externalURL\":\"http://VM-24-17-centos:9093\",\"version\":\"4\",\"groupKey\":\"{}:{alertname=\\\"集群发生切换\\\"}\",\"truncatedAlerts\":0}";//主备切换恢复String testStr2 = "{\"receiver\":\"webhook\",\"status\":\"resolved\",\"alerts\":[{\"status\":\"resolved\",\"labels\":{\"alertname\":\"集群发生切换\",\"host_name\":\"VM-0-17-centos\",\"instance\":\"188.131.177.169:9200\",\"job\":\"dmdbms_oa_dw\",\"severity\":\"严重\"},\"annotations\":{\"description\":\"主机：188.131.177.169:9200 数据库模式切换\",\"summary\":\"主机：188.131.177.169:9200:数据库发生模式切换，疑似发生集群切换请核实\"},\"startsAt\":\"2022-06-12T03:44:39.089Z\",\"endsAt\":\"2022-06-12T03:54:39.089Z\",\"generatorURL\":\"http://VM-24-17-centos:9090/graph?g0.expr=dmdbms_switching_occurs+%3D%3D+0\\u0026g0.tab=1\",\"fingerprint\":\"11ecdd9b1158f914\"},{\"status\":\"resolved\",\"labels\":{\"alertname\":\"集群发生切换\",\"host_name\":\"VM-0-4-centos\",\"instance\":\"62.234.115.217:9200\",\"job\":\"dmdbms_oa_dw\",\"severity\":\"严重\"},\"annotations\":{\"description\":\"主机：62.234.115.217:9200 数据库模式切换\",\"summary\":\"主机：62.234.115.217:9200:数据库发生模式切换，疑似发生集群切换请核实\"},\"startsAt\":\"2022-06-12T03:44:39.089Z\",\"endsAt\":\"2022-06-12T03:54:39.089Z\",\"generatorURL\":\"http://VM-24-17-centos:9090/graph?g0.expr=dmdbms_switching_occurs+%3D%3D+0\\u0026g0.tab=1\",\"fingerprint\":\"3f151f5edafed23b\"}],\"groupLabels\":{\"alertname\":\"集群发生切换\"},\"commonLabels\":{\"alertname\":\"集群发生切换\",\"job\":\"dmdbms_oa_dw\",\"severity\":\"严重\"},\"commonAnnotations\":{},\"externalURL\":\"http://VM-24-17-centos:9093\",\"version\":\"4\",\"groupKey\":\"{}:{alertname=\\\"集群发生切换\\\"}\",\"truncatedAlerts\":0}";JSONObject jsonObject = JSONUtil.parseObj(testStr);/*    String status = jsonObject.getStr("status");JSONObject commonAnnotations = jsonObject.getJSONObject("commonAnnotations");String description = commonAnnotations.getStr("description");
*/JSONArray alertsJsonArray = jsonObject.getJSONArray("alerts");for (int i = 0; i < alertsJsonArray.size(); i++) {JSONObject alertsjsonObject = alertsJsonArray.getJSONObject(i);//获取告警的类型状态String status = alertsjsonObject.getStr("status"); //firing//获取告警描述JSONObject labelList = alertsjsonObject.getJSONObject("labels");String alertname = labelList.getStr("alertname");//集群发生切换String instance = labelList.getStr("instance");//188.131.177.169:9200String severity = labelList.getStr("severity");//严重//告警具体内容JSONObject annotationsList = alertsjsonObject.getJSONObject("annotations");// String description = annotationsList.getStr("description");String summary = annotationsList.getStr("summary");if(status.equals("firing")){//修改时间Date startsAt = alertsjsonObject.getDate("startsAt",DateUtil.parse( DateUtil.today(), "yyyy-MM-dd"));//拼接字符串String firingSendStr = summary + ",故障时间：" + DateUtil.format(startsAt, "yyyy-MM-dd HH:mm:ss");//获取手机号List<String> mobileList = CustomUtil.convertList(personnels);for (String mobileStr : mobileList) {if(StrUtil.isNotBlank(mobileStr)){HashMap<String, Object> paramMap = new HashMap<>();paramMap.put("mobileTo", mobileStr);paramMap.put("sendMsg", firingSendStr);paramMap.put("userName", "ozab");paramMap.put("pwd", "D4F971D22B7E4C30F85036D466B49B5A");paramMap.put("organizationId", "000025");paramMap.put("sysId", "162");paramMap.put("messageType", "162");String result3= HttpUtil.get(smsUrl, paramMap);System.out.println(result3);/*                 //调用发送短信接口StaticLog.info("发送人：{},发送内容：{}",mobileStr,firingSendStr);JSONObject entries = new JSONObject();entries.set("PhoneNumberSet", new String[]{mobileStr});entries.set("SmsSdkAppId","1400692305");entries.set("SignName",  "学习分享个人网");entries.set("TemplateId",  "1436191");entries.set("TemplateParamSet",  new String[]{"1234"});// 设置请求头Map heads = new HashMap<>();heads.put("Content-Type", "application/json;charset=UTF-8");String result2 = HttpRequest.post("sms.tencentcloudapi.com").headerMap(heads, false).body(entries.toJSONString(1)).execute().body();System.out.println(result2);*/}}}else{//"status": "resolved",//恢复时间Date startsAt = alertsjsonObject.getDate("endsAt",DateUtil.parse( DateUtil.today(), "yyyy-MM-dd"));//拼接字符串String resolvedSendStr = summary + ",故障恢复时间：" + DateUtil.format(startsAt, "yyyy-MM-dd HH:mm:ss");//获取手机号List<String> mobileList = CustomUtil.convertList(personnels);for (String mobileStr : mobileList) {if(StrUtil.isNotBlank(mobileStr)){//调用发送短信接口//添加到日志StaticLog.info("发送人：{},发送内容：{}",mobileStr,resolvedSendStr);HashMap<String, Object> paramMap = new HashMap<>();paramMap.put("mobileTo", mobileStr);paramMap.put("sendMsg", resolvedSendStr);paramMap.put("userName", "ozab");paramMap.put("pwd", "D4F971D22B7E4C30F85036D466B49B5A");paramMap.put("organizationId", "000025");paramMap.put("sysId", "162");paramMap.put("messageType", "162");String result3= HttpUtil.get(smsUrl, paramMap);System.out.println(result3);}}}}//获取告警的警告时间JSONObject alertList = jsonObject.getJSONArray("alerts").getJSONObject(0);return "success";}
}

其他

163邮箱开启POP配置

开启 POP3/SMTP服务

资料包

链接：https://pan.baidu.com/s/1lo6AynZBR5JdRZwY_1arbA?pwd=o178 提取码：o178

本文为达梦在线服务平台【与达梦同行】征文活动投稿文章：详细文章
第一届达梦数据库技术征文大赛来啦

本文来自互联网用户投稿，文章观点仅代表作者本人，不代表本站立场，不承担相关法律责任。如若转载，请注明出处。 如若内容造成侵权/违法违规/事实不符，请点击【内容举报】进行投诉反馈！