记一次MQ发送失败的错误

  • Post author:
  • Post category:其他


症状:某个MQ发送失败

错误日志:

15:00:00.888 [HSFBizProcessor-DEFAULT-6-thread-23] ERROR com.bosideng.bsdyun.msg.service.MqServiceImpl – com.aliyun.openservices.ons.api.exception.ONSClientException: Connect broker failed, Topic=VIP_JOIN_ACTIVITY_PRD, msgId=null

15:00:07.265 [HSFBizProcessor-DEFAULT-6-thread-50] WARN io.netty.channel.AbstractChannel – Force-closing a channel whose registration task was not accepted by an event loop: [id: 0xd2b4893d]

java.util.concurrent.RejectedExecutionException: event executor terminated

排查 /home/admin/logs/rocketmq_client.log 发现出现过oom:

可以使用 jstat -gcutils pid 1000 查看gc情况,pid需换成业务程序的进程id

查看gc.lg 出现多次full gc


定位应用日志发现在14:11出现oom

重点排查业务系统的内存溢出问题,排查中…

Exception in thread “RebalanceService” java.lang.OutOfMemoryError: Exception From Host:[10.101.3.101]GC overhead limit exceeded14:11:25.197 [HSFBizProcessor-DEFAULT-6-thread-49] ERROR com.bosideng.bsdyun.msg.service.MsInterfaceMessageServiceImpl – org.springframework.jdbc.UncategorizedSQLException: Error attempting to get column ‘xml_content’ from result set.  Cause: java.sql.SQLException: Error

; uncategorized SQLException for SQL []; SQL state [null]; error code [0]; Error; nested exception is java.sql.SQLException: Error



版权声明:本文为fishernemo原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。