small bash or python scripting job to automate log parsing and emailing
Hi Andreas - hopefully we can work together again. Here is what I need to get done
1. I have a set of network devices that generate logs and dump it to a syslog CentOS 6.6 server.
2. below is the directory structure and the log files shown - where '2015' is year, then next directory is month then next directory is day and then the log files for that day.
[root@CRCTMC-SYSLOG log]# cd syslog/
[root@CRCTMC-SYSLOG syslog]# ls
[root@CRCTMC-SYSLOG syslog]# cd 2015
[root@CRCTMC-SYSLOG 2015]# ls
06 07 08 09
[root@CRCTMC-SYSLOG 2015]# cd 09
[root@CRCTMC-SYSLOG 09]# ls
01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16
[root@CRCTMC-SYSLOG 09]# cd 16
[root@CRCTMC-SYSLOG 16]# ls
CRCTMC-CORE_SW_messages.log CRCTMC-SRV_SW_messages.log DTOC-CORE_messages.log IVDS-srx550_1_messages.log
CRCTMC-DIST_SW_messages.log CRCTMC-USER_SW_messages.log DTOC-DISTRSW0_messages.log
CRCTMC-FW1_messages.log CRCTMC-VIDEO_SW_messages.log DTOC-FW0_messages.log
3. I need to have a process that watches (scans and parses, for each current day, not the past) these logs (any *.log) in real time, and IF the following string is found:
3.a UI_LOGIN_EVENT - email to a configurable email address right away the contents of the string, example of the string below:
Sep 14 16:39:14 CRCTMC-FW1 mgd: UI_LOGIN_EVENT: User 'pavel.stoev' login, class 'j-super-user' , ssh-connection '172.30.30.194 55680 192.168.205.11 22', client-mode 'cli'
3.b XAUTH username: - email to a configurable email address right away the contents of the string, example of the string below:
Jul 29 07:50:54 Tampa-VPN1 kmd: KMD_VPN_UP_ALARM_USER: VPN instance-INSTANCE-dyn-vpn_0003_0006_0000_268173694 from 220.127.116.11 is up. Local-ip: 10.25.25.3, gateway name: dyn-vpn-local-gw, vpn name: INSTANCE-dyn-vpn_0003_0006_0000, tunnel-id: 268173694, local tunnel-if: , remote tunnel-ip: Not-Available, Local IKE-ID: 18.104.22.168, Remote IKE-ID: steven.olmstedTampa-VPN, XAUTH username: steven.olmsted, VR id: 0
3.c UI_CMDLINE_READ_LINE: - then start waiting and collecting all subsequent UI_CMDLINE_READ_LINE: strings and then once no more appear in logs (for example something else is logged), then email everything collected to configurable email address. Example of what needs to be collected and emailed out is shown in the attachment - I only wish to receive a summary email of these commands, not one email per command