Trying to reduce log entries to a single copy per message and uniq doesn't seem to be working. Note, this is a SLES 9 syslog box so that is why you are seeing ufs log entries from my solaris systems. Here are some sample log entries, as you can see they differ only in the timestamp field: Jan 31 10:55:45 db-0202 ufs: [ID 845546 kern.notice] NOTICE: alloc: /u70: file system full Jan 31 11:00:44 db-0202 ufs: [ID 845546 kern.notice] NOTICE: alloc: /u70: file system full Jan 31 11:01:24 db-0202 ufs: [ID 845546 kern.notice] NOTICE: alloc: /u70: file system full First I tried ignoring fields: cat syslog | uniq -f 3 > tempfile According to the man page a field is defined as white space and then non-white space, so using -f3 above should ignore the entire timestamp field, but it is not behaving in this manner. I then tried ignoring the first 16 characters: cat syslog | uniq -s 16 > tempfile Same thing, the output generated by this command contains entries that differ only in the timestamp. I used 'cat -A' to examine my input file for non-printable characters that might be causing this but didn't see any. Maybe I am overlooking something obvious due to lack of sleep but nothing is jumping out at me. The real annoying thing is that uniq does reduce some of the duplicate entries but not all. Thx
On Wednesday 02 February 2005 6:07 am, Rhugga wrote:
Maybe I am overlooking something obvious due to lack of sleep but nothing is jumping out at me. The real annoying thing is that uniq does reduce some of the duplicate entries but not all.
From the man page for uniq NAME uniq - remove duplicate lines from a sorted file The key word there is 'sorted'. You need to provide uniq with a sorted file. Scott -- POPFile, the OpenSource EMail Classifier http://popfile.sourceforge.net/ Linux 2.6.8-24.11-default x86_64
participants (2)
-
Rhugga
-
Scott Leighton