On 2025-03-05 01:55, Robert Webb via openSUSE Users wrote:
On Tue, 4 Mar 2025 23:59:57 +0100, "Carlos E. R." <robin.listas@telefonica.net> wrote:
On 2025-03-04 13:33, Robert Webb via openSUSE Users wrote:
[...] It defined the function 'journal_exclude()'. In your script, you need to call it at the end with (make sure your script name is different): journal_exclude "$@"
Ah. Now it produces a lot of: jq: error (at <stdin>:2607): null (null) cannot be parsed as a number jq: error (at <stdin>:2608): null (null) cannot be parsed as a number
mixed with the output.
Right. :-)
Let's see your new version instead :-)
cer@Telcontar:~/Bugzilla/Bug_1237776 - Machine went unresponsive/two> ./journal-exclude-2.bash journal-json.log > journal-filtered.log jq: error (at journal-json.log:2674): string ("Feb 24 11:...) and array ([91,48,58,4...) cannot be added jq: error (at journal-json.log:2675): string ("Feb 24 11:...) and array ([91,48,58,4...) cannot be added
This is something I haven't seen. I better check whether the posted script is identical to what I have been using.
Would you be able to post those two JSON log lines?
If I'm not mistaken, these: {"_STREAM_ID":"aa9171a41bbd4258959287b1814df831","_SELINUX_CONTEXT":"unconfined\n","__REALTIME_TIMESTAMP":"1740393369322372","_AUDIT_LOGINUID":"1000","_PID":"5417","_SYSTEMD_OWNER_UID":"1000","_RUNTIME_SCOPE":"system","_AUDIT_SESSION":"4","_SYSTEMD_UNIT":"user@1000.service","_BOOT_ID":"0a3828f43b5649c188f1bbee6ab7f468","_SYSTEMD_USER_UNIT":"wireplumber.service","__SEQNUM":"2325584","SYSLOG_FACILITY":"3","_COMM":"wireplumber","_TRANSPORT":"stdout","_CMDLINE":"/usr/bin/wireplumber","_SYSTEMD_USER_SLICE":"session.slice","_MACHINE_ID":"2ce1d54548517a7307c1c2bc38206d00","_HOSTNAME":"Telcontar","_GID":"100","_CAP_EFFECTIVE":"0","_SYSTEMD_INVOCATION_ID":"bbebce4a78ee4adebf57c3a6fee1b699","_SYSTEMD_CGROUP":"/user.slice/user-1000.slice/user@1000.service/session.slice/wireplumber.service","SYSLOG_IDENTIFIER":"wireplumber","__SEQNUM_ID":"f1e7af9f29494abe99932f67c9d50b73","MESSAGE":[91,48,58,49,53,58,49,53,46,53,57,52,55,54,49,55,56,49,93,32,91,53,52,49,55,93,32,27,91,49,59,51,51,109,32,87,65,82,78,32,27,91,49,59,51,55,109,73,80,65,77,97,110,97,103,101,114,32,27,91,49,59,51,52,109,105,112,97,95,109,97,110,97,103,101,114,46,99,112,112,58,49,53,52,32,27,91,48,109,78,111,32,73,80,65,32,102,111,117,110,100,32,105,110,32,39,47,117,115,114,47,108,105,98,54,52,47,108,105,98,99,97,109,101,114,97,39],"__CURSOR":"s=f1e7af9f29494abe99932f67c9d50b73;i=237c50;b=0a3828f43b5649c188f1bbee6ab7f468;m=3692de29;t=62ee0e88c4384;x=4665cc0efc549185","_EXE":"/usr/bin/wireplumber","_SYSTEMD_SLICE":"user-1000.slice","__MONOTONIC_TIMESTAMP":"915594793","PRIORITY":"6","_UID":"1000"} {"__SEQNUM":"2325585","_COMM":"wireplumber","_CAP_EFFECTIVE":"0","_CMDLINE":"/usr/bin/wireplumber","_SYSTEMD_USER_UNIT":"wireplumber.service","MESSAGE":[91,48,58,49,53,58,49,53,46,53,57,52,55,56,57,51,52,49,93,32,91,53,52,49,55,93,32,27,91,49,59,51,50,109,32,73,78,70,79,32,27,91,49,59,51,55,109,67,97,109,101,114,97,32,27,91,49,59,51,52,109,99,97,109,101,114,97,95,109,97,110,97,103,101,114,46,99,112,112,58,50,56,52,32,27,91,48,109,108,105,98,99,97,109,101,114,97,32,118,48,46,50,46,48],"SYSLOG_IDENTIFIER":"wireplumber","_SYSTEMD_USER_SLICE":"session.slice","_UID":"1000","_SYSTEMD_CGROUP":"/user.slice/user-1000.slice/user@1000.service/session.slice/wireplumber.service","_STREAM_ID":"aa9171a41bbd4258959287b1814df831","_SYSTEMD_UNIT":"user@1000.service","_BOOT_ID":"0a3828f43b5649c188f1bbee6ab7f468","_AUDIT_SESSION":"4","_SELINUX_CONTEXT":"unconfined\n","SYSLOG_FACILITY":"3","_PID":"5417","_SYSTEMD_SLICE":"user-1000.slice","__REALTIME_TIMESTAMP":"1740393369322372","_TRANSPORT":"stdout","_MACHINE_ID":"2ce1d54548517a7307c1c2bc38206d00","_RUNTIME_SCOPE":"system","_HOSTNAME":"Telcontar","_SYSTEMD_INVOCATION_ID":"bbebce4a78ee4adebf57c3a6fee1b699","_AUDIT_LOGINUID":"1000","_EXE":"/usr/bin/wireplumber","__SEQNUM_ID":"f1e7af9f29494abe99932f67c9d50b73","_SYSTEMD_OWNER_UID":"1000","__CURSOR":"s=f1e7af9f29494abe99932f67c9d50b73;i=237c51;b=0a3828f43b5649c188f1bbee6ab7f468;m=3692de29;t=62ee0e88c4384;x=787c99946485c559","__MONOTONIC_TIMESTAMP":"915594793","PRIORITY":"6","_GID":"100"}
sed -n -e '2674,2675p' journal-json.log > add_errors.log
Since the script was expecting two strings to add (concatenate), and the second item in the error is an array of numbers, I suspect this is what is happening[2]:
"A field that contains non-printable or non-UTF8 is serialized as a number array instead. This is necessary to handle binary data in a safe way without losing data, since JSON cannot embed binary data natively. Each byte of the binary field will be mapped to its numeric value in the range 0…255."
I don't know if that is going to be easy to convert.
There are some differences in whitespace, I think you mentioned it:
Yes, I saw the same, but have not investigated it yet.
There are some differences in timestamps - original:
Feb 24 11:21:07 Telcontar systemd[1]: Mounting /boot... Feb 24 11:21:07 Telcontar systemd[1]: data-storage_c.mount: Directory /data/storage_c to mount over is not empty, mounting anyway. Feb 24 11:21:07 Telcontar systemd[1]: Mounting /data/storage_c... Feb 24 11:21:07 Telcontar systemd[1]: data-storage_d.mount: Directory /data/storage_d to mount over is not empty, mounting anyway. Feb 24 11:21:07 Telcontar systemd[1]: Mounting /data/storage_d...
filtered:
Feb 24 11:21:08 Telcontar systemd[1]: Mounting /boot... Feb 24 11:21:08 Telcontar systemd[1]: data-storage_c.mount: Directory /data/storage_c to mount over is not empty, mounting anyway. Feb 24 11:21:08 Telcontar systemd[1]: Mounting /data/storage_c... Feb 24 11:21:08 Telcontar systemd[1]: data-storage_d.mount: Directory /data/storage_d to mount over is not empty, mounting anyway. Feb 24 11:21:08 Telcontar systemd[1]: Mounting /data/storage_d...
I see more occurrences of timestamps differences in the output. Example, much later: [...]
Maybe these are related to the add errors above since they involved the timestamps.
This is the filtering at work: [...]
Thanks very much for the feedback.
Ah, this is important. Original:
Mar 03 01:50:01 Telcontar CRON[18600]: (root) CMDEND ([ -x /usr/lib64/sa/sa1 ] && exec /usr/lib64/sa/sa1 1 1) Mar 03 01:50:24 Telcontar systemd-coredump[18120]: Process 2805 (X) of user 0 dumped core.
Stack trace of thread 2844: #0 0x00007f636baa941c __pthread_kill_implementation (libc.so.6 + 0xa941c) #1 0x00007f636ba57842 raise (libc.so.6 + 0x57842) #2 0x00007f636ba3f5cf abort (libc.so.6 + 0x3f5cf) ... omitting lines ELF object binary architecture: AMD x86-64 Mar 03 01:50:24 Telcontar kglobalaccel5[29272]: The X11 connection broke (error 1). Did the X11 server die?
filtered - the contents of the coredump are missing:
Mar 03 01:50:01 Telcontar CRON[18600]: (root) CMDEND ([ -x /usr/lib64/sa/sa1 ] && exec /usr/lib64/sa/sa1 1 1) Mar 03 01:50:24 Telcontar systemd-coredump[18120]: Mar 03 01:50:24 Telcontar kglobalaccel5[29272]: The X11 connection broke (error 1). Did the X11 server die?
Thankyou for the script, it helps a lot. Just that missing coredump at the end is important. Big multiline message, I guess.
If it is too big, it might not have been included in the exported JSON.[2]:
"The JSON serializer can optionally skip huge (as in larger than a specific threshold) data fields from the JSON object. If that is enabled and a data field is too large, the field name is still included in the JSON object but assigned null."
[2] https://systemd.io/JOURNAL_EXPORT_FORMATS/#journal-export-format
Ah, I see. It would have taken me several long moons to get this far. -- Cheers / Saludos, Carlos E. R. (from 15.6 x86_64 at Telcontar)