Re: User support data anyalsis: Request to export specific subforums

Hey Malcom, thanks for your quick reply, Added heroes to addressees to my reply (this message). We are looking for: - the full contents of - all posts (original posts + replies) - thread titles (unless it's already included in original posts) - of the [2017-1-1, 2020-12-7] (Dec 12 of this month) period. - exported to json Somewhat doable? Thanks, Adrien Le mar. 8 déc. 2020 à 14:57, Malcolm <malcolmlewis@opensuse.org> a écrit :

On Tue, 8 Dec 2020 15:08:49 +0100 Adrien Glauser <adrien.glauser@gmail.com> wrote:
Hi Adrien Please keep the forum-admin group in the reply, there are two other Admins to keep in the conversation. My only concern would be having to take the forums offline for such a data dump. But again, are there some specific metrics your after? Seems to me a crafted sql query would also reduce the data amount? @Jim & Gertjan any other thoughts or concerns? -- Cheers Malcolm °¿° SUSE Knowledge Partner (Linux Counter #276890) Tumbleweed 20201205 | GNOME Shell 3.38.2 | 5.9.12-1-default Intel DQ77MK MB | Xeon E3-1245 V2 X8 @ 3.40 GHz | Intel/Nvidia up 1 day 11:18, 2 users, load average: 0.01, 0.39, 0.80

Sorry Malcom, changed email client recently and it's a quite a battle still.
Exporting from the forums allegedly requires taking down the DB I certaintly DO NOT want to have the forums taken down at any point. I am not a MySQL guy but in my book `mysqldump` tries to dump without blocking. If that's not enought I am quite confident they are other ways to export in a non-blocking way. More knowledgeable folks reading this are warmly invited to confirm or infirm.
What metrics am I after? I can't be more specific than I've been already. We'll be doing data analysis with ELK so we need not more and not less than I've characterized. The goal is to identify patterns in issues and pain points after performing full-text analysis.
Have a pleasant afternoon, Adrien Le mardi 8 décembre 2020, 15:41:00 CET Malcolm a écrit :

Op dinsdag 8 december 2020 15:41:00 CET schreef Malcolm:
Same here.
But again, are there some specific metrics your after? Seems to me a crafted sql query would also reduce the data amount?
I'd do this with a proper query rather than mysqldump.
@Jim & Gertjan any other thoughts or concerns?
Private Messages would also be included. The feature may not be used much, but to me private is private, and these data should be excluded.
-- Gertjan Lettink a.k.a. Knurpht openSUSE Forums Team

Knurpht-openSUSE wrote:
That should not be necessary, I think.
The issue is - that requires vBulletin internals knowledge that I don't have. If anyone can write the query, I would however be happy to run it. Anyway, there is an open ticket #80858, maybe it is best to continue there? -- Per Jessen, Zürich (1.5°C) Member, openSUSE Heroes

On Tue, 08 Dec 2020 17:22:44 +0100, Knurpht-openSUSE wrote:
Good catch, and I would agree with that. The same holds true for any private forums we have, even though some on the heroes team have access to that data anyways - we wouldn't want to expose that data publicly. -- Jim Henderson Please keep on-topic replies on the list so everyone benefits

On Tue, 08 Dec 2020 08:41:00 -0600, Malcolm wrote:
I think a database dump like this would be possible without taking the forums offline - it may even be possible to export from the admincp. Exported to JSON might require data transformation - but I think you've got it covered in terms of looking for information about what the metrics are that are being sought. It's possible that data might be available even without doing a database dump. -- Jim Henderson Please keep on-topic replies on the list so everyone benefits
participants (5)
-
Adrien Glauser
-
Jim Henderson
-
Knurpht-openSUSE
-
Malcolm
-
Per Jessen