Loading
Network error

Agorakit : Users & developers

Export/import

Started by Zerline in Agorakit : Users & developers May 29, 2025 10:42 AM

Should we allow group administrators to import/export data for their group?
If so:

  1. is there anything too private that should not be exported?
  2. what about invites or notifications? do we need them in an export?
  3. are "activities", "activity_log", "documents" tables still in use ?
May 29, 2025 10:46 AM

Answering to myself:

> 1. is there anything too private that should not be exported?

For the `users` table, I tend to keep only usernames. Drop email and encrypted password. And location. Paranoid?


May 29, 2025 1:45 PM

It depends on the use case. Imho, users should trust group admins, those are friends helping a group for technical stuff. This can be technically discussed.

However they must trust server admin and hosting company (no choice but trust in this case), since they have access to the raw data.

May 29, 2025 3:50 PM

I have 2 use cases in mind:

  1. backup: the groups admins might prefer to keep their own backups on their own machines
  2. server migration: when you want to move your group to another service provider ; this will happen rarely ; therefore, in that case imho users can be asked to re-register

Some users can be paranoid. Given what happens ordinarily on the net, that is legitimate.

May 29, 2025 5:36 PM

I would give the same data in the backup as the user is allowed to see.

May 29, 2025 5:37 PM

Which means a group admin can see emails but not passwords of course 

May 29, 2025 10:11 PM

Passwords :

  • either we export what is in the database -- ie encrypted passwords ; meaning that users can login directly into the new server
  • or we have to get into some re-registering in a way or another
May 29, 2025 10:24 PM

Technically, my proposal could be :

  • export to JSON  : the group, its users & membership, its actions, discussions+comments & files, relevant password_resets, reactions & social_profiles,  all taggable_tags -- as they don't belong to groups, relevant taggable_taggables
  • import :    
    1) rebuild objects

    2) with the help of table indexes, check if group exists in DB, same for users ; if they exist, print some information and terminate ; if they don't exist already, insert them and retrieve their IDs

    3) insert actions, discussions, files
    4) check tags, create them if necessary (colors?), finally insert taggable_taggables


May 30, 2025 8:45 AM

Note that this feature is already almost implemented as an artisan command. You can look in the code.


What is missing is tests, and ability to do it from the web UI instead of the command line.

If you could review this unfinished code it would be great.

laugh
May 30, 2025 9:02 AM
export to JSON

or to CSV: more readable format, better for the user

When migrating a group to a new server, I would:

  1. let all users register on this new server and send me their usernames ; then edit the CSV file to fix any usernames that might have changed
  2. create the group & fix slug in the CSV if necessary
  3. import everything else automatically

It looks complicated, but it is safe and realistic.

May 30, 2025 11:57 AM

from the web UI

within admin dropdown in views/partials/tabs/groups.blade.php ?


May 30, 2025 5:22 PM

yes

June 1, 2025 10:36 AM

Fyi: I've done something in this branch: https://codeberg.org/zerline/agorakit/src/branch/export

(mirrored, with a few hours delay, on https://github.com/zerline/agorakit/tree/export)

I've kept your JSON strategy for now.

June 1, 2025 9:41 PM

I'd prefer to have a new ExportService (with the related ImportService) or something like that instead of bloating the group model with even more code.

June 2, 2025 11:32 AM

Done.

Note: I created a temporary storage (Storage::disk('tmp')). And I just pushed some code to cleanup that storage every hour.

You've read everything in this discussion