By Philip on Thursday, 07 August 2014
Posted in General Issues
Likes 0
Views 593
Votes 0
I've been tasked with maintaining my school's student website. I'm in class 2017, and there are two classes before mine (2016, 2015).

The students from the previous two years primarily used Kunena to post outlines, and other study materials. The posts in Kunena were a breeze to import into ED, everything went as planned, no problems whatsoever.

The maintenance of the site was lacking during my first year, and subsequently all the outlines from my class were posted to a Facebook group.

I've created a Facebook developers account and have successfully used the Graph API to filter all posts with an attached file. The generated output contains a URL, date, username, and a message.

I'm looking for a way to get this data into ED because I believe that this will directly correlate into a quick adoption rate for the website. I'm imaging a excel spreadsheet which contains the Username, Joomla User ID, Message, Date, URL, and ED Category. Once complete I need a way to import this into ED.

What type of options are available for me to realize this goal?

any help is most appreciated.
Succes!

Facebook Group Files --> Kunena --> EasyDiscuss

Note: This assumes you are a Admin of the Facebook Group

1. Create Facebook Developer Account and Create an App.
2. In the FB Graph API run
{your Facebook group number}/files?fields=download_link,from,message,updated_time&limit=1000

3. Copy this into a text editor, and massage the info into an .html document
4. Grab a browser extension that can download all files on a page, open the .html and download the files.
5. Remove whitespace from file names. (I used Applescript)
6. Upload files into a new folder in
media/kunena/attachments/PICK_A_NUMBER

7. Go back to the .html and massage it into a semicolon separated .txt file, remove the
https://..../  from  https://..../FILENAME.pdf
decode the text. e.g. replace
%20
with a
space
, import this .txt into Excel
8. Create Excel sheets for jos_kunena_attachments, jos_kunena_messages, joe_kunena_messages_text, jos_kunena_topics, jos_kunena_user_topics
9. Use the title of each table column for the first row of the excel sheet
10. Make sure that kunena_topics
id
matches kunena_messages
thread
, and kunena_topics
first_post_id
matches kunena_messages
id
. (pick an arbitrary number above the last id number in your sql table)
11. In the other tables when you see
mesid
that should match kunena_messages
id

12. Copy the columns from Step 7 into the appropriate columns in the _kunena_ excel sheets
13. Match up the Joomla! Username & ID with the respective column in the all the tables (excel sheets)
14. Fill in the other rows using a previous post from your Kunena forum as a template (note: I used the decoded file name as the subject)
15. Create UNIX timestamps from the FB dates (good one here)
16. I used the command/terminal to get the file size of each file. (cd into folder and run
ls -l
, copy into the text editor and remove everything but the filename and file size)
17. Export the Excel sheets as a .csv, open the csv and remove the top row, remove any extra commas at the end of each line, and make certain the last line is an empty/new line.
18. In phpMyAdmin select the table and import the csv
19. Go back to JoomlaAdmin / EasyDiscuss and rerun migration


Note: When I did the above the new posts were created in a single category, despite me specifying otherwise. I manually changed the category in
jos_discuss_posts
. Small price to pay considering everything else.

I hope this helps someone one day. Cheers!
·
Sunday, 10 August 2014 02:54
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello Philip,

I am really sorry but unfortunately there's no way to actually import from an excel spreadsheet currently I wish it was possible though.
·
Thursday, 07 August 2014 02:05
·
0 Likes
·
0 Votes
·
0 Comments
·
Would it be possible (and I'm hoping you can answer the possibility aspect) to import a csv into mysql, merge the database with Kunena, and then run the ED migrator again?
·
Thursday, 07 August 2014 02:30
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello Philip,

If you are able to migrate these data into Kunena, you can then migrate them over from Kunena over to EasyDiscuss. It's possible because EasyDiscuss keeps a history of items that it has already migrated so it doesn't migrate the same data multiple times.
·
Thursday, 07 August 2014 11:46
·
0 Likes
·
0 Votes
·
0 Comments
·
This is what I'm going for.

Can you confirm the kunena tables the migrator uses?

jos_kunena_topics, jos_kunena_attachments, what else is required?
·
Thursday, 07 August 2014 23:50
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello Philip,

These are the tables that EasyDiscuss would search for:

- jos_kunena_topics
- jos_kunena_messages
- jos_kunena_messages_text
- jos_kunena_attachments
- jos_kunena_categories
- jos_kunena_polls
- jos_kunena_polls_options
- jos_kunena_polls_users
·
Friday, 08 August 2014 03:16
·
0 Likes
·
0 Votes
·
0 Comments
·
Thanks Mark.

I have one more question for you, and I know this is outside the scope of the forum, so all I'm really hoping for is a link to read.

I'm down to the final bit, and I'm walking into uncharted territory (for me at least). I'm noticing the hash column in jos_kunena_attachments, and I'm wondering how I can generate a hash.

I'm assuming this a hash of the filename column, but I don't know the type (md5 or sha1 or other?), and I don't know if phpAdmin can create this column for me?

thanks again.
·
Friday, 08 August 2014 08:12
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello Philip,

I believe it's probably just an MD5 of the original file name but I am not really certain about this. You should try posting this in Kunena forums
·
Friday, 08 August 2014 10:59
·
0 Likes
·
0 Votes
·
0 Comments
·
Hey Mark,

I'm trying to run some tests on another site before I migrate. When I try to change the active domain in my dashboard I'm receiving an error "Sorry, but you do not own this license."

best,
·
Saturday, 09 August 2014 21:56
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello Philip,

You don't really need to update this to your test site, as the domain is merely used for support only
·
Saturday, 09 August 2014 22:12
·
0 Likes
·
0 Votes
·
0 Comments
·
Good to know.


I have two questions for you regarding ED.

1. When I ran the migrator on the test site the topics were not categorized correctly. Is there a way to edit the sql table for ED topics so that they reflect the correct category, or is there a better way? I know this isn't optimal, but everything else seems fine with the migration, so I don't want to rerun if I don't have to. Although, I'm now running the migrator on the live site, so maybe there was a difference somewhere and the categories will be correct.

2. It seems that a user can edit the replies of others, and I'm not sure where in ACL I can change this. The 5th option down "Edit Replies" doesn't seem to affect this function.

If this migration works I'll post the method here for completeness.

thanks again:)
·
Saturday, 09 August 2014 23:13
·
0 Likes
·
0 Votes
·
0 Comments
·
Thanks for sharing Philip, glad that your issues are resolved now
·
Sunday, 10 August 2014 03:07
·
0 Likes
·
0 Votes
·
0 Comments
·
View Full Post