By JV on Monday, 16 March 2015
Posted in General Issues
Replies 18
Likes 0
Views 1K
Votes 0
When using Amazon S3 for Remote Storage of Story photos, when someone posts a photo, does the photo go to the local server first so it can be accessed immediately? And then is it later uploaded to S3 when the CRON job runs? Or does it wait for the CRON job to run before it can be seen on the front end?

If it is put on the local server first, does the local file get deleted after it is uploaded to S3?
Hello JV,

I am sorry for the delay of this reply.

When using Amazon S3 as remote storage, when the user upload a new photo, the photo will be stored in local server first. After the next cronjob is running, the photo will then get uploaded to the amazon s3 server and it will delete the photo in local server.

Hope these can clarify your inquiry
·
Monday, 16 March 2015 12:25
·
0 Likes
·
0 Votes
·
0 Comments
·
Please can you provide a step by step guide on how to set up the cron job. I know how to set up a cron job on my server. But I don't know what I need to actually run. Please can you point me to some simple, step by step documentation. I have had a good look and can't find what I need.

I have rummaged through the Forum and found:

https://www.mysite.co.uk/index.php?option=com_easysocial&cron=true

And run this. But I get:

[{"status":"200","contents":"There is nothing to dispatch currently.","time":"2015-06-15 21:05:45"},{"status":"200","contents":"No photos to upload to remote storage","time":"2015-06-15 21:05:45"},{"status":"200","contents":"No avatars to upload to Amazon S3 right now.","time":"2015-06-15 21:05:45"},{"status":"200","contents":"Nothing to process for files","time":"2015-06-15 21:05:45"},{"status":"200","contents":"Stream archive disabled.","time":"2015-06-15 21:05:45"}]

And there is definitely a load of stuff (about 350MB) that needs uploading from the Photos folders alone.

Please help
·
Tuesday, 16 June 2015 05:00
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello Adam.

You follow this cronjob setup documentation for easysocial. http://stackideas.com/docs/easysocial/administrators/cronjobs

S3 will be trigger using the EasySocial cronjobs.

Hope this help
·
Tuesday, 16 June 2015 10:29
·
0 Likes
·
0 Votes
·
0 Comments
·
Hi Izzan.

As per my post above, I have found the Cron Job information and added it in and I get:

[{"status":"200","contents":"There is nothing to dispatch currently.","time":"2015-06-15 21:05:45"},{"status":"200","contents":"No photos to upload to remote storage","time":"2015-06-15 21:05:45"},{"status":"200","contents":"No avatars to upload to Amazon S3 right now.","time":"2015-06-15 21:05:45"},{"status":"200","contents":"Nothing to process for files","time":"2015-06-15 21:05:45"},{"status":"200","contents":"Stream archive disabled.","time":"2015-06-15 21:05:45"}]

I have tried numerous different bucket paths.
I have checked the Secrect ID info

It is updating the database table "social_storage_log" to say that the files are on Amazon

BUT................. NOTHING is in the S3 Bucket.

I have looked on documentation for STORAGE and there is not really any detailed information or step-by-step explanation as to what to do or what the implications are (e.g. 10files sync at a time... Up it? To what? What are the consequences)

And there is no example as to what the bucket path should look like.

I have looked at:

connecttvt-website-easysocial
connecttvt-website-easysocial.s3-website-eu-west-1.amazonaws.com
https://s3-website-eu-west-2.amazonaws.com/connecttvt-website-easysocial
https://s3-website-eu-west-1.amazonaws.com/connecttvt-website-easysocial


Nothing seems to work.
·
Tuesday, 16 June 2015 13:50
·
0 Likes
·
0 Votes
·
0 Comments
·
I have TRUNCATED the table and tried again as it had got to the end of all the images. It is saying that it is uploading 10 Avatars... But AWS Bucket is still very very empty.
·
Tuesday, 16 June 2015 14:10
·
0 Likes
·
0 Votes
·
0 Comments
·
I have also add PERMISSIONS to the Bucket and granted permissions to List and Upload/Delete to all Authenticated Users....

Still not working.
·
Tuesday, 16 June 2015 14:11
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello Adam.

Can you provide us with joomla backend, FTP, Cpanel and Amazon s3 access so we can check on the issue directly?
·
Tuesday, 16 June 2015 15:41
·
0 Likes
·
0 Votes
·
0 Comments
·
Yes... Can you confirm what is most secure way of sending these to you?
·
Tuesday, 16 June 2015 22:52
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello Adam.

You can write all your access under site details tabs. Only stackideas team can view this site details

http://screen.stackideas.com/2015-06-17_1039.png
·
Wednesday, 17 June 2015 10:40
·
0 Likes
·
0 Votes
·
0 Comments
·
Hiya.... Please find info requested in the TABS as per your email. THANKS !

: -)

Adam
·
Thursday, 18 June 2015 02:39
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello Adam.

I am really sorry because I can't access your AWS by using the provided information.

Please advise me
·
Thursday, 18 June 2015 11:43
·
0 Likes
·
0 Votes
·
0 Comments
·
Ummmm site details below:

I have tested them. They log in great for me. There is no IP blocking or any whitelisting or anything.
·
Thursday, 18 June 2015 15:23
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello Adam.

I have re-create the new bucket in your s3 and set the region as US standard and create a new access key for your s3. But not all of the photo upload to your s3.
http://screen.stackideas.com/2015-06-18_1635.png

I suggest you to update your easysocial the latest version because we have make some fixed for amazon s3.


Looking forward for your reply
·
Thursday, 18 June 2015 16:42
·
0 Likes
·
0 Votes
·
0 Comments
·
Hi Izzan,

Great stuff. Thanks for the support. We have now upgraded Joomla AND EasySocial as well.

We cleared out the "social_storage_log" table and everything now seems to be uploaded onto AWS...

However.... Just a final question:

When do the images get cleared off the server. And, what happens if there are some that seem to be 'left over' on the web server.

I have checked through the folders and there seems to be some that haven't been deleted. They have been uploaded to AWS... but not cleared off. The CRON job has definitely completed ALL the images and run multiple times since it finished uploading...

Thoughts?
·
Friday, 19 June 2015 02:52
·
0 Likes
·
0 Votes
·
0 Comments
·
Sorry... one more thing. Evidently, there are a lot of empty folders now in: /media/com_easysocial/photos on the web server.

Can I just remove the ones that are genuinely empty... Or do they need to remain on the server for any reason ?
·
Friday, 19 June 2015 02:54
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello Adam.

By default all the easysocial's photos will be store in S3. May I know with folder haven't been deleted from web server.

Regarding the second question. Please don't remove the empty folder because your user upload a new photo, this photo will be store in your web server first before the CRON run.

Hope this help
·
Friday, 19 June 2015 10:22
·
0 Likes
·
0 Votes
·
0 Comments
·
Thanks Izzan....

Most are empty. But here are a few that aren't:

/media/com_easysocial/photos/100/176
/media/com_easysocial/photos/103/187
/media/com_easysocial/photos/103/189
/media/com_easysocial/photos/103/192

/media/com_easysocial/photos/107/200
/media/com_easysocial/photos/107/201
·
Friday, 19 June 2015 15:43
·
0 Likes
·
0 Votes
·
0 Comments
·
Hi Adam Clark,

I'm really sorry that delayed of this reply,

After investigated long time ago, I still can't find out what is the reason some of the photo can't upload in your Amazon S3 and those image uploaded in your Amazon S3 but the image file still exist on web server.

By the way, I already manually help you uploaded those image which stored in your web server. But those image stored in your web server, you have to manually delete it.
/media/com_easysocial/photos/100/176
/media/com_easysocial/photos/103/187
/media/com_easysocial/photos/103/189
/media/com_easysocial/photos/103/192
/media/com_easysocial/photos/107/200
/media/com_easysocial/photos/107/201
·
Wednesday, 24 June 2015 12:18
·
0 Likes
·
0 Votes
·
0 Comments
·
View Full Post