Hello Gavin,

Hm, there's no reported cases of such issue so far. Did you integrate it with Amazon or anything like that? By the way, do you have a backup of these photos?

I don't use Amazon bucket no, but I do use a CDN to cache the images.
I have been checking logs and my web server logs show the 404 errors on all ES images (photos, cover photos etc.) starting at 13:52 on the 1st May. Looking through my linux server logs and there are no SSH or any other logins around that time at all so it's not something I have done by mistake. There were no disk errors or anything that I can find - and it's just the photos folder - everything else in the media/com_easysocial folder is still intact.

In fact, at around the same time a user posted a photo on the stream but it didn't show because I actually commented that I couldn't see it and could they upload it again. I have posted a link to the story item in the site details, if you wouldn't mind taking a look Mark?

Luckily I have a backup from the end of March so I have recovered most of the photos - just ended up losing April really.

Hello Gavin,

Can you try monitoring this for the coming weeks, I need to know what and why it happened (Also, remember to make more backups for just in case purposes)

Yeh I'll keep a close eye - and yes, more backups would be wise

Thanks Gavin!

Sadly this just happened again tonight. The photos folder in /media/easysocial/ has completely gone from my system.
The only thing I did with the server today was edit some CSS files and upload some images to the joomla /images folder. No changes to any ES settings were made or anything, only my template CSS files were changed.

This is actually a brand new server that I purchased just last week, with a new installation of Ubuntu 12, so it's not a hardware related or OS problem.

I'm currently in the process of restoring 6g+ of photos from my backup but I'm really not happy about this happening again. I believe there must be a nasty bug somewhere in ES because as I said, I haven't touched any ES settings or files today except for a few CSS files, but they are in my template, not even in the ES folders.

Please take a look at my setup and see if you can find anything. You have full root access. Details attached.

In fact, just about an hour ago I deleted a photo somebody had posted. I deleted it by viewing it in the popup viewer and selecting Delete Photo from the dropdown menu.
May or may not be related but it's the only thing I can think of related to photos today.

Confirmed it is Easysocial deleting the entire photos folder. I was just going through some users albums and cleaning up the photos that were lost, ie showing broken links, and I got an internal server error warning (see screenshot). After that, the entire photos folder is missing again.

To replicate this error:
Delete a photo from the photos folder so you get a broken link in the ES album/photo view.
View the missing image in ES popup viewer and select Delete Photo.

I believe that is exactly what happened for me but I'm not trying it again because I just had to restore them all again for the 2nd time tonight.

I would really appreciate a fix for this as soon as you can work it out because I expect my users will start cleaning their albums of broken images and I'll be back to square one again.

Hello Gavin,

I am sorry for the delay of this reply.

Regarding your issue, I cannot replicate this issue from my local. I updated my EasySocial to version 1.2.15 and then i followed your reproduce steps but I cant replicate the issue.

By the way, may i know when this issue happened, did the system delete the entire photos folder or just the photos in that particular album? The reason I am asking this so that I can run some test on your site to see the issue but I need to make sure if the issue only deleted up to user album level and it will not affect other users photos. Or is there any better way to test this issue on your site?

Please advise.

Yes it's the entire photos folder as I said before, so it's all users photos. The photos folder in /media/com_easysocial/ is getting deleted.

I'm not sure exactly what time it happened but it was some time yesterday morning (12th June). I can see from my nginx logs all the errors trying to find the images.

If you want to experiment with my site, go ahead, but please make a copy of the photos folder first so you can just replace it each time it gets deleted and I don't have to dig through backups again.

Hello Gavin,

This is what I tried on the site:

1. Created a new album
2. Uploaded a photo in this album
3. Clicked on the photo to render the popup
4. Clicked on the delete button in the popup menu
5. All other photos are still on the site

Looking at the codes, it really wouldn't delete the main "photos" folder because it always appends the album id and the photo id to the path.

Mark, you need to remove the image from the actual photos folder for this to happen I believe. I was going through deleting some broken images and when I deleted one of them in the ES front end, the error occurred as I posted in the screenshot a couple of posts up.

It is definately ES deleting the folder. It happened in front of my eyes lol.
I will try it again today after making a fresh copy of the photos folder and try to pinpoint the exact photo that does it.

ok as promised I tried again today and exactly the same thing happened again. After deleting about 10 broken images, the entire photos folder was deleted by Easysocial.

This is exactly what happened:
Using the popup image viewers menu I selected Delete Photo for broken images 1 thru 8.
Using the popup image viewers menu I selected Delete Photo for image 9 (/media/com_easysocial/photos/6627/12772/melon_original.jpg) and it looked like it was successful and moved onto the next image but right at that moment the photos folder was deleted.
Pressed the left arrow (the one that appears when you hover over an image in the popup viewer) to go back to the previous image and that is what you see in the attached screenshot. "Internal Server Error" warning.

I have added details of the user's profile to the site details.
I made a copy of the photos folder before and have now copied it back. When viewing this users album, the broken photos are now gone as expected because they were removed from the database when I deleted them.
However, there are more broken photos in their "Story Photos" album which you can try deleting and see if the same happens again for you.

Please check at your earliest convenience because it is not realistic for me to make hourly backups of a 6gb photos folder and is disheartening for my users to see their photos deleted all the time.

Subscribed.

Subscribed

Hello Gavin,

Can you please provide me with one of the album that has these broken photos so that I can try to delete them? Also, remember to make a backup first.

Hi Mark, I already provided you with the users profile url in the site details tab of the last reply and said to check the Stories album since it contains broken photos.

Hi Gavin,

Regarding your issue, I've added a checking in the code so that before the system attempt to delete a photo, it will check first if the physical files are there or not to avoid wrong folders / files being removed from your server. I then tried remove few photos ( broken photos ) from this album '......../6626-story-photos/user/36938-squid' and the fix is working fine. No other photo folders being removed after the test.

Can you please try again and see if the fix working for you?

Attached is the file I've modified. For people to want to test, download the file and copy to folder ' JOOMLA/administrator/components/com_easysocial/includes/storage/adapters/joomla/' and overwrite the existing 'joomla.php'.

Remember to backup the original file first before you overwrite the file.
Hope this help and have a nice day

Thanks for this fix Sam,
So far it seems fine. I have deleted several broken photos and albums and the photos folder isn't getting deleted anymore.
I'll keep backing up regularly and keep a close eye just incase, but it looks to be fixed.

Thanks again

Hi Gavin,

You are welcome and glad to hear the fix worked

Sure, do let us know if you still encounter the same issue and we will then check again on your site. The above fix will be added into next release of EasySocial

Have a nice day

Please check my site - the photos folder was deleted yet again. The oldest file/folder in there now is from around 2:40pm today so it must have happened shortly before that.

Please please please fix this because I'm really not impressed that this software can just delete an entire folder from my drive. It's really quite upsetting to see content lost again and again when trying to build a community. I do love your software but this really is a showstopper for me - I now have a site full of broken images again and I spent absolutely hours going through everyone's albums removing them from the last times this happened.

I have been with ES since beta and would have accepted something like this happening during that period, but now it's just not acceptable to have such a bug in paid software.

I would really appreciate it if you could look into this at your earliest convenience. I have not recovered the photos from my backups - I don't want to touch anything until you have checked it.

I have added more details and urls to the site details.

I have recovered from my backup now and luckily only lost a couple of days worth of images.
Please go ahead and do what you need to do with my site to fix this problem guys. I can apply the backup again if necessary.

Hi Gavin,

I am sorry for the delay of this reply and thank you for your patient for sorting this issue with us.

I've tested at your site by removed some of the broken images ( followed your reproduce steps in your early posts ) and i then check the photo folder and no other folders being removed. It seems like everything working just fine.

I also tried delete the broken image from this item 'yoursite.com/stream/item/29264' and the db record was removed but the rest of the photos folders are still intact. So i can't really replicate the issue from your site

Anyway, I've added a small block of code to detect if the folder removal is coming from EasySocial or not. If you take a look at file JOOMLA/administrator/components/com_easysocial/tables/photo.php at line 650 ~ 659, when someone attempt to delete a photo and the path is equal to path that you've specified in storage path setting, then the script will create a log.txt in the same directory.

Can you please monitor for us if this log.txt file created in 'JOOMLA/administrator/components/com_easysocial/tables' or not. If yes mean something is not working right and we will then investigate further.

Please advise.

Thanks Sam
I'm not absolutely certain that it's when you delete photos - or certain that is the only cause. It is what happened before as I mentioned in my previous posts but this time I'm not so sure. It's hard to know if one of my users deleted a photo and that is what caused it this time - all I can see happening around the time the folder was deleted was somebody posting a story with an image on the stream (the url I pasted in site details).

So this code will do what exactly? Create a log file every time somebody deletes a photo, or just when the entire folder gets deleted again?

Hello Gavin,

A couple of us were troubleshooting your issues earlier Gavin as we too are pretty puzzled as to why this is specifically happening on your site. There is no way on earth the code could have done this because we added tons and tons of checking on the folders. However, we added a specific log file whenever it tries to delete an album folder or the entire photos folder so that we know where to trace it back to.

Thanks Mark but I have to disagree with you because it most certainly did do it, on several occasions. Nothing else could have done it in my opinion - it's a specific Easysocial folder and nothing else uses it or even knows about it to delete it. Why would it only be this folder that keeps getting deleted? It's nothing to do with my server hardware or software either since I moved to a completely new dedicated server a few weeks ago, new hardware and new OS.

I do have a couple of modifications directly in the ES code but they are relating to registrations - just a piece of code to add new accounts to an SQL Server database and update passwords in SQL Server when they are updated in the profile, and a small modification to the joomla password field to check existing names in the SQL Server database. Other than those it's just a few theme changes for the toolbar, profile and login. Everything else is standard.

It does puzzle me slightly that nobody else has had this problem. Could it perhaps be something relating to the folders size? It's over 6gb of photos in there. I'm not sure how many there are exactly but it's 7.4k folders in /photos and each of those has at least one folder and image (x6 because of all the different sizes), so it's quite a lot. Most of these were migrated over from Jomsocial.

Permissions seems fine too. www-data owns folders(0755) and files(0644).
Is there any way to make the photos folder owned by root so that it can't be deleted by the webserver? I realise this would probably stop folders and images being deleted when a photo or album is deleted in ES but it would still get removed from the database and just leave the files orphaned on the drive wouldn't it? Not ideal ofc but just thinking aloud really.

Hi Gavin,

I am sorry for the delay of this reply.

Just wondering if you have any cron job or backup script running on your server that do some file cleaning process?

>> Could it perhaps be something relating to the folders size? It's over 6gb of photos in there.
Em, possible but if the issue related to folder size, then your user shouldn't be able to upload any photos ( I can still upload photos with EasySocial ).

Is there any way to make the photos folder owned by root so that it can't be deleted by the webserver? I realise this would probably stop folders and images being deleted when a photo or album is deleted in ES but it would still get removed from the database and just leave the files orphaned on the drive wouldn't it?

Yes, if you set your photos folder to owned by root, the db entries will still get deleted. Also, this might cause your user no longer able to upload photos

Anyway, I somehow suspect the photos gone removed from 'outside' EasySocial. Im just guessing. If you want, you can edit Joomla JFolder libraries and do a similar log like the one we added into your /tables/photo.php to see 'who' or 'what' actually calling the Joomla JFolder::delete method.

Edit the file 'JOOMLA/libraries/joomla/filesystem/folder.php' and add the below code at line after 366 and line after 359:

$jConfig = JFactory::getConfig();



ob_start();

var_dump(debug_backtrace(2));

$contents = ob_get_contents();

ob_end_clean();

JFile::write($jConfig->getValue( 'tmp_path' ) . '/delete_log.txt', $contents);

Please make sure you backup the original file 1st before you apply the above hack.

The above hack will create a log file at your JOOMLA/tmp/ folder when ever there is a folder deletion going on.

I have a cron job setup for Easysocial so that it can send out mails - that's about it. No automated backups are setup either, I do it manually at the moment.

I don't think anything outside of easysocial is deleting the folder - there is nothing that I know of that does anything with that particular folder apart from easysocial itself.

Thanks for the code - I'll add it now.

Hello Gavin,

As posted earlier, the only way to determine this is when the next event of this folder being deleted (It will be logged by our scripts) and we can then see if it's going through our codes or not

Hi Gavin,

Just wondering how is everything goes? Is there anything being logged in delete_log.txt ?
By the way, just wanna ask if recently you deleted some of your users from Joomla user manager? Easysocial do have a user plugin which will delete photos when the user get deleted. But this plugin should only delete photos for that particular user. So I just want to make sure your issue was not caused by this user plugin.

Please advise.

Hi Sam,
So far nothing has been logged from the ES side of things - ie, the code you guys added to the photos.php but the logging from the code you told me to add to JOOMLA/libraries/joomla/filesystem/folder.php is logging stuff when an album is deleted.

So far the photos folder has not been deleted again but I check it everyday and make an incremental backup just in case.

I do sometimes delete users via the Joomla user manager yes. I'll delete some today and see if anything happens.

Hello Gavin,

Perhaps when it happens again, let us know because if it goes through our library for "deletion", it will generate a log file for us.

Hi,

For me too, ti is the second time i've lost all my easysocial photoS

Réseau Vendée Pôle wrote:

Hi,

For me too, ti is the second time i've lost all my easysocial photoS

Hello,

We need more inputs regarding this matter so that we can replicate the issue. At the moment, we have no clue why the photos got removed from the server but if can you recall what you've did before all photos got deleted, then that might help us to identify the root cause of this matter. We are trying to isolate all the possibility of why this is happening.

Please advise.