Size of Storage Needed
By David Judah on Sunday, 02 March 2014
Posted in General Issues
Replies 4
Likes 0
Views 633
Votes 0
Hi there,

I posted a few days ago question about hosting but now wanted to see if people can share their experience/opinions about storage size. As with a lot of you, I am about to launch my EasySocial based social network and am feeling positive about it all. When I started thinking about the storage size I felt a bit overwhelmed and it looks like I am about to gauge an expensive and nearly impossible to find hosting company to fit the needs for such a venture. Here are my calculations:

Based on a 10,000 users uploading photos only (as I will not allow videos or file sharing) at a rate of 1 photo per day at an average size of 1mb (that is 10,000 x 1mb per day = 10GB). Of course I would need to mirror my site which doubles the size per day to 20GB. Even if I was able to afford a dedicated server with 1TB, it will only keep me going for 50 days, and that is only with 10,000 users.

Now, this calculation seems simple but I think it is logical and not over generous. I know that some users will not upload an image a day and some will do more then one. I know that image sizes vary but even if I was too pessimistic, and you could half my calculations, I it is pretty bad!

I know about Amazon S3, but it is not that cheap either. I presume I won't be able to user the Amazon glacier (which is very cheap), so I will have to use the standard which is only a touch cheaper then other companies:

Amazon S3: 0.085 per GB per month (meaning that 100GB will cost 8.5 euros per month)
Rackspace: 0.01 per GB per month (meaning that 100GB will cost 10 euros per month)


I would very much appreciate any member of the community thoughts on the matter:)
David.
Hi Mark,
10,000 users is not a big deal IMO, I used to admin a social site with +10,000 users but ES wasn't born yet.

Hi David,

1- if your site has 10,000 users then as Mark has mentioned you should try to monetize it as much as possible through ads. That money could cover the hosting and space cost.

2- Limit the number of pictures a user can upload/day (not sure if this is available in ES, I know it was available in JS)

3- Resize the picture before uploading (max image size set in the backend), that will save you tremendous amount of space ( not sure if this will be available in 1.2, I know I have asked for it many times) . Set the quality to 80%, you can shrink a picture from 10MB to 500k, better would be to integrate Yahoo's Smush It, they have an API and and i am using it with one of JA component to reduce the size of the picture before uploading. Nowadays, pictures are huge in sizes because of these high Megapixel cameras, and I don't like to put a restriction on my users by telling them you can only upload a 5MB file, I usually put a high number like 20MB in the backend and let ES do the resizing. This will definitely save you a lot of space.

100 GB for $8 / month from amazon, I don't think you can beat that. 1T would be around $100, again if your website is poplular, the ads should be able to pay for it. People spend $1000+ to $100,000+ for very popular site with many users, and where do you think the money comes from ---> ads.
checksum
·
Monday, 03 March 2014 10:36
·
0 Likes
·
0 Votes
·
0 Comments
·
I don't really have the stats for you but what I do know is that, if you have 10,000 "active members" on your site that is constantly posting photos, you better be sure to activate Google ads on the site and start raking in money from it.

However, with a 10,000 "active member site", I don't think your above question wouldn't even be a problem You should worry more about how Google is going to pay you the cheque or perhaps what name should the cheque be written to each month
Mark
·
Sunday, 02 March 2014 20:10
·
0 Likes
·
0 Votes
·
0 Comments
·
Add into account also some RAID configurations and the things gets more expensive in terms of disk space.

Anyway, if your community is going to grow to that scale you better start monetizing it.

Also, if you reach that scale you must start to think BIG in term of infrastructure, think about dedicated DB servers, dedicated storage servers, load-balancing, serious server tune-up (NGINX instead of Apache) ... cache ... cache ... cache and again ... cache (memcache or other)

You will have more things to worry about, it's not just the storage space. (How about the bandwidth? Users will not only upload data but also "consume" it ... so, you better wire that server with some serious traffic pipeline )

In term of just storage needs, i know a pretty decent dedicated hosting provider in Europe (i am their client) where you can find 15 HDDS X 3 TB dedicated storage server for around 300 euros monthly (that is 45 TB of space)
Mist
·
Monday, 03 March 2014 10:54
·
0 Likes
·
0 Votes
·
0 Comments
·
Something that I don't really like with "cheap disk spaces" is the speed of the disks. I would have more preferences over SSD but of course you get for what you pay for
Mark
·
Monday, 03 March 2014 12:02
·
0 Likes
·
0 Votes
·
0 Comments
·
View Full Post