By Mark on Friday, 16 May 2014
Posted in General Issues
Replies 44
Likes 0
Views 1.9K
Votes 0
Hey guys,

We're looking for a couple of sites that we can run some performance tuning on. If you have an active site that has more than 2,000 users or more than 200 users online concurrently, we would like to run some benchmarks and performance tuning on your site.

Here's what you need to do,

1. Provide us with your site admin access
2. Provide us with your FTP access
3. Provide us with your PHPMyAdmin access
4. Provide us with your root access (If you have a dedicated server, we would also like to review your LAMP stack setup).
5. We need your Skype access so that we can easily get in touch with you.

What will happen?
Basically, nothing will happen to the site. What we want to do is to find bottleneck areas on the site and try to fine tune it.
Hi,

Currently it works on 8 Core + 32 GB RAM Dedicated Server Aslo have another server -> 8 core 32 Ram + 512 SSD Storage (Clean Server)
Here my last results with Memcahed, Nginx, Mysql (works on tmpfs ramdisk) + CDN

3,32 sec loading min loading between 5-6 sec time (with my template+es+esgroup module+cometchat)
2,62 loading min loading time between 3-4 sec (with prostar+and only easysocial)

So, if you believe easysocial is not slow, and you can make it under or near 1 sec loding time i will give you 1000$, everyone Please witness My word!


I tested with SSD Storage
·
Sunday, 18 May 2014 16:19
·
0 Likes
·
0 Votes
·
0 Comments
·
Hi Mark,
Why not use your own site (Stackideas.com) , I believe you must have more than 2000 users. Then just move everyone over to the demo site. This will be a good way to test it.
·
Sunday, 18 May 2014 19:24
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello checksum,

We don't actually have the time to style and customize it yet currently so we don't really want to add this into our site yet
·
Sunday, 18 May 2014 22:16
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello gokhan,

Getting it to load < 1s is almost impossible unless you can offer a bounty of $100,000 then I will make it possible for your site with a super server / architecture
·
Sunday, 18 May 2014 22:39
·
0 Likes
·
0 Votes
·
0 Comments
·
You can't do it via hardening server. That is not a server side issue. we tried more then 100 maybe more best known way. So understand that all issue with easysocial query arhchitecture. So you must re-check your code structure and queries... I cant think easysocial with more query for next release. it would be load 5-6 sec itself...........

Aslo if i have 100.000, i will built my own sytem
·
Sunday, 18 May 2014 22:55
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello gokhan,

Yes, EasySocial takes a bit more load than usual extensions but considering that it performs much more functions, it's actually perfectly fine. You have to sacrifice something right

Just for your comparison, see this page, http://dev.edusosyal.com/testing . This loads purely Joomla articles (I have modified the toolbar module in EasySocial so it don't get processed) and probably jreviews plugin is modifying the output. This page alone also takes around 1.5 - 1.6 seconds and sometimes 2s - 4s too, see my screen shot here, http://screencast.com/t/G2w9JXRQfO . That is why I said it's close to impossible to reach < 1s

So, can you imagine what happens when you combine everything into one?
·
Sunday, 18 May 2014 23:22
·
0 Likes
·
0 Votes
·
0 Comments
·
By the way, turn off jqueryeasy because it's also taking a fair bit amount of load on the server. From what I have tested, disabling it shaves off about 300ms - 500ms
·
Sunday, 18 May 2014 23:30
·
0 Likes
·
0 Votes
·
0 Comments
·
Mark,

I dont have speed problem with other pages! All problems with dasboard and profil pages. It doesnt matter how other pages they are. if easy social loads 2-3 sec itself we cant use only ES alone...

what people say, speed speed speed, nothign more. You have a ferrari like car but it goes like a turtle. So what i can say?
·
Sunday, 18 May 2014 23:47
·
0 Likes
·
0 Votes
·
0 Comments
·
also that testing page loads 1.23 sec , i cant add screenshot here
·
Sunday, 18 May 2014 23:54
·
0 Likes
·
0 Votes
·
0 Comments
·
My home page loads 594 - 834ms
·
Monday, 19 May 2014 00:02
·
0 Likes
·
0 Votes
·
0 Comments
·
So you say there is no speed problem with easysocial? I wasted more then 8.000$ and more then 1000 hours to built my site. You should understand my complains about speed!
·
Monday, 19 May 2014 00:07
·
0 Likes
·
0 Votes
·
0 Comments
·
Commenting to receive e-mail notification of updates to this thread. I'm particularly interested in learning objective outcomes of this performance optimization initiative including a list of best practices or lessons learned.
·
Monday, 19 May 2014 00:41
·
0 Likes
·
0 Votes
·
0 Comments
·
Guys, after gokhan thrown the bounty of $1000, I think it's getting serious I want that money haha! Just kidding!

I believe we have discovered where the biggest bottleneck is coming from and it seems to be from the language loading. Joomla's language loading is extremely slow and we use it a lot!

I hope I am right on this because I could shave off about 600ms from my local instance by only loading 2 main language file. Gokhan, can you test your site now? Ignore the language string issues because in the next version we'll merge all the language file into a single file instead.
·
Monday, 19 May 2014 01:12
·
0 Likes
·
0 Votes
·
0 Comments
·
Gokhan,

I am very sure that your Joomla template is taking more than 500ms. I tried with the protostar with all plugins and modules enabled, and the page load is only within 1.2s -> 1.3s

With your other template, I am hitting 1.7s - 1.8s

Also, your front page (Non EasySocial pages) takes about 1s to load. If EasySocial can load everything with (language fixes), it only takes around 300ms - 400ms which is extremely good
·
Monday, 19 May 2014 01:35
·
0 Likes
·
0 Votes
·
0 Comments
·
Also, if you are able to plugin a SSD on your site, I think it will definitely speed things up by at least 20% - 30%
·
Monday, 19 May 2014 01:39
·
0 Likes
·
0 Votes
·
0 Comments
·
@Sam Famma,

That is not possible! Can i have a look at your web site?
·
Monday, 19 May 2014 02:22
·
0 Likes
·
0 Votes
·
0 Comments
·
Hi Mark,

I think you are right about language files issue, now loading time increased little, only one file will be good. I have some plans with my template but the most important problem with ES. After increasing ES loading time. Also you should consider some requests to load after page loading, sometings with ajax loading


I can get extra 0.5 - 1 sec. from other tweaks. So speed problems will be solved

By the way where is my toolbar ?
·
Monday, 19 May 2014 02:37
·
0 Likes
·
0 Votes
·
0 Comments
·
Subscribing.

(Mark, we really could do with a 'subscribe to this topic button').
·
Monday, 19 May 2014 07:08
·
0 Likes
·
0 Votes
·
0 Comments
·
Thanks Mark, we'll add this in the future
·
Monday, 19 May 2014 11:18
·
0 Likes
·
0 Votes
·
0 Comments
·
Hi Mark,

You can also combine some small stylesheets into one file if possible, for exmaple modules and apps
·
Monday, 19 May 2014 20:30
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello gokhan,

It would break IE because IE has a limitation on the number of css selectors allowed in a single css file
·
Tuesday, 20 May 2014 00:03
·
0 Likes
·
0 Votes
·
0 Comments
·
Hi Mark,

Language file eliminating or something didnt work at all

Here last result with new server 32GB Ram + 512 SSD storage + Apache+ Nginx + Memcahed = 2.37 sec best result with with Prostar Template (only dasboard, no modules, no chat, no others , etc)

And you can see also another result for one of my page (my template+ cometchat +Es toolbar+ some moules+ a jquery map with a big image, jreviews based page) Loads only : 940ms also easydiscuss page loads 1,3 sec with (my template+ cometchat +Es toolbar+ some moules+ )

I wonder what you will say Can I say Easysocial needs recoding? Or ?

Mysql and server status attached, Here my.cnf also;

################

[mysqld]
default-storage-engine=innoDB
innodb_file_per_table=1
innodb_open_files=1024
open_files_limit = 150000
key_buffer = 1024M
join_buffer_size = 16M
sort_buffer_size = 8M
read_buffer_size = 6M
read_rnd_buffer_size = 8M
myisam_sort_buffer_size = 128M
tmp_table_size = 256M
max_heap_table_size = 128M
query_cache_size = 4096M
query_cache_limit = 1024M
query_cache_type = 1
table_open_cache = 195535
table_definition_cache = 85535
thread_cache_size = 512
innodb_buffer_pool_size = 4096M
wait_timeout=60
max_connections = 4096
max_connect_errors = 1000
connect_timeout = 10
net_write_timeout = 900
net_read_timeout = 900
max_allowed_packet = 128M
max_delayed_threads = 4
local-infile=0
innodb_log_buffer_size=64M

##################################
·
Thursday, 22 May 2014 09:57
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello gokhan,

I believe your app / custom field language files are still being loaded somehow but if you already have SSD it shouldn't take that long. Because the last I tested on your site, without loading all these language files, it's between 1.2s - 1.6s
·
Thursday, 22 May 2014 11:29
·
0 Likes
·
0 Votes
·
0 Comments
·
Yes, without loading them, only a file with 2400 strings, but now there is a file with 5000 strings. Any way, that is not an issue.

So you must accept that there is a problem with dasboard and profil page, so need to find it and fix it.

Look at group page loading time
·
Thursday, 22 May 2014 17:56
·
0 Likes
·
0 Votes
·
0 Comments
·
Hm, yeah your live site is different than your dev site earlier. I think the "data" that you have on your live site is different than your staging / dev site.
·
Thursday, 22 May 2014 23:54
·
0 Likes
·
0 Votes
·
0 Comments
·
I will try to analyze the data on your live site once I have more spare time but please leave the access open
·
Friday, 23 May 2014 00:09
·
0 Likes
·
0 Votes
·
0 Comments
·
Hosted on a dedicated server with 24gb ram, i7 processor, SATA drive. Using nginx 1.6 and PHP5.3 with APC, mySQL 5.5, Linux.

Accounts: 47,000

Dashboard 6.17s (includes easyblog latest blogs and easydiscuss latest discussions modules)
My Profile 2.31s
Groups 1.15s
Photos 2.86s
·
Friday, 23 May 2014 00:32
·
0 Likes
·
0 Votes
·
0 Comments
·
Hi Mark,

My dev site was an exact copy of live site. there were no differencies between them. I have 2 strong competitives, their sites load 800ms-1200ms, i need at least 1 - 1.2 second max loading time.
·
Friday, 23 May 2014 01:31
·
0 Likes
·
0 Votes
·
0 Comments
·
Guys,

People dont like slow sites, it doesnt matter how qulified or which properties has. The first think is speed. We are wasting our money and time, so we are waiting for speed improvement immediately. If someone complain "we" word i can turn it to "I". And in 2014, if you have a site with more then 1 sec loading time your site SLOW! if you have a site with 3-7 sec loading time, its not a site actually its a turtle! VIRTUAL TURTLE. And we have turtles currently!

Anyway, I want to see an Express Solution, getting angry anymore, because i changed 3 dedicated servers in a month! I wasted nearly 1 year (will be 10th June, 2014) and tons of money, and i continue... After per version ES getting slow! instead of getting speed. I wont continue with this speed, because I am to be cracked. If you wont do anything about speed immediately, let me know to find another open source system.
·
Friday, 23 May 2014 06:59
·
0 Likes
·
0 Votes
·
0 Comments
·
Our redesign launches this Sunday (5/25) with 3600 active users and usually active for 20 minutes.

Can let you test after the launch, just message me after June 3rd.

MK
·
Friday, 23 May 2014 09:12
·
0 Likes
·
0 Votes
·
0 Comments
·
Mark, I have a site with 60k users, with peaks of 3000 online. I am migrating to Easysocial within 30 days, I hope this making the right choice.
·
Friday, 23 May 2014 09:24
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello gokhan,

Your dev site takes about 1.6s to load vs your live site which is 3s. So I am pretty sure that your live site data has something that your dev site doesn't?
·
Friday, 23 May 2014 11:15
·
0 Likes
·
0 Votes
·
0 Comments
·
Cristiano wrote:

Mark, I have a site with 60k users, with peaks of 3000 online. I am migrating to Easysocial within 30 days, I hope this making the right choice.

Yep, this would be good!
·
Friday, 23 May 2014 11:26
·
0 Likes
·
0 Votes
·
0 Comments
·
MD King wrote:

Our redesign launches this Sunday (5/25) with 3600 active users and usually active for 20 minutes.

Can let you test after the launch, just message me after June 3rd.

MK

Excellent! Would love to run some stress / performance tests on your site
·
Friday, 23 May 2014 11:26
·
0 Likes
·
0 Votes
·
0 Comments
·
Yesterday night, i copied my live site. So dev site an exact copy of my live site, nothing different!
·
Friday, 23 May 2014 17:58
·
0 Likes
·
0 Votes
·
0 Comments
·
I think, the speed problem related to easysocial sql query structure. I dont have a table bigger then 1mb. (easysocial sql tables). Why easysocial executes some litle sql query in nearly 2 sec? So Guys what are you doing about speed ? Let us to know
·
Friday, 23 May 2014 21:13
·
0 Likes
·
0 Votes
·
0 Comments
·
This happens with all scripts, the activity feed and notifications feed are responsible for the largest number of sql queries. If there is a way to increase the time between these queries, might lessen the load on the server
·
Friday, 23 May 2014 21:25
·
0 Likes
·
0 Votes
·
0 Comments
·
@Cristiano

did you check my posts above, you will see that the problem is not about server side
·
Friday, 23 May 2014 21:32
·
0 Likes
·
0 Votes
·
0 Comments
·
Hey guys,

Just to update you, we're still noticing some slight "perfomance issues" and we're working on this. It's not as simple as it seems but we're going through most of the extensive and heavy queries as well as delegating heavy queries to be used only when necessary.
·
Monday, 26 May 2014 11:11
·
0 Likes
·
0 Votes
·
0 Comments
·
Mark,

What is the latest on this performance tweaking?

Cheers,

Andrew
·
Thursday, 17 July 2014 22:22
·
0 Likes
·
0 Votes
·
0 Comments
·
Hello Andrew,

I am not really sure what do you mean by "latest" here?
·
Friday, 18 July 2014 01:23
·
0 Likes
·
0 Votes
·
0 Comments
·
well, your last message said:

"Just to update you, we're still noticing some slight "perfomance issues" and we're working on this. It's not as simple as it seems but we're going through most of the extensive and heavy queries as well as delegating heavy queries to be used only when necessary."

thanks
·
Friday, 18 July 2014 05:26
·
0 Likes
·
0 Votes
·
0 Comments
·
@gökhan
based upon what you wrote below you could increase your buffer pool size significantly e.g. 5x or more than what you have there. I don't see what version of mysql you are using but that can make a difference for innodb settings. Also you might consider using mariaDB in place of mysql. mariaDB is like Joomla in the sense that the original creator and much of the lead left mysql to create mariaDB (like Mambo>Joomla). I have not done it yet but I plan to replace all of my mysql with mariadb and pretty much everyone agrees that it performs better. Percona is another alternative: http://www.percona.com/software/percona-server


gökhan wrote:
Here last result with new server 32GB Ram...
Mysql and server status attached, Here my.cnf also;

################

[mysqld]
default-storage-engine=innoDB
innodb_file_per_table=1
innodb_open_files=1024
open_files_limit = 150000
key_buffer = 1024M
join_buffer_size = 16M
sort_buffer_size = 8M
read_buffer_size = 6M
read_rnd_buffer_size = 8M
myisam_sort_buffer_size = 128M
tmp_table_size = 256M
max_heap_table_size = 128M
query_cache_size = 4096M
query_cache_limit = 1024M
query_cache_type = 1
table_open_cache = 195535
table_definition_cache = 85535
thread_cache_size = 512
innodb_buffer_pool_size = 4096M
wait_timeout=60
max_connections = 4096
max_connect_errors = 1000
connect_timeout = 10
net_write_timeout = 900
net_read_timeout = 900
max_allowed_packet = 128M
max_delayed_threads = 4
local-infile=0
innodb_log_buffer_size=64M

##################################
·
Friday, 18 July 2014 07:51
·
0 Likes
·
0 Votes
·
0 Comments
·
Andrew, we have already optimized the streams on the latest release of EasySocial.
·
Friday, 18 July 2014 10:34
·
0 Likes
·
0 Votes
·
0 Comments
·
View Full Post