Hello,

Deleting the records in #__social_points_history will affect user's points, user's points history details, user's leaderboard ranking and any other places that referring to the data from this table.

Removing records from database table will definitely speed up the query performance however, a big size table doesn't mean its always cause performance issue. If the table has added proper index keys, these should help to speed up the query performance.
Most of EasySocial's tables has added the table index accordingly to the SQL query to speed up the performance.

In my opinion, the best way to check if this table giving any performance issue is by checking the mysql slow logs.

Hope this help.