Hello Humza,

Thanks, this will be updated shortly as we are still discussing internally over what features we plan to add and the future direction of our plans.

One thing I feel that would be awesome to add is the option for users to choose if they want specific items indexed on google. For xample, photos, profile or blog posts. I think that's something key nowadays considering security is a big factor man.

It's best to restrict url's via the robots.txt file. This file is in the root directory of Joomla.

Here's a tutorial on using it: http://www.robotstxt.org/robotstxt.html

Yeh but that's a pain in the ass :P it's easier if it is done via the GUI. Some people prefer to have their pictures public and some don't

Can it be done without the robots.txt file? This is a search engine we are working with rather than a in-site feature.

I am sure it's possible. I mean Mark might know.

Hello Humza,

Josh is right because spiders would normally crawl for robots.txt but in addition to that, you can still programmatically restrict spiders from accessing contents but here's the thing, there's so many different crawlers / spiders around that you really can't stop them from crawling. There's no real way to identify these crawlers / bots.

The best method (is also a standard) to have a robots.txt hoping that most search engines would crawl the same file

Hmmm. So then how do we hide profiles based on the privacy status.

Hello Humza,

If their privacy is restricted, they will still appear on search engines but their content is not visible so depending on the search engine whether or not they want to crawl the page

Hmmm interesting. But the question I wanted to know is then how do you make it so that the robots.txt file doesn't index certain people? :S

You need to manually do it Humza

Are those specific people visible to non logged in users? If not, I would think they are pretty secure from search engines. To test, be sure to copy a url of one of the profiles and paste it into Google. A result might show up, but it might not have any real content for it.

Thanks for sharing Josh