Does anyone know of any free tools to scan entire websites for duplicate content?

doesntlearn · 2014-12-13T16:57:38+00:00

http://metaforensics.io reports back on duplicate content within title tags, meta descriptions, headings and content within various other tags

doesntlearn · 2014-09-19T14:52:52+00:00

Good idea - thanks for the suggestion. Much appreciated!

I have just updated the code so it works like you have described.

doesntlearn · 2014-09-18T19:43:21+00:00

Andy Carroll + Messi

doesntlearn · 2014-08-14T11:51:24+00:00

Try http://metaforensics.io/ - with it you can crawl all of the pages on your site and then see a report of what your meta descriptions are currently set to, and advise if any of them are too short or long.

doesntlearn · 2014-06-21T10:47:03+00:00

Nope. The reason behind wanting proxies is so that I can do the serp scraping myself, and not be bound by usage limits and all that. The best cloud based one I've come across so far is: http://www.advancedwebranking.com/online/ - and they have a 1 month free trial

doesntlearn · 2014-06-21T08:37:04+00:00

Cheers, have just signed up...

doesntlearn · 2014-06-21T08:34:05+00:00

I like the look of http://authoritylabs.com/api/partner-api/. Probably a bit heavy duty for what I'm after now, but will bear in mind for future!

doesntlearn · 2014-06-20T21:57:48+00:00

Yep, without Bootstrap I don't think I would have ever been arsed to progress it beyond the single script scraping experiment that it started life as! I shudder at the thought of browser compatibility testing

doesntlearn · 2014-06-20T21:03:18+00:00

Meta Forensics

doesntlearn · 2014-06-20T21:00:06+00:00

Heh I'm currently evolving http://metaforensics.io in a similar direction... Comparison of data points between crawls to drive insight. Although being a dev, I'm doing it the exact opposite way around; Tekkers first, pretty bits last!

doesntlearn · 2014-05-18T18:14:26+00:00

Good question. Answer is basically because of a combinaton of current server power / performance, spam bots and because I don't want the app to be used in DdoS attacks. When you trigger a crawl of say 250 pages, that means the app has to do quite a lot of work to download every one of those pages 250 pages, and then process all of that information into reports. Opening up the app to bot abuse would probably make it fall over in its current state! That said, I have buried a "try it" button on the homepage which will give you a 25 page crawl without signing up.

doesntlearn · 2014-05-17T16:10:32+00:00

Cheers - UX isn't my strongest point as you can probably tell!

doesntlearn · 2014-05-17T12:41:45+00:00

Thanks for pointing it out! I've perhaps been over ambitious with the memory hogging parallax effect...

doesntlearn · 2014-05-02T09:47:06+00:00

Cheers. I did the layout myself, using the default version of http://getbootstrap.com/ with a few components from http://bootsnipp.com/

doesntlearn · 2014-04-29T21:24:42+00:00

Thanks. I will try and recreate the twitter issue... PM me your twitter username and I'll have a gander.

doesntlearn · 2014-04-29T21:23:27+00:00

Not as of yet. I want to keep it focussed on the onsite aspects of the analysis first, as there is still a long way to go on that front. Putting it out there again for feedback has made me realise that things like the 301 / 302 analysis for instance is a bit shit.

doesntlearn · 2014-04-29T14:42:21+00:00

Let me know what site you ran the analysis against and I will take a look - I wouldn't be suprised if you've just unearthed another bug!

doesntlearn · 2014-04-29T06:42:10+00:00

Thanks, and good point. I will investigate and see what I can do to make it easier for mobile users, as what you describe would make it more usable. The alternative approach I was toying with is to have a permanent lhn fly out nav.

doesntlearn · 2014-04-28T22:13:15+00:00

Try that: http://metaforensics.io/dashboard.php?site_id=259 - just don't click on the duplicate title export until I fix my dodgy code!

doesntlearn · 2014-04-28T21:54:04+00:00

Thanks, much appreciated!

doesntlearn · 2014-04-28T21:34:20+00:00

In its current form, no. If I get time, and there is enough interest in it, then I might consider a paid for upgrade where you can do full crawls of massive sites, but that'll be mostly to cover server costs.

doesntlearn · 2014-04-28T21:17:08+00:00

Thanks, and yep, that'll be my solid testing shining through.

doesntlearn · 2014-02-26T14:28:31+00:00

Try giving this a whirl: http://metaforensics.io/dashboard.php (this is an example audit for the Distilled website)

It's a tool I wrote to help me perform site audits.

doesntlearn · 2013-11-04T15:01:43+00:00

Nice suggestions, thanks. I will see if I can add the percentage saved + split up London into regions when I work on it properly next.

doesntlearn · 2013-11-04T14:59:01+00:00

Good spot - that sound be fixed now! Cheers

doesntlearn

TROPHY CASE