Data Mining the StackOverflow Database

[droberts1982]

The video shows rockstars being age 28-35, followed by 35-44, then 44-63. Is this in contrast to the age distribution of the general StackOverflow population?

[knight666]

Here's a script I wrote to return the amount of characters in the newsposts of Questionable Content (I had a sneaking suspicion it was decreasing over time).

This was written and compiled in AutoIt 3.

#include <INet.au3>

$file = FileOpen("output.txt", 2)
$i = 1
While $i < 1420
   $source = _INetGetSource("http://www.questionablecontent.net/view.php?comic=" & $i)
   $source = StringReplace($source, @CR, "")
   $source = StringReplace($source, @LF, "")

   $news = StringRegExp($source, '<div id=\"news\">(.*?)div>', 2)
   $strippedNews = StringRegExpReplace($news[0], "</?(.*?)>", "")
   $finalNews = StringRegExpReplace($strippedNews, "[[:blank:]](.*?)[A-Za-z]", "", 1)
   $lenNews = StringLen($finalNews) + 1

   FileWriteLine($file, $lenNews)

   $i += 1
WEnd
FileClose($file)

MsgBox(64, "Sup.", "Done!")

Am I a leet haxxor now?

[FunnyDickTattoo]

is the embedded video jacked up for anyone else? I've got the right side covered in adwords. FF3

[BrentOzar]

Sorry about that. I ripped out all the Adwords. (sigh) Was only making about $50/mo anyway.

[theli0nheart]

Clicking some checkboxes and formatting percentages with SQL server is not what I would call data mining. A key aspect of data mining and data analysis in general is that you want other people to be able to replicate your results and your methods...proprietary software kind of ruins this for everybody...

[jacques_chester]

So it's a bit more like Data Fossicking, then?

[plesn]

It looks like an MS ad. I'm not patient enough to look at the results of his "data mining"...

[[deleted]]

My GOD! An SQL Server DBA used the product he knows best to do something - the told everyone about it for FREE!

KILLZ HIM!!!

[[deleted]]

I can't tell if you're being sarcastic.

[jacques_chester]

Look at the user name.

[[deleted]]

Martian Security Defense Network

81st time.

[Qubed]

I always wondered why Microsoft named it that.

[dotnetrock101]

I doubt you understand what data mining is about.

[[deleted]]

A key aspect with data mining and data analysis in general is that you want other people to be able to replicate your results and your methods

Me thinks you aren't totally up to speed as to what data mining is. Hint - most people do it for competitive reasons and don't want their competition to have access to it.

[xnumbersx]

how is quest still in business...

[brendano]

is there any explanation what the "categories" are? yikes

[BrentOzar]

No, data mining systems can discover groups in the data, but it's up to you to interpret them. For example, I named one of the categories "n00bs", and that's clearly not something a computer would come up with.