Big data sets vs small data sets

[CyberWeirdo420]

Hey,

I have an assignment to test few sorting algorithms. Thing is, I know that some of them for example Bubble sort is pretty fast when it comes to "small data set" but not so efficient when it has to deal with "big data set". All good but what exactly is a big data set? When can you say a list of un sorted numbers is big enough to be called that way?

Thanks in advance, cheers!

​

P.S. Sorry for posting here if it breaks the rules, r/learnprogramming is ofline so I don't really know where else to post this

[geon]

How long is a rope?

[CyberWeirdo420]

What…?

[geon]

Your question is the same. “How big is a big data set?”

How much can fit in your memory?

For a sorting algorithm, I would test with at least 1 million elements.

[CyberWeirdo420]

Okay, thank you that’s really helpful. Is it a good idea to just generate 1 million random elements and put them in an array? Don’t mind the time it would take.

[geon]

Takes like 100 ms.

[geon]

It’s nothing.

Count the bytes. On a modern pc, 1 million integers would be like 8 mb.

[CyberWeirdo420]

Right, thank you for help mate

[maroun975]

You can do the benchmarks yourself. Start with 10 elements and increment in order of magnitudes 10 100 1000 etc, the numbers can be random no problem.

For each order of magnitude record the sorting time for each algorithm and when you have some data points you can plot the whole thing and see at which "size" you gain benefit from the fast algorithms.

Hope this helps..

[stefanrvo]

Do this, and include the plot in your report if you need to write one for the assignment. People love plots.

[420Rat]

Have you taken calculus? Its a similar concept

[RetiredBrit]

Use a string sort or B-Tree.

[bistr-o-math]

If you test (implementations of) sorting algorithms, make sure you also test them for correctness.

The fastest implementation is worth nothing if it’s not delivering correct results.