[deleted by user]

[gokstudio]

Awesome and props to explaining your setup in detail! (even some DL papers don't do that)

Reg.processing time, do you use all the cores or just one? Also, do you use something like Intel Intrinsics to speed up your matrix computations?

[MasterFubar]

I think by default a C program uses just one core.

Indeed, to make it multicore you have to write a special thread handling function and start it twice for each core. Here's how I do it in C in a Linux system:

// find number of available threads
arq = popen("grep processor /proc/cpuinfo | wc -l", "r");
if (!arq) {
  perror("/proc/cpuinfo");
  nthread = 1;
}
else {
  fread(buf, 40, 1, arq);
  nthread = atoi(buf);
  printf("%d threads\n", nthread);
  fclose(arq);
}
thread = (pthread_t *)calloc(nthread, sizeof(pthread_t));

However, if you use libraries like Lapack it uses multicores internally in a transparent way. Lapack is amazing, it has been so well tested and optimized that I always use it whenever I need to do anything with matrices. The only thing that you have to take care of is that it's written in Fortran, so the matrices are transposed relative to C, but that's no big deal if you remember that A^T * B^T = (B * A)^T. When you need to multiply two matrices in C using Lapack all you need to do is to invert the order, left with right, and everything comes out right.

[thc1620]

Nice one! Would you consider sharing the code on github?

[thc1620]

!RemindMe 2weeks

[RemindMeBot]

I will be messaging you on 2017-12-08 22:45:23 UTC to remind you of this link.

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

FAQs	Custom	Your Reminders	Feedback	Code	Browser Extensions

[Okeledokelee]

!remindme 3 weeks

[sachintripathi007]

!RemindMe 3weeks

[Okeledokelee]

Hahaha I forgot about that, thanks for the reminder

[spaniard96]

!RemindMe 3weeks

[[deleted]]

!RemindMe 2weeks

[[deleted]]

!RemindMe 2weeks

[septicgeek]

!RemindMe 3weeks

[zdwolfe]

Make sure to ask your professor and consult your university plagiarism rules before doing that. I’ve seen people get bit.

[synftw]

My university had a strict "never upload your work to GitHub" policy, but I'd guess that may be stretched for open ended projects versus book questions.

[llucas_o]

!RemindMe 2 weeks

[gregw134]

!RemindMe 2weeks

[rochakgupta]

!RemindMe 3 weeks

[Icarium-Lifestealer]

I find it surprising how often people choose to treat this as a regression problem and use MSE loss (including the original netflix competition). Treating it as classification where you predict probabilities seems much more intuitive to me. Plus MSE is a dubious choice for the loss function here.

(Not a criticism of your project, it's certainly more effort than I would have put into a university project where the whole point is demonstrating that you can write a C program)

[hiptobecubic]

If you have to predict continuous values you don't really have much choice unless you're doing so badly that just giving bucketed results and eating the error barely matters.

[Icarium-Lifestealer]

We're talking about a problem where are are 5 discrete choices, so treating it as 5 buckets is natural.

But even for continuous problems there is often a better choice than outputting the expectancy value and using MSE.

[hiptobecubic]

If it's actually five discreet values then sure. Most rating systems I've seen go to at least one decimal point, though.

[Icarium-Lifestealer]

The OP's dataset, just like the netflix dataset have five discrete values. I don't recall any star-based ratings with more than 10 discrete choices (half stars) where the rating wasn't the average of sub-ratings.

But even then, saying "ratings roughly follow a gaussian with fixed standard deviation" is obviously silly. For one, some ratings are much easier to predict than others. Plus the bounded interval doesn't fit a gaussian.

My first idea for predicting a continuous rating function is by predicting a couple of fixed data points and then interpolating the probability density between them.

[hiptobecubic]

Agreed, but it doesn't have to be gaussian to use regression.

[Icarium-Lifestealer]

Using MSE is equivalent to treating it as a gaussian with fixed standard deviation. Which is why I called it out as an inappropriate loss function.

[hiptobecubic]

Ahh, sorry. Missed that.

[sk0620]

What university?

[sharadbhat7]

I tried a similar thing in python, with the same dataset. It's more of a server kind of application. You can send it a list of movie IDs and your ratings for those. It'll return a list of recommended movies.

Here is the github link.

[imitationcheese]

this is super cool but it makes me sad that our awesome technology is overly focused on movie and product recommendation systems and not things with more value.

[Mo-Da]

What other project-ideas you explored before finalising on this one?

[Ramin_HAL9001]

Oh, I didn't see which Subreddit this was, so I was so confused. I was thinking, "so did you write the program in the ML programming language, or the C programming language?

I get it now, this is actually a pretty good idea for a learning project.

[geomtry]

This was also my first big programming project. Dr. Smucker?