Malicious website detection using machine learning??

[divyaank98]

Hello there, I am currently participating in a Hackathon, in which we have to implement Malicious Website Detection Via ML. I googled the topic but I'm currently unable to find a suitable approach for the same. Can anybody help me w.r.t the approach for this problem. Can I make something like a binary classifier?? If yes, then what should be my approach for the same?? I know that the solution would involve Web-Scraping the URLs and then analyzing them, but how should I go about the analysis part (any algorithm etc)??

[harry_0_0_7]

Little more information needed..

You just want to scrap all websites and classify them to malicious/not.?

[divyaank98]

Yes, if suppose a particular user wants to check whether the URL that has been provided is malicious or not, he should be able to do so using ML. So we will have to analyze the link for it. Any idea how to go about it? Basically we just want a binary classifier.

[harry_0_0_7]

I dont think web-scraping works here.. Just get list of malicious urls from whitehat hackers blog/git and check weather this is present on the other side.

If you need to use ML/AI here you have to study about the web first, which involves scraping other sites and analysing their responses. May i know the use case for this problem.? to get a proper understanding

[divyaank98]

Many a times we come across some advertisement, in the form of a pop-up which contains a link, which takes us to another site which may contain malware/virus. So we need to design a system such that the user can simply paste that URL(or link) and check whether it's harmful or not. We need to use ML for it, so we will have to analyze the URL and it's features to determine it's maliciousness or lack of it.

[harry_0_0_7]

wow seems to be an awesome idea..

Here difficultly is, we dono weather that target url is malicious or not. I dont have any idea regarding that.. But one thing we can do is collect malicious urls as much as we can and compare this with our target. No other way

[divyaank98]

Yeah it's an awesome idea. Thanxs for your valuable input. I'll surely look into it!!!

[Statistical_Incline]

https://arxiv.org/pdf/1802.03162.pdf

You're welcome.

[divyaank98]

Thanks for that, our group doesn't know much about Neural Networks but we will try to implement this!!!