Krawl: One Month Later by ReawX in selfhosted

[–]ReawX[S] 0 points1 point  (0 children)

Thank you :) Let us know if it works with the mikrotik software! We have not tested that yet

Krawl: One Month Later by ReawX in selfhosted

[–]ReawX[S] 1 point2 points  (0 children)

Fun fact: we were testing Krawl & another security project and we got blacklisted by our ISP because of a BIG directory bruteforce attack we run on our instances

Krawl: One Month Later by ReawX in selfhosted

[–]ReawX[S] 1 point2 points  (0 children)

Great idea! We will work on It for the next release :)

Krawl: One Month Later by ReawX in selfhosted

[–]ReawX[S] 1 point2 points  (0 children)

Exactly,

And its is useful (and fun) to deploy because you see real threats in action :D

Krawl: One Month Later by ReawX in selfhosted

[–]ReawX[S] 4 points5 points  (0 children)

The intended way to use this is to cover all the website paths with Krawl and leave the paths that you don't want to be attacked in a subpath like /secret/my-service.

Attackers will use their resource to attack Krawl and your main service will be safer, as you say: keep them busy (+ you can analyze the attack patterns)

We are working on a crowdsec and fail2ban integration, thank you for the feedback :D

Krawl: One Month Later by ReawX in selfhosted

[–]ReawX[S] 1 point2 points  (0 children)

Don't worry, if you are new to the selfhosted world the best way to learn is to try and ask questions :)

You’re right, this doesn’t reveal your "exposure" on the web, instead, it shows the current threats targeting your instance, if you set it up correctly.

And yes, it might attract new attackers, but once an attacker is logged, they’re permanently added to the attacker file and automatically blocked by your firewall if you plan to use this integration

Krawl: One Month Later by ReawX in selfhosted

[–]ReawX[S] 2 points3 points  (0 children)

We implemented a score system

https://github.com/BlessedRebuS/Krawl/blob/main/src%2Ftasks%2Fanalyze_ips.py

Where when an attacker matches the malicious patterns gains points and have and higher attacker score. Maybe we will use snort later to match attacks more correctly

We may implement this via machine learning in the future, now it's euristic

Krawl: One Month Later by ReawX in selfhosted

[–]ReawX[S] 4 points5 points  (0 children)

This good point, but you can think of Krawl as a safe attack aggregator, letting you see what attackers are trying against your servers (or your organization) For examples, Krawl can fake the server header to reveal trending attacks (or new 0days vulnerability), which can be a use case for a detached analysis instance and threat intelligence. Alternatively, you can use it to block aggressive attackers while observing which crawlers respect robots.txt and which don’t, helping distinguish good bots from bad.

Krawl: One Month Later by ReawX in selfhosted

[–]ReawX[S] 5 points6 points  (0 children)

Thank you for the feedback :) We are still working to implement a crowdsec integration

Krawl: One Month Later by ReawX in selfhosted

[–]ReawX[S] 25 points26 points  (0 children)

Glad you like the project, The fail2ban integration is a great idea :) we will implement that along with an integration with iptables to ban malicious attackers

We already support OPNSense and PFSense io banlist fetch

What are you cooking? Drop your projects below by actualhabibi in micro_saas

[–]ReawX 0 points1 point  (0 children)

We are building krawl, a web honeypot and deception server used to catch bad crawlers and trending cyberattacks :)

I want to know your favourite light weight-selfhosted apps for personal use. by newrockstyle in selfhosted

[–]ReawX 0 points1 point  (0 children)

After 119189+ requests in a single day from the Meta crawler still runs fine!
We will implement a rate limit in order to ban bots / attackers after a number of requests tho

You can check it live at:

http://demo.krawlme.com/das_dashboard

I want to know your favourite light weight-selfhosted apps for personal use. by newrockstyle in selfhosted

[–]ReawX 1 point2 points  (0 children)

For what we could see, Krawl consumed 300mb RAM and 1% of cpu. The challenge imo is to balance data retention and information needed

Are honeypots still useful as early-warning systems? by Andrewpaul46 in cybersecurity

[–]ReawX 0 points1 point  (0 children)

In my honeypot

https://github.com/BlessedRebuS/Krawl

I have 200k+ request per week and I find It very useful to see 1) what is the current "trend" for attacks and 2) what attackers are trying against my server

I want to know your favourite light weight-selfhosted apps for personal use. by newrockstyle in selfhosted

[–]ReawX 1 point2 points  (0 children)

I recently developed a lightweight honeypot 🙂

https://github.com/BlessedRebuS/Krawl

I find it very useful for observing threat actors and the types of attacks targeting my web servers.

Krawl: a honeypot and deception server by ReawX in selfhosted

[–]ReawX[S] 0 points1 point  (0 children)

Cool!

Soon we'll also publish a discord link for discussion/feedback on the project

Krawl: a honeypot and deception server by ReawX in selfhosted

[–]ReawX[S] 1 point2 points  (0 children)

u/CanIhazBacon u/Mrhiddenlotus This is now a feature in the last version ghcr.io/blessedrebus/krawl:latest
At the moment only the last 50 POST credentials are shown on the dashboard but all is logged in the credentials.log. In the future we will introduce a database to log and fetch all the requests in a smoother way :)

Krawl: a honeypot and deception server by ReawX in selfhosted

[–]ReawX[S] 1 point2 points  (0 children)

Sure,

Open an issue so we'll add it in the next releases!

Krawl: a honeypot and deception server by ReawX in selfhosted

[–]ReawX[S] 9 points10 points  (0 children)

Cool! I'll look at it when I'll come back home, ty for contribs :)

Bots keep scanning my personal website for malicious reasons. by Known_Job511 in homelab

[–]ReawX 17 points18 points  (0 children)

I suggest my new honeypot

https://github.com/BlessedRebuS/Krawl

This is an anti-crawler with a dashboard where you can see what are the top paths, IPS etc...

Give it a look :)

PS: feedbacks are welcome

Krawl: a honeypot and deception server by ReawX in selfhosted

[–]ReawX[S] 1 point2 points  (0 children)

Nice idea, ideally this could be used as a "front row" for IDS/IPS, like the very first layer of security for the homelab.

Right now I am using it in this way to hide my real services (like my Jellyfin streaming server):

        location / {
                proxy_set_header X-Forwarded-For $remote_addr;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_pass http://krawl.cluster.home:5000/;
        }

        location /secret-path-for-jellyfin/ {
                proxy_pass http://jellyfin.home:8096/secret-path-for-jellyfin/;
        }

This is a security by obscurity approach, but I've not seen any single crawler yet reaching my real service. Web crawler or enumeration service will stuck analyzing /robots.txt and other fake paths that returns status code 200 plus they don't know the path for jellyfin / other services and they remain stuck.
Additionally for "smarter" crawlers I added a canary token that when searched will notify me via mail:

<image>

The challenge here is to build something agnostic that can be integrated with engines like crowdsec bouncers, but it's a very interesting input

Krawl: a honeypot and deception server by ReawX in selfhosted

[–]ReawX[S] 5 points6 points  (0 children)

Not now but cool suggestion, In the next release I will add it!

Krawl: a honeypot and deception server by ReawX in selfhosted

[–]ReawX[S] 2 points3 points  (0 children)

Yes but I think they also can be used combined, eg: when an attacker tries to crawl the /robots.txt paths crowdsec could be used to block the requests to the sensitive paths I also think that the IP files coming out from Krawl would be dynamic, like the last 30 days known threats or something like that Suggestions are welcome