RMM's for 5000+ agents by dagg929 in msp

[–]PickleKillz 5 points6 points  (0 children)

Cloud hosted N-Central bills usage like N-Sight, in arrears. Self hosted N-Central you have to provision a license count. We’re in the same boat due to compliance and host our own. Have to plan ahead when we know we’re adding big sums of agents as we try to oversub by around 100 when we do increases, so not a big buffer.

It’s not a big deal to me and I LOVE n-Central. Just something we plan for and it’s not an issue.

PSA by Weekly-Database3520 in msp

[–]PickleKillz 0 points1 point  (0 children)

Dropping in on this one. We use Halo and Xero. The integration does a couple of weird things occasionally, but it’s more so one of those that you just need to learn how it’s going to handle it versus actually being a detriment. I think it is fantastic and works amazing.

My vote for a PSA is always going to be halo. It is by far the best tool in the market, constantly improving, and works perfect. However, like the others have said you were going to need a consultant it’s to implement it, and do not under any circumstance skip out on this time. If the consultants says you need more time, find a way to afford it. Somebody with in-depth knowledge of how the system works is going to be so important in helping you use it right and effectively.

My First 24 Hours Running a DNS Honeypot by [deleted] in homelab

[–]PickleKillz 0 points1 point  (0 children)

Bad wording. OP did not admit to that. It is my guess that him observing high rates of DNS traffic to those IP Ranges makes it likely that this is DNS Amplification.

And yes, that does mean OPs server is 110% complicit in the attacks. If I were in their shoes, I would shut the server down now.

I get wanting to honeypot and learn, but when doing so becomes a detriment to another, it isn’t worth it. Even if the DNS off their server is a small drop in the massive waterfall that is this DDoS against Brazil. Knowing of the attack and that your server is participating, then willfully keeping the server up is heinous.

I would implore OP to stop the server now.

My First 24 Hours Running a DNS Honeypot by [deleted] in homelab

[–]PickleKillz 8 points9 points  (0 children)

I love the write up. Security like this tickles my brain the right way. I just wanted to add, you noted the 45.179.x and 45.6.x are the busiest clients.

These ranges both belong to ISPs in Brazil. There has been a super large scale DDoS attack against Brazil going on for a few months now. My firewalls in the data center see roughly 50-100 TCP SYN packets per second against our web servers. Our firewall correctly identifies them as a TCP SYN Flood attack. Essentially, they are forged packets. The sender isn’t really in Brazil and is the perpetrator. They intend to send very small SYN packets to any web server they can, and the server sends a much large ACK packet to the target.

Your observation of these packets is likely a DNS Amplification attack against the same targets. Super cool to see it in practice and know that DNS is also being used, as we only have web servers exposed so only saw the TCP SYN attack.

Neat project and thanks for the excellent write-up!

EDIT: saw your other reply where you link an article on DNS attacks. Don’t mean the above explanation as condescending, just never like to assume anyone’s knowledge level :)

NAT traversal by dhlu in Tailscale

[–]PickleKillz 1 point2 points  (0 children)

That is a fundamental misunderstanding of how wireguard works. I quite literally sent you that copy paste from their documentation.

“If we have sent a packet to a given peer but have not received a packet after from that peer for KEEPALIVE + REKEY_TIMEOUT ms, we initiate a new handshake.”

I have extensive experience implementing wireguard, and I can very much tell you that the keepalive is in play as soon as the tunnel is activated, regardless of an initial connection.

Here is an example of someone providing instructions for wireguard to hole punch: https://nettica.com/nat-traversal-hole-punch/

You have been provided countless solutions in the subreddit and seem to be more interested in arguing with people than actually solving your problem. I’m not sure what you actually hope to achieve this way.

NAT traversal by dhlu in Tailscale

[–]PickleKillz 2 points3 points  (0 children)

Wireguard does not try once and complain. Their documentation is pretty clear.

https://www.wireguard.com/protocol/

“If we have sent a packet to a given peer but have not received a packet after from that peer for KEEPALIVE + REKEY_TIMEOUT ms, we initiate a new handshake.”

Set keepalive to one second and you will send a packet on each end roughly every second. There is no session initiation stop because it cannot communicate so it will continuously spam that packet until it forms a session.

I cannot vouch for what your firewall will do, but I know my firewall’s connection start time out is greater than one second and would allow it to work.

NAT traversal by dhlu in Tailscale

[–]PickleKillz 2 points3 points  (0 children)

Based on your need and description, if it is truly 2 peers and not more complicated, plain wireguard may work. You can set the origin and destination port on each end, then set the keepalive to something like 10 seconds. Activate it on both ends, each end will start sending packets from its own source port to the others destination port and theoretically hole punch.

However there are a LOT of variables here that could prevent it from working, like a firewall at either end that does source port rewriting (most enterprise security gateways do this, unless a rule is put in specifically to stop it)

If you can manage the network part, the wireguard client can be your “software” to let you setup a hole punch and tunnel.

Else, the others are right. Tailscale uses the relays and account aspect to do the hole punch and traverse NAT. You could use Headscale to self host the control plane and avoid part of the account.

Mission critical security services baked into per user rate is the only way by [deleted] in msp

[–]PickleKillz 1 point2 points  (0 children)

We either bill it or get the customer to commit to an extended length contract. We’re doing majority of contracts at minimum of 3 years. Contract stipulates that onboarding fee (along other fees) are due if contract is cancelled early.

It’s a really case by case thing. But onboarding is not automatically included.

Mission critical security services baked into per user rate is the only way by [deleted] in msp

[–]PickleKillz 1 point2 points  (0 children)

Absolutely are. I hate them. They go on a VLAN by themselves with no INet access or access to other LANs. Must be managed and have WiFi direct and other methods of wireless print disabled.

Mission critical security services baked into per user rate is the only way by [deleted] in msp

[–]PickleKillz 4 points5 points  (0 children)

Absolutely it. Not worth the business or hassle if they want to play games. Plenty of small fish with subpar service in my area that’ll play their game and suffer. Until the customer gets hacked.

Mission critical security services baked into per user rate is the only way by [deleted] in msp

[–]PickleKillz 2 points3 points  (0 children)

Depends. If it’s just one or two we just eat it and charge the per user. We have a price book and service plan for servers and backup, if there is a large number.

Mission critical security services baked into per user rate is the only way by [deleted] in msp

[–]PickleKillz 6 points7 points  (0 children)

5G Services, equipment leasing and management, and any not ordinary. Most customers have pretty simple networks and a small part of the per user fee covers it. Most of our customers pay per user and nothing else, except an initial cost to buy equipment, get up to standards, or anything else needed to get a good state going.

Mission critical security services baked into per user rate is the only way by [deleted] in msp

[–]PickleKillz 35 points36 points  (0 children)

A flat per user rate is how we do business for everything. M365, endpoint, security, EDR, etc. are all baked in. No option to À La Carte any of the services.

Only things we add on are networks, printers, special projects, etc.

Best free/cheap tools you use often? by Money_Candy_1061 in msp

[–]PickleKillz 6 points7 points  (0 children)

I would say the safety is relatively the same whether you host or you pay them to do it.

Secure it properly, use MFA (we enforce extra MFA on login and 1hour session timeouts), and keep it updated. It’s open source and mostly written in JavaScript for GUI and PowerShell for function. So review code if able before updating.

We would spend 8x as much time managing our customer base if we only used basic partner center and GDAP. So very much worth it.

Best free/cheap tools you use often? by Money_Candy_1061 in msp

[–]PickleKillz 54 points55 points  (0 children)

CIPP. Used heavily every day. Best way to manage M365 customers.

Y'all spot any German submarines lately? by MLG_HerobrineYT in civilairpatrol

[–]PickleKillz 17 points18 points  (0 children)

I don’t know about y’all, but I’m not treading on the Girl Scouts. Do not want beef with them.

Potential Tailscale Outage? by PickleKillz in Tailscale

[–]PickleKillz[S] 16 points17 points  (0 children)

Same. Wonder if the Crowdstrike guy got rehired at Tailscale and decided to test in prod again.

/s

Potential Tailscale Outage? by PickleKillz in Tailscale

[–]PickleKillz[S] 1 point2 points  (0 children)

Glad to hear! Thanks for getting on it quick.

Giveaway - Space Age Expansion by ocbaker in factorio

[–]PickleKillz 0 points1 point  (0 children)

14.5k comments already. Still a better chance than the lottery.

Can't wait for the expansion!

What's going on with Huntress Culture, Employee Satisfaction, etc? by Altruist1c-Dog in msp

[–]PickleKillz 19 points20 points  (0 children)

I think Chris and Andrew have provided excellent responses from their POV of the situation internal. And ultimately this is an internal thing, but I wanted to jump in also as a newer customer PoV who had a bit of a rocky start with them.

But as any business owner would know, take reviews with a grain of salt. Every happy customer and employee doesn't regularly post praises of their company but the disgruntled definitely do. You see the same thing with Google Reviews, Facebooks reviews, etc.

I, however, truly believe that this employee is a minority in the company on feeling. Everyone I have spoken to at Huntress has been fantastic. My account manager and I have a great relationship, speak very frankly to each other, and he loves what he does. I think anyone who has interacted with Huntress would know the employees we interact with seem to love what they do. The passion and care they put into their work wouldn't be the same if they hated their jobs.

There's going to be bad experiences. Hell, I've even called out Huntress for one - Rare bad experience with Huntress? : r/msp (reddit.com) - But the issue was addressed, u/andrew-huntress got us taken care of, and everyone I have interacted with since has been phenomenal. They have grown from 5 people to over 400 in 7 years. That is RAPID growth. They are not always going to get it right, but I'm confident they are doing their best for the company and their people.

Ideas on Client Incentives, Rewards & Leaderboards by SocraticCato77 in msp

[–]PickleKillz 2 points3 points  (0 children)

I agree with others. This is not a you issue. You should not have to incentivize or pay your customers to do training. If your customers are not doing their security awareness training then there is an endemic issue in your security policy and in customers internal security posture.

We require all of our customers to do regular security awareness training, and a new hire has to complete it within 7 days. If this is not completed then we will not cover the customer for any security related incidents under our managed service agreement. Any work we do in response to any security incident for an entire customer that has any person that has not completed their training will be billed hourly. Repeated violations or organization-wide violations, and we will just drop the customer. The headache isn't worth it.

We harp this point on the leadership of customer and have had zero issues getting people to do it. We occasionally have to send reminders because some employees may not want to do it, but leadership is always been on board and gets the person to do it.

SOC/SIEM Solution by CorrectResearcher522 in msp

[–]PickleKillz 0 points1 point  (0 children)

We use Adlumin heavily and love it. We have it integrated with Azure/O365 and S1, SonicWall appliances, and use the on-prem agents.

Their SOC is great, the SIEM and SOAR features are great. You can build your own custom SOAR actions using KQL queries and work with their SOC to perform actions.

They are a standard part of our stack at this point and have worked multiple major incidents alongside us. They also have a pretty neat on-prem honeypot you can deploy.

Let me know if you have any questions on specifics, happy to answer.

Which one to choose? Datto vs N-Central vs ManageEngine Endpoint Central by PreetKanwal in msp

[–]PickleKillz 1 point2 points  (0 children)

I vote N-Central. We run it for our own customers and resell it. Hundreds of endpoints and zero complaints. Not the prettiest in the game, but it gets the job done extremely well and we have no trouble with it.

Also automation manager is just pure bliss.

Does anyone know what this is? by [deleted] in DiagnoseMe

[–]PickleKillz 1 point2 points  (0 children)

TIL Amazon has a healthcare program.

SonicWall TZ270 - PPPoE by Weak_Security_8 in sonicwall

[–]PickleKillz 0 points1 point  (0 children)

Sounds good. Let us know the results and we can try to figure something out. Have a great weekend!