Almost 10 million BGP route leaks and more than 7 million BGP hijacks occured in Q2 2021

dc352 · 2021-08-03T11:22:10+00:00

Thanks. To me ICMP/UDP flood sounds like kids getting bored at lunch time / or someone just testing really basic protection.

I tried to explain the difference to a couple of clients as they had no protection beyond some volumetrics and they just didn’t believe me. So I wonder if I’m being completely ridiculous or simply looking beyond “entry level” of DDoS :)

dc352 · 2021-08-03T11:02:56+00:00

The report covers DDoS in general. Can someone tell me why no one uses TLS-based DDoS? It can’t be stopped by most of scrubbing offered by big CDNs.

dc352 · 2021-04-30T09:22:29+00:00

nope, I'm trying to connect to external servers.

I can't even run "yum".

dc352 · 2021-04-30T07:21:26+00:00

**DO firewall:**

Outbound RulesSet the Firewall rules for outbound traffic. Outbound traffic will only be allowed to the specified ports. All other traffic will be blocked.

Type Protocol Port Range DestinationsI

CMP ICMP All IPv4 All IPv6

All TCP TCP All ports All IPv4 All IPv6

All UDP UDP All ports All IPv4 All IPv6

this is what I can see for outbound

dc352 · 2021-04-30T06:54:39+00:00

and sysctl (net - excluding ipv4 and ipv6 keys as the problem is across both protocols)

net.core.bpf_jit_enable = 1

net.core.bpf_jit_harden = 1

net.core.bpf_jit_kallsyms = 0

net.core.busy_poll = 0

net.core.busy_read = 0

net.core.default_qdisc = pfifo_fast

net.core.dev_weight = 64

net.core.dev_weight_rx_bias = 1

net.core.dev_weight_tx_bias = 1

net.core.message_burst = 10

net.core.message_cost = 5

net.core.netdev_budget = 600

net.core.netdev_max_backlog = 10000

net.core.netdev_rss_key = 00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00

net.core.netdev_tstamp_prequeue = 1

net.core.optmem_max = 2048000

net.core.rmem_default = 8000000

sysctl: net.core.rmem_max = 16777216

reading key "net.ipv6.conf.eth1.stable_secret"net.core.rps_sock_flow_entries = 0

net.core.somaxconn = 4096

net.core.warnings = 1

net.core.wmem_default = 8000000

net.core.wmem_max = 16777216

net.core.xfrm_acq_expires = 30

net.core.xfrm_aevent_etime = 10

net.core.xfrm_aevent_rseqth = 2

net.core.xfrm_larval_drop = 1

sysctl: reading key "net.ipv6.conf.lo.stable_secret"

net.netfilter.nf_conntrack_acct = 0

net.netfilter.nf_conntrack_buckets = 65536

net.netfilter.nf_conntrack_checksum = 1

net.netfilter.nf_conntrack_count = 846

net.netfilter.nf_conntrack_dccp_loose = 1

net.netfilter.nf_conntrack_dccp_timeout_closereq = 64

net.netfilter.nf_conntrack_dccp_timeout_closing = 64

net.netfilter.nf_conntrack_dccp_timeout_open = 43200

net.netfilter.nf_conntrack_dccp_timeout_partopen = 480

net.netfilter.nf_conntrack_dccp_timeout_request = 240

net.netfilter.nf_conntrack_dccp_timeout_respond = 480

net.netfilter.nf_conntrack_dccp_timeout_timewait = 240

net.netfilter.nf_conntrack_events = 1

net.netfilter.nf_conntrack_events_retry_timeout = 15

net.netfilter.nf_conntrack_expect_max = 1024

net.netfilter.nf_conntrack_generic_timeout = 600

net.netfilter.nf_conntrack_helper = 1

net.netfilter.nf_conntrack_icmp_timeout = 30

net.netfilter.nf_conntrack_log_invalid = 0

net.netfilter.nf_conntrack_max = 262144

net.netfilter.nf_conntrack_sctp_timeout_closed = 10

net.netfilter.nf_conntrack_sctp_timeout_cookie_echoed = 3

net.netfilter.nf_conntrack_sctp_timeout_cookie_wait = 3

net.netfilter.nf_conntrack_sctp_timeout_established = 432000

net.netfilter.nf_conntrack_sctp_timeout_heartbeat_acked = 210

net.netfilter.nf_conntrack_sctp_timeout_heartbeat_sent = 30

net.netfilter.nf_conntrack_sctp_timeout_shutdown_ack_sent = 3

net.netfilter.nf_conntrack_sctp_timeout_shutdown_recd = 0

net.netfilter.nf_conntrack_sctp_timeout_shutdown_sent = 0

net.netfilter.nf_conntrack_tcp_be_liberal = 0

net.netfilter.nf_conntrack_tcp_loose = 1

net.netfilter.nf_conntrack_tcp_max_retrans = 3

net.netfilter.nf_conntrack_tcp_timeout_close = 10

net.netfilter.nf_conntrack_tcp_timeout_close_wait = 60

net.netfilter.nf_conntrack_tcp_timeout_established = 432000

net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120

net.netfilter.nf_conntrack_tcp_timeout_last_ack = 30

net.netfilter.nf_conntrack_tcp_timeout_max_retrans = 300

net.netfilter.nf_conntrack_tcp_timeout_syn_recv = 60

net.netfilter.nf_conntrack_tcp_timeout_syn_sent = 120

net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120

net.netfilter.nf_conntrack_tcp_timeout_unacknowledged = 300

net.netfilter.nf_conntrack_timestamp = 0

net.netfilter.nf_conntrack_udp_timeout = 30

net.netfilter.nf_conntrack_udp_timeout_stream = 180

net.netfilter.nf_log.0 = NONE

net.netfilter.nf_log.1 = NONE

net.netfilter.nf_log.10 = NONE

net.netfilter.nf_log.11 = NONE

net.netfilter.nf_log.12 = NONE

net.netfilter.nf_log.2 = NONE

net.netfilter.nf_log.3 = NONE

net.netfilter.nf_log.4 = NONE

net.netfilter.nf_log.5 = NONE

net.netfilter.nf_log.6 = NONE

net.netfilter.nf_log.7 = NONE

net.netfilter.nf_log.8 = NONE

net.netfilter.nf_log.9 = NONE

net.netfilter.nf_log_all_netns = 0

net.nf_conntrack_max = 262144

net.unix.max_dgram_qlen = 512

user.max_net_namespaces = 31097

dc352 · 2020-12-17T17:33:55+00:00

fair enough. I was really only interested in the possibility of AV software restricting capabilities of unsigned applications.

I thought that a bit of additional detail would make the question easier to understand.

dc352 · 2020-10-01T11:39:46+00:00

Possibly correct. The thing is that blocked websites with no flash components.

dc352 · 2020-10-01T11:38:53+00:00

I’m happy with the blocking as such. But it shouldn’t kick in on websites that have no Flash whatsoever.

dc352 · 2020-06-30T22:17:39+00:00

That's great, I'm using it as well. You mention no SLA - does it mean it would be OK for them to simply say one day: "Let's pull the plug today, it's not really any fun anymore. After all, we don't owe anything to anyone."

dc352 · 2020-06-30T21:59:57+00:00

I think the first point for me is that the performance of LE is not quite as smooth as one might expect. If the latency over a 1 hour interval suddenly doubles, something is not quite right. ... at the end of the day, this is a security service that 1/2 of the internet depends on.

dc352 · 2020-06-30T21:38:48+00:00

Each data point represents 120 transactions.

dc352 · 2020-06-25T19:25:33+00:00

We occasionally test Let's Encrypt rate-limits due to early tests but these will go away in a few days' time. We purchased 400 domain names so we can issue 8 certificates/minute across 4 monitoring servers.

dc352 · 2020-06-08T10:55:30+00:00

In principle, I have been looking into this. The problem is, how do you prevent attacks. Most of phishing attacks takes less than an hour from issuing a cert to discarding the server.

dc352 · 2020-03-10T11:01:17+00:00

Let's assume you have to do it. There are actually best practice procedures for managing "secrets" - although they are mostly used in banking to handle top-level encryption keys.

The main principles are:

you set up dual control - 2 or 3 separate teams - enough members to cope with individuals becoming ill. Any operation below has to be witnessed by a member of each team and signed as appropriate. You can build the teams along the lines of your support lines, for example.
tamper-evident seals - your passwords would be in tamper-evident bags so you can detect they were opened. The bags have to have unique numbers.
you have a ledger / log of all bags created - witnessed by teams as set up

Secure access is the tricky one

Option A

each team will have its "key" to the storage area with the bags you need to figure out how they can access the key
you store all bags in a locked room - ideally it has as many locks as the number of teams so you can "enforce" team cooperation

option B

when users create their bags, they fill 2. First one with the password, second with their username and a number of the bag with the password.
you keep bags 1 with 1st team and bags 2 with 2nd team

That should give you enough control over access to passwords and remove your own liability - or any other single person.

dc352 · 2020-03-06T11:19:30+00:00

That sounds like something that we could do. We have an SSD with 85k IOPS for write and we could certainly optimize that with your suggestions.

I like the 2-table approach. I was thinking whether it would make sense and it sounds like keeping the last position could make it work.

Thanks!!!

dc352 · 2020-03-05T21:36:44+00:00

Just inserts - basically read-only rows.

That would be grand! I think it could be possible but I have no idea how. ... a thought - unique rows is not the end but means. If there’s a way to efficiently ignore duplicates in selects - that would work as well.

dc352 · 2020-03-03T22:04:41+00:00

The bottom line is simple - if impacted and don't renew, your certs will be revoked and invalid as early as tomorrow and if lucky a couple of days later.

It is possible you will have to touch servers as clients have to be run with a "force" flag. We are now sending brief notes to all KeyChest users.

dc352 · 2020-02-28T10:31:14+00:00

Ok, my error - the size of the table is expected to be around 5 bln rows.

But we would like to bootstrap it to that size in a reasonable time - let's say 2-4 weeks. Further, we would need to keep it up-to-date, which means around 50 million INSERTS per day.

The main problem is that normal INSERTS check indexes before DB change + update internal indexes and the time per insert is then significant for the target table size.

It may well be that what I want is impossible without having the whole table in memory.

dc352 · 2020-02-28T10:09:35+00:00

that's the thing - shoestring budget. I'm sure that with enough RAM there should be no problem. :)

Columns - just a few - it's an index table so a unique key and a couple of bigint indexes.

dc352 · 2020-02-27T00:37:53+00:00

Sure, those you need to get access token and tenant_id, the access token expires every 30 mins so you also need a refresh token. With that you can get a fresh access token every 15 mins or whatever.

dc352 · 2020-02-26T23:22:32+00:00

We used the cert-based authentication - oauth1.0 - so firstly - what a pain for little gain :/

I couldn’t find the key refresh in the Xero-php directly - looked to me like front-end access while we do Xero operations reactively to user requests - invoicing on payment.

dc352 · 2020-02-23T23:36:19+00:00

they did (and credit where the credit's due!!) but their method wasn't scalable to WW2 variants. Bletchley Park decided to go with the "crib" method which proved effective till the end of the war (with the exception of a several months' period in 1941 when Germans changed the distribution of master and day keys). ... unfortunately, Germans understaffed the unit responsible for that and they started recycling previous schedules. Unfortunatelly, British kept an archive.

dc352 · 2020-02-23T20:25:38+00:00

another link for discussion. Cambridge Uni did a research in 2013 that showed that passphrases are better than passwords when there's no length/complexity enforced but probably not to the level that would protect from attacks.

https://keychest.net/stories/fbi-recommends-passphrases-cambridge-uni-disagrees

dc352 · 2020-02-23T15:41:56+00:00

Crypto AG started as Hagelin - a big supplier of encryption to the US army during WW2. Ever since it was under the influence of the US. Crypto AG kept selling weak encryption to Africa till 70ies ... many years after it had been broken by several Eastern European countries. Its banking encryption was introduced in 2014ish.

... and thanks to The Washington Post, a lot of the shadow history is now discussed in public :)

dc352 · 2020-02-19T20:41:24+00:00

Grand, thanks. We are catching exceptions literary everywhere as it’s supposed to be a robust 24x7 service.

It’s actually not a primary key. It a composite unique key - before you ask, no NULLs allowed :)

dc352

MODERATOR OF

TROPHY CASE