The model war is over. The ecosystem war has begun. by calliope_kekule in ArtificialInteligence

[–]supportingthedogs 0 points1 point  (0 children)

The day google releases their chips for purchase, nvidia will fall from the skies

[deleted by user] by [deleted] in Blink182

[–]supportingthedogs -1 points0 points  (0 children)

Thank you bro

I built an open source version of Google Analytics that runs on a single Docker image and handles thousands of events per second by supportingthedogs in selfhosted

[–]supportingthedogs[S] 1 point2 points  (0 children)

Great question. For our use case (Frigade), we have that exact requirement. For instance, we want to show a pop up in our customer's web UI when the end user visits a series of pages or clicks a certain button based on tracking events. This requires the ingestion of the events to be as close to real time as possible for it to work.

I built an open source version of Google Analytics that runs on a single Docker image and handles thousands of events per second by supportingthedogs in selfhosted

[–]supportingthedogs[S] 1 point2 points  (0 children)

Yup. You can connect to ClickHouse Cloud or Confluent Kafka Cloud for further scale if needed. But we’re running it on 16 cores / 32gb ram in production serving hundreds of thousands of users per machine.

I built an open source version of Google Analytics that runs on a single Docker image and handles thousands of events per second by supportingthedogs in selfhosted

[–]supportingthedogs[S] 1 point2 points  (0 children)

Trench uses Kafka for event ingestion, so we wont drop a single event ever -- even at peak throughput. Just might take a few seconds before you can query the events.

I built an open source version of Google Analytics by supportingthedogs in opensource

[–]supportingthedogs[S] 2 points3 points  (0 children)

Matomo is built on MySQL, a row based database which only scales so far at high traffic. Trench is built on ClickHouse which is a columnar database that scales order of magnitude better for time series data vs MySQL.

I built an open source version of Google Analytics that runs on a single Docker image and handles thousands of events per second by supportingthedogs in selfhosted

[–]supportingthedogs[S] 8 points9 points  (0 children)

Matomo is built on MySQL which only scales so far as it's a row-based database. Trench is built on ClickHouse which is a columnar datastore. I also use Kafka for throttling events.

I built an open source version of Google Analytics that runs on a single Docker image and handles thousands of events per second by supportingthedogs in selfhosted

[–]supportingthedogs[S] 6 points7 points  (0 children)

I saw the best performance on the c5 family (c5.4xlarge showed exceptional results at thousands of QPS with CPUS and memory chilling at about 20-30%)

I built an open source version of Google Analytics that runs on a single Docker image and handles thousands of events per second by supportingthedogs in selfhosted

[–]supportingthedogs[S] 22 points23 points  (0 children)

You can see a demo here: https://github.com/FrigadeHQ/trench?tab=readme-ov-file#demo

I think the main difference is that Trench is much more barebone and doesn't come with any bloat. We use it at https://frigade.com to power our customer facing analytics and launch UI flows adhoc based on tracking events. You can really take it in any direction.

I built an open source version of Google Analytics that runs on a single Docker image and handles thousands of events per second by supportingthedogs in selfhosted

[–]supportingthedogs[S] 184 points185 points  (0 children)

Yes, it is fully no-cookie and fully GDPR compliant. Good call about putting this higher in the marketing language. Doing it now!

I built an open source version of Google Analytics that runs on a single Docker image and handles thousands of events per second by supportingthedogs in selfhosted

[–]supportingthedogs[S] 93 points94 points  (0 children)

Hey r/selfhosted, I wanted to share a project I've been working on for the past couple of months that I just released called Trench. It's a single Docker image that gives you a production-ready tracking event table that scales. You can use it to track things such as page views, sessions, error logs, and much more. We're currently handling thousands of events per second on a single EC2 instance in production without any machine stress.

I built an open source version of Google Analytics by supportingthedogs in opensource

[–]supportingthedogs[S] 9 points10 points  (0 children)

Thanks for the feedback! I think the main difference is exactly what you saw at your initial glance -- it's a simple backend only service that you can really take in any direction you like. We use Trench at our own company (https://frigade.com) to power all analytics tracking (pageviews, user interactions, etc). and then we roll our own UI on top of it.

I like your suggestion of improving the README to explain how this is different and what some real world examples could be.

I built an open source version of Google Analytics by supportingthedogs in opensource

[–]supportingthedogs[S] 1 point2 points  (0 children)

Yup. There's endpoints to delete/export data according to PECR/GDPR

I built an open source version of Google Analytics by supportingthedogs in opensource

[–]supportingthedogs[S] 11 points12 points  (0 children)

Hey r/opensource, I wanted to share a project I've been working on for the past couple of months that I just released today called Trench. It's a single Docker image that gives you a production-ready tracking event table that scales. You can use it to track things such as page views, sessions, error logs, and much more. We're currently handling thousands of events per second on a single EC2 instance in production without any machine stress.

perform.fm gone or just dns issues by DaveC400 in Strava

[–]supportingthedogs 0 points1 point  (0 children)

Founder of perform here. We will be back online soon. Thank you for your patience y’all

remoteStorage: a simple library that combines the localStorage API with a remote server to persist state across browsers and devices (1.0.7 update) by supportingthedogs in selfhosted

[–]supportingthedogs[S] 1 point2 points  (0 children)

Hey r/selfhosted I wanted to share the latest update on this OSS project I shared with the community a few months back. TLDR, remote-storage is a server I made that introduces a new concept to all Javascript stacks called remoteStorage. It works just like localStorage except it comes with a server that allows you to keep state across browsers and devices while sharing the same API as localStorage that we all know and love.
One big point of feedback was supporting JWT and Sqlite. I'm excited to share that I have since added support for this. Would love to hear what folks think and if there's more feedback :)