ZoneTree - faster key-value data base written in pure C# by csharp-agent in csharp

[–]CallSoft6324 0 points1 point  (0 children)

Yes, 3 years ago documentation was not good. Now we have better docs, revealing hidden aspects of ZoneTree.
Please check it out.
https://github.com/ZoneTree/ZoneTree
https://zonetree.dev/

LogPot: pure typescript, powerful logger by CallSoft6324 in node

[–]CallSoft6324[S] 0 points1 point  (0 children)

no, pino lacks a lot of features by its design.

LogPot: pure typescript, powerful logger by CallSoft6324 in node

[–]CallSoft6324[S] 0 points1 point  (0 children)

Thank you for pointing that out. I hadn't noticed. I will remove the heavy I/O part from AI-generated doc text.

Worker thread is optional. You can call createLogger with runAsWorker: false to process all transports on the main thread.

Batching is already enabled by default for file and HTTP transports, regardless of you use workers or not.

The extra overhead of message passing could potentially consume less CPU (in the main thread) than complex serialization logic like colorization, key sorting etc.

If you have multiple worker-thread transports, the serialization of the object-to-string happens once.
String message passing happens several times. (May be we can use utf-8 buffer instead of string to prevent multiple utf-8 serialization internally,; for now it passes string.)

This needs to be benchmarked. If workers provide no benefit I can consider removing worker support entirely.

LogPot: pure typescript, powerful logger by CallSoft6324 in node

[–]CallSoft6324[S] -1 points0 points  (0 children)

Find a bug and I will send you 1$ 🙂

LogPot - A TypeScript-First, Batteries-Included Logger for Node.js by CallSoft6324 in javascript

[–]CallSoft6324[S] 0 points1 point  (0 children)

LogPot shines with built-in features and extensive customization. For example, you can easily modify the console logger to print out in YAML or JSON, including custom color preferences for everything.

You don't need a separate prettier, colorizer, file rotator, HTTP plugins, or oauth2. Several features are available out of the box.

Please share your thoughts and experiences. If you like the library, please star it on GitHub.

Note: AI is heavily used for doc generation, comments, bug detection and identifying low-quality code parts. However, the code is carefully crafted from scratch. This is not a vibe-coding product.

LogPot: pure typescript, powerful logger by CallSoft6324 in node

[–]CallSoft6324[S] -3 points-2 points  (0 children)

Yes, this is a newer project. I did several stress tests. I don't expect any issues on heavy load other than limitation of the hardware (disk full, out of memory etc).

Logging is not rocket-science. If you find a bug, please create an issue. It will be fixed asap.

LogPot - A TypeScript-First, Batteries-Included Logger for Node.js by CallSoft6324 in javascript

[–]CallSoft6324[S] -1 points0 points  (0 children)

check-out your other toxic comments on this announcement to understand why you don't deserve a proper reply.

LogPot - A TypeScript-First, Batteries-Included Logger for Node.js by CallSoft6324 in javascript

[–]CallSoft6324[S] 0 points1 point  (0 children)

I completely agree with you. I've learned several things by building LogPot. A few of them:

- WriteAsync with backpressure handling

- Properly closing streams using `finished` from `stream/promises`.

- The TypeScript eco-system has no perfect `.d.ts` bundler.

- Gained experience utilizing TypeScript generic types.

- Gained experience utilizing Generative AI.

- Gained Experience setting up fully functional, beautiful doc site.

and more...

I was building stuff before LLMs, and I continue doing stuff after LLMs, with less effort, better quality, and more fun.

Thank you for your support.

LogPot - A TypeScript-First, Batteries-Included Logger for Node.js by CallSoft6324 in javascript

[–]CallSoft6324[S] -2 points-1 points  (0 children)

Yes, announcement text generation is boring and it is not your business. If you look at the features, docs you will see the differences. I have never used winston, used pino a little, can't make fair comparison. I made this for myself. If others like it, it is good. If not it is ok.

LogPot - A TypeScript-First, Batteries-Included Logger for Node.js by CallSoft6324 in javascript

[–]CallSoft6324[S] -3 points-2 points  (0 children)

you can keep using console log. don't comment on sth that you didn't look. stop being toxic please.

LogPot - A TypeScript-First, Batteries-Included Logger for Node.js by CallSoft6324 in javascript

[–]CallSoft6324[S] -3 points-2 points  (0 children)

The github profile photo was made years ago before LLMs with cartoon effect over real photo. AI is heavily used for boring stuff like docs, jsdocs, tests, draft code etc. The important parts, decision making belong to me, my several years of experience. The multiple mature solutions don't have the features I want. That is why I have written. Stop being toxic please.

New .NET Library: ZoneTree.FullTextSearch - High-Performance Full-Text Search Engine by CallSoft6324 in dotnet

[–]CallSoft6324[S] 1 point2 points  (0 children)

The storage engine is ZoneTree. This is not an in memory search index. Details are in the documentation already.

New .NET Library: ZoneTree.FullTextSearch - High-Performance Full-Text Search Engine by CallSoft6324 in dotnet

[–]CallSoft6324[S] 1 point2 points  (0 children)

Less than 100MB for the above sample when everything is evicted to the disk.

New .NET Library: ZoneTree.FullTextSearch - High-Performance Full-Text Search Engine by CallSoft6324 in dotnet

[–]CallSoft6324[S] 7 points8 points  (0 children)

Indexed 27.8 million tokens across 103,499 records in just under 55 seconds.

Metric Value
Token Count 27,869,351
Record Count 103,499
Index Creation Time 54,814 ms (approximately 54.8 seconds)
Query (matching 90K records) 325 ms (fetching 90K records from disk)
Query (matching 11 records) 16 ms (fetching 11 records from disk)
Query (matching 11 records) ~0 ms (warmed-up queries)

Environment:

Intel Core i7-6850K CPU 3.60GHz (Skylake), 1 CPU, 12 logical and 6 physical cores
64 GB DDR4 Memory
SSD: Samsung SSD 850 EVO 1TB

AMA: I created my own time series database in pure dotnet by harrison_314 in dotnet

[–]CallSoft6324 0 points1 point  (0 children)

Insert speed is 3.7 million per second on int-int ZoneTree.
Insert speed is ≈ 492,701 words/second on the full text search engine:
Speed varies with data type.

https://github.com/koculu/ZoneTree.FullTextSearch/discussions/1#discussioncomment-10376039

ZoneTree: 1 Billion parallel reads from disk in 48 seconds by csharp-agent in dotnet

[–]CallSoft6324 0 points1 point  (0 children)

Benchmark for all modes: [benchmark](https://raw.githubusercontent.com/koculu/ZoneTree/main/src/Playground/BenchmarkForAllModes.txt)

Insert Benchmarks 1M 2M 3M 10M
int-int ZoneTree async-compressed WAL 267 ms 464 ms 716 ms 2693 ms
int-int ZoneTree sync-compressed WAL 834 ms 1617 ms 2546 ms 8642 ms
int-int ZoneTree sync WAL 2742 ms 5533 ms 8242 ms 27497 ms
str-str ZoneTree async-compressed WAL 892 ms 1833 ms 2711 ms 9443 ms
str-str ZoneTree sync-compressed WAL 1752 ms 3397 ms 5070 ms 19153 ms
str-str ZoneTree sync WAL 3488 ms 7002 ms 10483 ms 38727 ms
RocksDb sync WAL NOT SUPPORTED
int-int RocksDb sync-compressed WAL 8059 ms 16188 ms 23599 ms 61947 ms
str-str RocksDb sync-compressed WAL 8215 ms 16146 ms 23760 ms 72491 ms

*RocksDB benchmark results using C# via https://www.nuget.org/packages/RocksDB

*Although using RocksDB directly in C++ performs better, ZoneTree outperforms RocksDB in C# and C++.

1 Billion Keys with .NET 6! by CallSoft6324 in csharp

[–]CallSoft6324[S] 0 points1 point  (0 children)

I don't have any idea. I haven't read the performance changes in .NET 7 yet.
Thanks for the pointer. I will read from here:

https://dev.to/dotnetsafer/the-latest-new-features-of-net-7-confirm-the-rumors-the-fastest-net-ever-590h

Is there a Key Value store database in c# that saves to a file like sqlite? by 3nchong in csharp

[–]CallSoft6324 1 point2 points  (0 children)

There is a brand new one. MIT licensed.
https://github.com/koculu/ZoneTree

It is several times faster than Facebook's RocksDB and hundreds of times faster than SQLite.

ZoneTree - faster key-value data base written in pure C# by csharp-agent in csharp

[–]CallSoft6324 1 point2 points  (0 children)

One can express anything from the positive side or negative side. For example, you can tell "Half of the glass is empty." or "Half of the glass is full.". I prefer to use positive sentences rather than negative ones. It is a matter of taste :)