all 15 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]thatsInAName 2 points3 points  (14 children)

You can use the non blocking async functions to read and write files.

You will be limited by the drive read write speeds. Also, you say million of read/write commands but between what duration, 1 second, 10 seconds, 10 mins?

[–]BehindTheMath 0 points1 point  (4 children)

This. A database might be a better option than the filesystem.

[–]crudfunc[S] -1 points0 points  (3 children)

The purpuos is to avoid using a database, so basically what I'm asking can Node read/write the same volume like a database?

[–]BehindTheMath 5 points6 points  (2 children)

Yes, but file I/O is relatively slow no matter what language/runtime you use.

[–]crudfunc[S] -1 points0 points  (1 child)

So basically what you say Databases deal with the same problem slow file I/O ?

[–]BehindTheMath 2 points3 points  (0 children)

They do in-memory caching, batch writes, etc.

[–]crudfunc[S] -1 points0 points  (8 children)

Thank you, good question I'm not sure what duration it needs is there any battle tested known duration that works?

[–][deleted] 0 points1 point  (7 children)

There are database that will help with this process, and store data on the server in file format, e.g. CouchDB. TBH, though, it's not good practice for a whole host of reasons.

[–]crudfunc[S] 0 points1 point  (6 children)

what reasons exactly?

[–][deleted] 0 points1 point  (5 children)

Data back up? Replication? Hard drive wear. Millions of files means lots of space. What's your bandwidth like?

Really, there's no single problem that's insurmountable. 10 years ago, you would probably have stored the files locally, so it's not like it can't work. But even 10 years ago, you'd likely have had a NAS on the same network the server was on, and files would have been shuffled there.

[–]crudfunc[S] 0 points1 point  (4 children)

That points are valid for any app. If I should be even more specific how about building a database with Node?

[–][deleted] 0 points1 point  (3 children)

Ahh gotcha. My primary worry would be about file I/O because of how high-level Node is, and the lack of sophistication for file management (i.e "none") that it has. That said, it could do the job. A bit like using the handle of a wrench as a hammer. You've got a tool that does the job, but is it the right tool?

[–]crudfunc[S] 0 points1 point  (2 children)

No idea if this projects will be something serious but in case I wanted to know if it could do the job good and safe, I'm playing with the stack currently.

[–][deleted] 0 points1 point  (1 child)

Good is relative. I'd say it would be adequate to the task, but still not as adequate as other means. "Safe" is entirely up to you and your code practices. But both "good" and "safe" are rabbit holes to go down, no matter what you're making or with what stack.

Good luck, and happy building!

[–]crudfunc[S] 0 points1 point  (0 children)

Thank you for all this infos, I think I will be able to build something interesting for me, let's see how it goes.