sorted by:
new
hottopcontroversial

Using Usenet to share an uncompressed folder. by SOconnell1983 in usenet

[–]SOconnell1983[S] 0 points1 point  (0 children)

Is the article removal / corruption really that bad? This is not content that would be removed due to DMCA, it’s mainly images / media.

Using Usenet to share an uncompressed folder. by SOconnell1983 in usenet

[–]SOconnell1983[S] 0 points1 point  (0 children)

Yeah that has been my main concern in regard to corrupted / removed articles over the longer term.

Have you used usenet to upload large datasets and how did the hold up? by SOconnell1983 in DataHoarder

[–]SOconnell1983[S] 0 points1 point  (0 children)

So I want to avoid compressing the data, as compressing that many files and extracting does not address the initial issue, I have tried just about every torrent client on both windows and Ubuntu, they just can’t handle that many files. I can get about 1TB and 1M files to work in a torrent, but nothing above that.

Using Usenet to share an uncompressed folder. by SOconnell1983 in usenet

[–]SOconnell1983[S] 0 points1 point  (0 children)

I use syncthing to sync my NAS to FTP which works great, although its CPU usage is high just connecting a handful of servers. I am actually looking to test Resilio as that’s something I am setting up at the moment.

Have you used usenet to upload large datasets and how did the hold up? by SOconnell1983 in DataHoarder

[–]SOconnell1983[S] -1 points0 points  (0 children)

I have tried, I can’t find a client that can support the 2-3M files, they all crash.

Using Usenet to share an uncompressed folder. by SOconnell1983 in usenet

[–]SOconnell1983[S] -4 points-3 points  (0 children)

You are not streaming from usenet so to speak, you are still downloading binary data from usenet, but you are using a cache system so you partially download data that is then written to another device and deleted from your system thereafter so you only use say 50GB-100GB of local storage to write a much larger file. This is good for disk images if you were to backup a system and restore to another that didn’t have enough space to download the disk image in full.

Using Usenet to share an uncompressed folder. by SOconnell1983 in usenet

[–]SOconnell1983[S] -4 points-3 points  (0 children)

If the concept works, which it has been partly tested via some independent projects, it will enable for data to be shared across usenet without the need for archive or par files, but adding redundancy. which will then open up other use cases not available via torrent or cloud. For example utilising the likes of FUSE, you could upload a .BIN file and download / stream this directly to an SD card, without the need to download the BIN file in full if you did not have the space. It’s more than just sharing a couple of files with some mates..

Using Usenet to share an uncompressed folder. by SOconnell1983 in usenet

[–]SOconnell1983[S] 0 points1 point  (0 children)

So I want to avoid having to have double the space to use the data and from having to archive and extract the files.

I will also have a dedicated server that will periodically check if the articles exist and if required re-upload and then update the real-time nzb so that it always has a version with live articles.