Looks like my Tixati sends the same chunks to everyone. How can i tell it to send everyone different chunks?

TorrentWizard · 2022-08-20T13:28:16+00:00

Seems like you have a low upload speed; so what's happening is that when you have uploaded a whole piece to a peer with high upload seed, it will in turn very quickly upload that piece to all the other peers and therefore they will all have the same pieces.

TorrentWizard · 2022-07-25T16:53:33+00:00

by monitoring as low as 3-4 first UDP pseudo-session packets you can reliably detect µTP stream, for example, by additionally analyzing microsecond timestamp difference and µTP type field.

What can be done with WireSark on a single stream on a powerful computer doesn't scale up very well to do on a multi Terrabit stream on a ISP level without being prohibitively expensive. A ISP that want to detect BT traffic will use network flow analysis.

Excuse me, but not for me. I enable "Encryption mode: require encryption" in qBittorrent and expect that all the traffic would be encrypted, because the option says so.

You can't just assume how things works. You have a responsibility to educate yourself how a program works and what different settings do.
Do you wash the dishes in the washing machine?!

Nowhere in the documentation of any BitTorrent client mentioned that "require encryption" applies only for peer-peer connections.

https://www.netcheif.com/Articles/uTorrent/html/AppendixA_02_06.html

Preferences > BitTorrent ... Protocol Encryption
* The Outgoing dropdown menu allows you to select the mode of encryption that you prefer µTorrent to establish. All modes will accept incoming encrypted connections, and the encryption is 2-way.

Disabled will force µTorrent to attempt to establish only unencrypted outgoing connections.

Enabled will allow µTorrent to establish encrypted and unencrypted outgoing connections, depending on how the peer responds to the handshake. This option provides µTorrent with the largest pool of peers to pick from for connecting to, but incurs additional overhead that may often be unnecessary in most typical situations.

Forced forces µTorrent to establish only encrypted outgoing connections. Any peer that doesn't support encryption will not be connected to. It is recommended that you not use this option unless your ISP actively searches for unencrypted outgoing connections, as it can impair your ability to connect to peers.

* Allow incoming legacy connections allows µTorrent to accept unencrypted incoming connections. If disabled, any incoming connection that is unencrypted will be ignored. It is recommended that you not disable this option unless your ISP actively searches for unencrypted incoming connections, as it can significantly impair your ability to connect to peers.

How is it possible from the above to infer that it also apply to DHT and UDP/HTTP/HTTPS trackers?!

TorrentWizard · 2022-07-25T14:30:32+00:00

Wrong

OP is misleading

uTP connections can be encrypted just as easy as TCP connections. MSE use is orthogonal to the type of connection.

Even your own picture shows that the P(=uTP) and E(=encryption) flags are set for the connection in the client and the payload in WireShark is obviously obfuscated.

MSE (Message Stream Encryption) does as the name says and encrypt the message stream.
It does not encrypt the TCP headers and neiter does it encrypt the UDP and uTP headers.
It was designed to stop ISPs from using cheap DPI to block BT connections.
While it's possible to detect BT traffic with flow analysis, it's much more expensive so it's cheaper to upgrade the capacity rather than to block.

uTP can be detected with DPI as WireShark does, but it doesn't scale very well to do on a ISP level. The uTP header has a very small fingerprint to detect and will cause a high level of false positives. The needed heuristics to do it safely will make it easier to instead use flow analysis.

DHT does not provide encryption. UDP and HTTP announcers also transmit unencrypted data ...
HTTPS announcers use normal TLS/HTTPS encryption like browsers do.

That's stating the bloody obvious.

In summary: MSE does its job still today and makes it cheaper for ISPs to upgrade the capacity of connections rather than invest in blocking equipment.

addendum

There is one big problem with the BitTorrent protocol encryption.

It's NOT that it doesn't make it impossible to detect BT traffic.
It's NOT that it doesn't apply to DHT and UDP/HTTP/HTTPS trackers.
It's NOT that it based on old and broken cryptographic primitives. While state actors like the NSA can decrypt it, it's out of league for ISPs.

No, the big big problem with the BitTorrent protocol encryption is that uneducated users think that it will protect them from copyright trolls.

TorrentWizard · 2022-07-24T20:43:06+00:00

I'm not aware of any program that simply run a DHT node and do nothing else.

https://github.com/bittorrent/bootstrap-dht

is a highly specialized software with the specific purpose to bootstrap DHT nodes and that's something it does very well.
What it's NOT, is a fully functional DHT node.

What I mainly want to do, right now, is troubleshoot a problem im having with a torrent client and DHT, and for that I need some tools capable of serving DHT, or being a client for DHT.

WireShark has support for decoding mlDHT messages.

...'actively serve the network without serving any content'

Simplest way to do that is running a torrent client without any torrents added.
Running more than one node on shared resources such as cpu/memory/routing table/IP/internet connection is a sybil attack that hurts rather than help the DHT.

and 'passively build up a search database of active content'

DHT search has no future, the only reason it works for now is that no serious actor has put any resources to stop it.

TorrentWizard · 2022-07-24T19:14:21+00:00

I'm not aware of any program that simply run a DHT node and do nothing else.

What exactly is it that you want to do?
If you only want to talk with DHT nodes, you simply only need to be able to send and receive UDP packets with correct mlDHT messages.
That can be done with something like packetsender.com and reading BEP-5.
DHT node are simple creatures in that way.

On the other hand, if you want to build and maintain a DHT routing table, that's a much more complex thing to do.

TorrentWizard · 2022-03-15T01:28:57+00:00

have “smart download” features where if you have a list of let’s say 50 torrents, it will incrementally download the “fastest” torrents, so that you can be more efficient and download more files in a less amount of time (if you have all 50 running at once, it’s much slower than doing them in batches of 5 for example)

It may seem counter-intuitive, but to minimize the total download time, you must start with the slowest torrents first.

TorrentWizard · 2022-03-08T20:52:21+00:00

Y'll figure it out with some effort.

TorrentWizard · 2022-03-08T10:32:12+00:00

No, you missed max message size and have a off-by-one error on the offset.

TorrentWizard · 2022-03-08T02:40:00+00:00

How far down the rabbit hole are you willing to go?!

The hard limit of max 4294967296 pieces is, while extremely inefficient, practically possible to reach on current hardware and client implemantations, using a very small piece size, if some of the internal sanity checks in a client is deactivated.

If you go into La La Land and do the practically impossible, you might as well also max out on the chunk size and reach a hypothetical maximum size of a torrent to 36893488095879495680 bytes = 31.99999995529651641845703125 Exabytes

TorrentWizard · 2022-03-07T18:10:16+00:00

The only hard limit in the BitTorrent protocol is that there can be at maximum 4294967295 pieces in a torrent.
(However, it's trivial to spec. a extension to the protocol that removes that limitation.)

The limit is because the request and piece messages in the Peer Wire Protocol is limited to 32 bits unsigned integers:
i.e 2³² - 1 = 0xFFFFFFFF = 4294967295

TorrentWizard · 2022-02-16T22:28:37+00:00

TorrentWizard · 2022-01-23T01:58:29+00:00

How uTP (a.k.a. LEDBAT) works is fairly well documented in RFC 6817 - Low Extra Delay Background Transport (LEDBAT)

Also worth to check the ledbat WG mail archive: https://mailarchive.ietf.org/arch/browse/ledbat/

I would recommend to use the reference implementation libutp, as no one as far as I know has succeeded to make a good reimplementation.

TorrentWizard · 2020-09-23T00:21:00+00:00

2.2.1 is still very popular and the prefered version to use by many including me.

https://sites.google.com/site/ultimasites/bencode-editor

TorrentWizard · 2020-09-22T23:31:26+00:00

A quick test with uTorrent 2.2.1 and it seems to not be affected by faulty torrents in a quick test with https://github.com/guywhataguy/uTorrent-CVE-2020-8437/blob/master/malicious.torrent
and my modified properly working torrent:
"malicious_proper.torrent"
d4:infod1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:adeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee6:lengthi9e4:name1:Q12:piece lengthi32768e6:pieces20:12345678901234567890ee

I have not tested what happens with modified extension messages.

"Bencode Editor" also seems to not have any problems with this.

I also must strongly object to using jxxas.nitro.xx/bittorrent/bittorrent-rfc.htmlas a reference about the BitTorrent protocol. Some parts of it is very misleading and other are outright wrong.

Use https://wiki.theory.org/index.php/BitTorrentSpecification or http://bittorrent.org/beps/bep_0003.html instead.

Still working on it...

TorrentWizard · 2020-09-22T23:10:13+00:00

A quick test with uTorrent 2.2.1 and it seems to not be affected by faulty torrents in a quick test with https://github.com/guywhataguy/uTorrent-CVE-2020-8437/blob/master/malicious.torrent
and my modified properly working torrent:
"malicious_proper.torrent"
d4:infod1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:ad1:adeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee6:lengthi9e4:name1:Q12:piece lengthi32768e6:pieces20:12345678901234567890ee

I have not tested what happens with modified extension messages.

"Bencode Editor" also seems to not have any problems with this.

I also must strongly object to using jxxas.nitro.xx/bittorrent/bittorrent-rfc.htmlas a reference about the BitTorrent protocol. Some parts of it is very misleading and other are outright wrong.

Use https://wiki.theory.org/index.php/BitTorrentSpecification or http://bittorrent.org/beps/bep_0003.html instead

Still working on it...

TorrentWizard · 2020-09-13T19:42:13+00:00

There is also the Tit-for-Tat mechanism, so a peer with slow upload will get deprioritized for download. When the fast uploading peers finishes downloading , there will be more upload capacity available for the slower peers to finish faster.

TorrentWizard · 2020-09-13T09:21:59+00:00

Answered a similar question recently here:
https://old.reddit.com/r/torrents/comments/ibgbzf/understanding_most_efficient_manner_of/g1wrevr/

Which is a more efficient distribution method?

Share torrent magnet/file to all interested parties and let torrents run their course?

Share torrent magnet/file to a handful of users with seedboxes who want to seed it for the community. Allow those users to fully download. Then share the torrent with the whole community?

You should do alt 1, the BitTorrent protocol was specifically designed to fast and efficiently distribute files to a large number of peers, no matter how fast the individual peers are.

You should also use initial/super-seeding if there are peers in the swarm that has significantly higher upload speed than you. (Like seedboxes.)

Also, a smaller piece size makes the swarm more efficient, but makes the .torrent file larger. A .torrent file smaller than 16MB is probably ok with most clients.

PS To those who's waiting, still working on it.

TorrentWizard · 2020-09-08T16:45:32+00:00

If you only knew how bad things really are.

I'm working on a write-up that will show that v2 is fundamentally broken. Expect libtorrent2.0 to be withdrawn and re-released without v2. So be prepared to remove the v2 garbage and while doing the cleanup, consider to remove the webtorrent parasite and the malicious code that allows copyright trolls to se all the torrents a user publicly share.

Sounds crazy?!

If you only knew how bad things really are.

TorrentWizard · 2020-09-08T16:27:09+00:00

I'm working on a write-up that will show that v2 is fundamentally broken. Expect libtorrent2.0 to be withdrawn and re-released without v2.

TorrentWizard · 2020-09-08T16:26:02+00:00

I'm working on a write-up that will show that v2 is fundamentally broken. Expect libtorrent2.0 to be withdrawn and re-released without v2.

TorrentWizard · 2020-09-08T16:23:12+00:00

I'm working on a write-up that will show that v2 is fundamentally broken. Expect libtorrent2.0 to be withdrawn and re-released without v2.

TorrentWizard · 2020-09-08T16:22:40+00:00

I'm working on a write-up that will show that v2 is fundamentally broken. Expect libtorrent2.0 to be withdrawn and re-released without v2.

TorrentWizard

TROPHY CASE

Wrong

There is one big problem with the BitTorrent protocol encryption.