Client File Terminology

[anthony-joal]

Since we most likely be multiple contributors to this sub-section, we should agree on the same terminology. This thread is a non-exhaustive proposal for this concern.

Concerning key types:
This section define which types of keys are used by BitTorrent clients (which characters sets).

hash: The key is an HASH containing only hexadecimal characters [0-9a-z].
hash_no_leading_zero: The key is an HASH containing only hexadecimal characters [0-9a-z] (can be lower-cased or upper-cased), but the key will never start with a 0, if it does the leading 0 MUST be removed (0AB92E9F will become AB92E9F).
numeric: The key must be a random number [0-9]{x}.
digit_range_transformed_to_hex: A random digit that is transmited as it's hexadecimal representation, without leading zeroes.



Concerning peerid types:
Each client use his own subset of characters. Characters set will be defined in clients thread.

Concerning PeerId or Key refresh trigger:
There are a plenty of generation algorithm that refresh key / peerId. The below list explicitly give a name to each of these algorithms. (In the below list, subject refers to either a key or a peerid)

NEVER: The subject MUST NOT be refreshed until application restarts.
ALWAYS: The subject MUST be refresh on each announces.
TIMED: The subject MUST be refresh every X seconds.
TIMED_OR_AFTER_STARTED_ANNOUNCE: The subject MUST be refreshed every X seconds OR immediatly after an announce with type STARTED has been fired.
TORRENT_VOLATILE: Given a torrent, the subject will always be the same, MUST be refreshed when a STOPPED announce has been fired for this particular torrent. Two torrents MUST have two different subjects.
TORRENT_PERSISTENT: Given a torrent, the subject will always be the same, MUST NOT be refreshed when a STOPPED announce has been fired for this particular torrent, however if the application is restarted the subject MUST be refreshed. Two torrents MUST have two different subjects.


Fell free to comment for improvements or missing types.

[cloud99]

Hi there,
Can you add a new sintax to your client file knowledge section? "Scraping parameters" is what I request should be included. Though many low level trackers don't care about multiple scrapes, the high level trackers care about them. Some I might add disable them for more than one (in a single instance) torrent. Including the parameters used i.e.,



<scrape protocol="[PROTOCOL]
<infoHash isLowerCase="[YES OR NO]"
exceptions="[~!@$%^&*(()))_+{}:"<>?/.,';\][|=-`+ANY OTHER CHARS]" />
<header showDefaultPort="[YES OR NO]">
<field>[ANY OTHER INFO SENT TO THE TRACKER WHILE A SCRAPE ANNOUNCE IS SENT]</field>
<field>[i.e., IP type, IP mode, Static Number, Infohash, Passkey, etc]</field>


This isn't mandatory since some Torrent Clients don't have this feature and rely on the "update intervals". However, some client(s) do use this and if information regarding these are added, then creating client files based on this/these information would be easier.
~cloud99

[anthony-joal]

Hi cloud 99,
Sure, a forgot about scrape, i'd be glad to add them, can you send on each client topic your scrape format (if you have it)? :)

[cloud99]

Hi cloud 99,
Sure, a forgot about scrape, i'd be glad to add them, can you send on each client topic your scrape format (if you have it)? :)

Hi there,

I use a PHP-Torrent-Scraper scraper here: https://github.com/Edward-Stryfe/PHP-Torrent-Scraper to scrape HTTP trackers. This mainly limited to those sites which don't allow scraping at all or a limited number of times (to prevent server overload or faking to bypass minimum number of announces).

I mainly used this script to scrape data from trackers so that I can spoof some fake data more realistically without blowing my cover.

This is the code of the PHP script; I use to scrape data from certain trackers:


torrents = array();
foreach($infohash as $hash){
$ehash = pack('H*', $hash);
if (isset($arr_scrape_data['files'][$ehash])){
$torrents[$hash] = array('infohash'=>$hash,
'seeders'=>(int) $arr_scrape_data['files'][$ehash]['complete'],
'completed'=>(int) $arr_scrape_data['files'][$ehash]['downloaded'],
'leechers'=>(int) $arr_scrape_data['files'][$ehash]['incomplete']
);
} else {
$torrents[$hash] = false;
}
}


Something similar to this is what is expected. [Once again this format isn't mandatory, this is what I prefer, however, others may or may not agree and also once again they (the clients/trackers scraping algorithms) aren't similar].


Client.scrape({ announce: announceUrl, infoHash: [ infoHash1, infoHash2 ]}, function (err, results) {
results[infoHash1].announce
results[infoHash1].infoHash
results[infoHash1].name
results[infoHash1].complete
results[infoHash1].incomplete
results[infoHash1].downloaded
results[infoHash1].min_request_interval


Though I must say that each torrent client will have a different algorithm of presenting this data. Some may report certain parameters which are available for the other. Therefore the explorer [the one providing the information] would need to capture the scrape announce and check what are the parameters used.

After I had typed this entire conversation, I realized this isn't anywhere close to answering your question! I can provide you a scrape conversation format by Vuze. However, that has to be later on as I'm busy encoding some data right now.

Regards,
~cloud99

[anthony-joal]

After I had typed this entire conversation, I realized this isn't anywhere close to answering your question! I can provide

Haha the much i was reading the much i was asking to myself when you'll anwser the original question x).
It was instructive through. But when i'll add scrapping support for JOAL i most likely code my own scrapper.


Sadly the Vuze Thread is not created yet. If you have suficcient information about vuze maybe you can create and maintain the Vuze Thread.

[cloud99]

https://tracker:port/PRIVATE PASS KEY/scrape?hash_id=xxxxxxxxxxxxxxxxxxxx

GET /PRIVATE PASS KEY/scrape?info_hash=%thisiswheretheinfohashofatorrent isdisplayed%
User-Agent: Azureus 5.7.5.0
Accept-Encoding: gzip
Connnection: close
Host: thisisthetracker'shot.com
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2

Response:

d5:filesd20:(torrenthashnamehere)d8:completei4e10: downloadedi726e10:incompletei0eee5:flagsd20:min_re quest_intervali1800eee

DEcoding the response manually.
(torrenthashnamehere)= this is the torrent hash name.
completei4e10= 4 seeders.
incompletei0eee5= 0 leechers.
downloadedi726e10= downloaded 72 or 726 times (not sure).
min_request_intervali1800eee= next scrape after 1800 seconds or 30 minutes.


This is what a scrape announce look like or these are the parameters it (Vuze) sends.

---------- Post Merged at 15:25 ---------- Previous Post was at 15:24 ----------


If you have suficcient information about vuze maybe you can create and maintain the Vuze Thread.

Sadly I don't :frown:

[anthony-joal]

Thanks @cloud99 i'll try to add them when i'll have some free time :)