'Manual announce'

[Sansho]

Hi guys ... I've got a kind of technical question.


Background : I've got a 'modest' fiber connection ... and the respective router ports are open (for sure) ... so I guess that I am 'connectable'. On the other hand, my client is Ktorrent. I am reasonably happy with both of them (the fiber connection and the torrent client).


Well, the specific case : I am seeding ONE torrent ... together with, let's say 20 more seeders. During hours NO leechers ... that's normal ... but, at an specific moment, I reload the tracker website and ONE leecher appears. I check Ktorrent and it's NOT uploading anything. Then I right-click to 'manual annouce' ... and then it STARTS uploading (sadly many times when the leecher is almost 'completed' ... so I don't get much upload).

The question : Is that 'normal'? Shouldn't it have to be automatic? ... without me having to regularly paying attention at the tracker website and to 'manual announce' myself?

Obviously ... what I would like to happen is that AS SOON AS a 'leecher' appears ... to start uploading '''my part''' ...


Am I very wrong? ... maybe too ambitious?


Thank you very much !!!!!!!



Ps.- I forgot to mention, and it MIGHT be important ... that, what I usually seed is placed at an external USB3 HDD (I assign both 'usb-storage' and 'jbd2/sbd*' tasks a high-priority) ...



Edit: Actually I am seeding about two hundred torrents (6 TiB) scattered at two external USB3 HDDs. On the other hand, frecuently Ktorrent seems to upload regularly without me intervening, but ... as described before, if I pay attention, it sometimes seems to be working in a 'sub-optimal' way. I don't really know whether I might need to adjust a couple of parameters ... I am a bit lost ...

[anon]

I check Ktorrent and it's NOT uploading anything. Then I right-click to 'manual annouce' ... and then it STARTS uploading (sadly many times when the leecher is almost 'completed' ... so I don't get much upload).

That sounds like you aren't connectable at all. Otherwise, you'd be uploading as soon as the leecher starts the torrent and subsequently connects to you, which is why this is so important to begin with! Anyway, here's a summary of what you should be checking.

• Does the test at canyouseeme.org say it can connect to you? (Run it for the same port and from the same IP you're using your torrent client from.)
• Does your ISP use CGNAT? If so, forget about connectability.
  
  • Well, using the security camera/PlayStation excuse to request a public IPv4 address could work, but there may be additional charges.
  • Some providers may also implement ingress filtering, but that's usually for a small amount of privileged ports abused by botnets and very unlikely to get in your way.
• How are you forwarding the ports?
  
  • If using UPnP, don't. I have never seen a router implement it correctly; some even interpret a lease time of zero (meant to represent an indefinite period) literally and delete the port mapping immediately after creation *sigh*
  • If using DHCP, either switch to setting a fixed IP on your computer or configure a static lease for its MAC address. Forwarding ports won't do much if the destination IP is no longer the one you're using. Same goes for the listen port in KTorrent - don't change it.
• The firewall/DDoS protection feature on routers is often overzealous and blocks small unsolicited packets as an alleged attack, and it so happens that "small unsolicited packets" perfectly describes incoming BitTorrent connections.
  
  • Turn it off, or if not possible, set the lowest level available.
• If you have more than one router at home, make sure they're all bridged to the same network segment and not doing double NAT. Such a setup is highly undesirable.
• Last but not least, you also need to make the firewall on your computer (likely iptables in your case) allow incoming connections.

Note that for you to be connectable, all of the above must be a-okay. The port check site will help a lot, and you can use Wireshark or nc to see the connection attempt arriving.

[Sansho]

Well ... first of all, 'anon' , let me say a very big 'THANK YOU!!!' for such an extensive and literate answer ...

I will go through all the steps you've provided me. Just please have in mind that ... I am not such a network computer specialist at all ...


Of course I may be wrong ... but let me explain why I said that I was 'connectable' "for sure" ... ... it is through an a maybe stupid kind of 'canary test' :

I started my 'grey hat' days with a 3G connection and 'Amule' (emule for linux) ... ... evertything went OK. At some point in time my ISP enabled a CGNAT ... and 'Amule' switched from 'High ID' to 'Low ID' ... yes, it was worse than before. The years passed by and I migrated to a ... what the technician called a 'coaxial fiber' [to some point 'optical fiber' ... and then 'coaxial cable' ... ... yes, I know, the slowest is the 'important' one] ... either way, I am happy with my modest 'fiber' connection. For 'Amule' NOTHING changed ... ... until a couple of months after running on the 'coaxial fiber' ... I dared to 'forward' the router ports myself. Then 'Amule' came back to the green arrows of 'High ID' again.




- By the way ... 'CanYouSeeMe.org' says :

Ktorrent main port : OK
UDP tracker Ktorrent port : Fail (strange!)
DHT UDP (disabled) : Fail (strange!)



- All these previous three ports have their respective exceptions on the laptop GUI Firewall ... for 'incoming' and 'outgoing' connections. Ktorrent main port also have its respective exceptions for IPv6 enabled.

I ignore whether my unique router at home have any kind of 'internal' firewall. Edit: the router configuration page says "Firewall: On".

Edit:There is a sub-menu where it seems that I can de-activate the router firewall, but before doing anything it says : "Warning: By using the exposed host function you bypass the firewall of your router. Please make sure that your computer is protected against attacks from internet.
The following ports will NOT be forwarded: %s."

It seems it might de-activate also the port-forwarding function !!!




- I think my 'coaxial fiber' connection is NOT 'under' a CGNAT.




- I do have managed to remember how to access the 'router configuration page' ... and this is what I can see :

* UPnP is OFF

* DHCP is ON ... from aaa-bbb-ccc-10 to aaa-bbb-ccc-254 ... [I think 'ifconfig -a' says mine is aaa-bbb-ccc-10]

* Static DHCP - LAN fields are EMPTY

'ifconfig -a' gives me one MAC address ... and the router configuration page gives me several MAC addresses (LAN, WiFi 2.4GHz & WiFi 5GHz) all of them different from the one remited by the command line ...




- Regarding 'port forwarding' ... this is what I see from the router configuration page :

Ktorrent main port : (the one that was OK) is set for 'BOTH' protocols
UDP tracker Ktorrent port : (one of the Failed) is set just for 'UDP' protocol
DHT UDP (disabled) : (one of the Failed) is set just for 'UDP' protocol

There is also a switchable (ON/OFF) parameter I have never touched : 'NAT Traversal' which is ON


One of my pending tasks is learning how to use 'Wireshark' ... ... it's FOSS ... and it has a nice GUI ... so it's a matter of time ... ...



Conclusion : I agree with you that someway it seems I am NOT 'fully' 'connectable'. It is strange to me that the UDP ports failed the 'CanYouSeeMe.org' test ... even though the configuration seems OK. I am not sure whether this last point is important (because the 'main port' seems to run properly). On the other hand I guess I CAN switch to a fixed IP by turning DHCP range from [from aaa-bbb-ccc-10 to aaa-bbb-ccc-254] to [from aaa-bbb-ccc-10 to aaa-bbb-ccc-10]. Finally I don't mind switching off the router internal firewall as long as port forwarding remain intact.




So ... I am going to proceed. I'll do it first by deactivating the router internal firewall ... if it doesn't work, I will modify the DHCP range ... ... wish me some luck !!!


To verify ... I will need some time ... in order to the specific casuistry appears again ...





Thank you !!!!!!!!!

[Sansho]

First impressions after deactivating the router internal firewall ...


I was worried with : "Warning: By using the exposed host function you bypass the firewall of your router. Please make sure that your computer is protected against attacks from internet.
The following ports will NOT be forwarded: %s."

The '%s' is NOT configurable ... I guess it stands for 'string'.



'CanYouSeeMe.org' test responds exactly the SAME as before ... ... 'Amule' ... thanks god, keeps on 'High ID' ... and 'Ktorrent' apparently behaves ... as always [it is SOON to verify any 'change'] ...


Apart from that ... internet interacction seems a bit 'crispier' , I felt it instantaneously ... probably the general latency has improved ...


What I expect ... is reaching a point where ... MOST of my 'starting' uploads are over 'leechers' with a 'very LOW' level of completeness ... ... that would be 'the real proof' ... or ... as we say in Spain ... 'la prueba del algodón' ...



[after SIX sleeping hours]

Apparently NO change ... everything operating as always ... I'll keep my eyes wide open ...

[Sansho]

First instance: a couple of torrents started uploading ALONE (without my intervention [through 'manual announce'] ) . Both torrents from the same 'leecher' (same IP) . I noticed it more or less soon ... and when I check the level of 'completeness' of both torrents (from the 'leecher' perspective) ... they were around 2% full ... so ... they started uploading probably from THE BEGINING ... ... it looks good ...

I will wait for several more instances ... ... this could just be some good luck ...


Negative instance: I see a NEW leecher at the tracker site ... I check Ktorrent ... and NOTHING is happening. I right-click to 'manual announce' ... and then it starts uploading [in this ocassion on a low level of 'completion' ... and, to be fair, I was heavy uploading concurrently some other torrents ... but, after me intervining, that wasn't seem to be a problem] ... it doesn't look good ...


Some more background: This is all about 'HD-Torrents' ... in order to create a confortable 'bed' ... and 'enjoy' the experience ...


Another negative instance ... ... ... ... I am starting to think there might be an additional 'factor' : due to my particular circumstances, my 'Ktorrent' version is VERY 'old' [4.3.1] ... maybe it does not support I don't know what 'updated' network protocol version ... I don't know ...


Regarding the repeated 'failed' tests on 'CanYouSeeMe.org' on UDP ports ... ... it might be due to they just test TCP protocol ...



Another 'negative' instance ... and another new one ... ... it seems the router internal firewall was NOT the 'CAUSE' ...


I would like accentuate again my concern regarding my data is residing on external USB3 HDD [sometimes when I 'click' on a torrent after some hours without any 'interaction' ... it takes 'Ktorrent' MANY!!!!! seconds (about 10seconds) to respond] ... so I will keep a subset of NEW torrents in my SSD main drive ... just to discard it as a potential 'cause' ...


As I previously described ... the NEXT STEP will be switching to a fixed IP by turning DHCP range from [from aaa-bbb-ccc-10 to aaa-bbb-ccc-254] to [from aaa-bbb-ccc-10 to aaa-bbb-ccc-10].


To be sincere ... I tend to believe that the 'external HDD factor' is more a 'probable cause' than 'modifying the DHCP range' . Either way, we will see ...


Note: The next 'report' will take some more time ...




Regards.

[anon]

'CanYouSeeMe.org' says :

Ktorrent main port : OK
UDP tracker Ktorrent port : Fail (strange!)
DHT UDP (disabled) : Fail (strange!)

That's very good. Note CanYouSeeMe only tests TCP since UDP port scanning is far more complex (see https://www.sb-innovation.de/showthread.php?p=367380), but port mappings can be configured or duplicated for both protocols. If the port numbers differ, you'll have to forward all of them, but it seems you've already done that.

There is a sub-menu where it seems that I can de-activate the router firewall, but before doing anything it says : "Warning: By using the exposed host function you bypass the firewall of your router. Please make sure that your computer is protected against attacks from internet.

Sounds like you're attempting to activate the DMZ function, which forwards all incoming traffic not matching any rules to a single host in your local network. Easy and potentially tempting, but not recommended since it completely exposes that host to the outside world. (DMZ does not otherwise bypass NAT or firewall rules set on your computer, but it's still a bad idea.)

'ifconfig -a' gives me one MAC address ... and the router configuration page gives me several MAC addresses (LAN, WiFi 2.4GHz & WiFi 5GHz) all of them different from the one remited by the command line ...

Not possible unless you're looking at the wrong interface or are not in the same network segment as the router.

- Regarding 'port forwarding' ... this is what I see from the router configuration page :

Ktorrent main port : (the one that was OK) is set for 'BOTH' protocols
UDP tracker Ktorrent port : (one of the Failed) is set just for 'UDP' protocol
DHT UDP (disabled) : (one of the Failed) is set just for 'UDP' protocol

There is also a switchable (ON/OFF) parameter I have never touched : 'NAT Traversal' which is ON

All of this looks okay.

On the other hand I guess I CAN switch to a fixed IP by turning DHCP range from [from aaa-bbb-ccc-10 to aaa-bbb-ccc-254] to [from aaa-bbb-ccc-10 to aaa-bbb-ccc-10].

Don't do this, you will break DHCP for everything else in the network. Set up a static lease or reserved address instead.

Other things to check include...

• Maximum connection and upload slot limits. These will be reached easily if set too low, and can result on missed uploading opportunities. With FiOS, you can afford to be generous, so do. (I'm old enough to remember the 254 connection cap in Linksys' BEFSR series...)
• Maximum connection attempts per second and half-open connections limit. These do not apply to incoming traffic, but should still be tuned.
• Protocol encryption. If forced, unencrypted incoming connections are dropped, and it's up to that peer to realize this, enable it for you and try again. This will not necessarily occur in reverse as they may accept encryption while never initiating it (thus leading to the behavior you have experienced).

As for the external disks, some advice against seeding from those, but you'll be fine as long as you kill their power management (see https://www.sb-innovation.de/showthread.php?p=367491). HD-T has small and competitive swarms where the advantages of being connectable truly shine, so it's a good place to test and quickly validate the effectiveness of any changes.

[Sansho]

More background: My 'local network' at the moment is just ONE laptop ... ...

Sounds like you're attempting to activate the DMZ function, which forwards all incoming traffic not matching any rules to a single host in your local network.

EXACTLY !!! ... to be more specific ... everything is sent to aaa-bbb-ccc-10 [my laptop]

It's been like this for five days ... the diference is, as previously stated, internet interaction feels a bit 'crispier' than before ...

My laptop firewall, of course, is ON ... with the previously mentioned exception rules (besides those from 'Amule') ...

If you think it's NOT a good idea ... I can turn it OFF ... as it were at the beginning ...



I haven't modify DHCP range YET ... it would work since my ONLY 'local network' is just me ... ... I will wait because you have given me another promising cue [I'll talk about it later on] ...

In case I would want to configure a static lease for my laptop ... I would use the MAC address 'ifconfig -a' have given me. But I won't touch this neither ... yet ...

Protocol encryption. If forced, unencrypted incoming connections are dropped, and it's up to that peer to realize this, enable it for you and try again. This will not necessarily occur in reverse as they may accept encryption while never initiating it (thus leading to the behavior you have experienced).

This looks like BINGO !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

shit !!!! ... it seems like FAKE 'BINGO' ... when I double check the setup ... 'Use ciphered protocol' is ON [to avoid ISP throttling] ... BUT ... 'Allow UN-CIPHERED connections' is also ON ... ... so it won't be this ...

Maximum connection and upload slot limits. These will be reached easily if set too low, and can result on missed uploading opportunities. With FiOS, you can afford to be generous, so do

I don't know what FiOS stands for ... I will assume you mean 'fiber' ...

'Max connections per torrent: 220'
'Global limit of connections: 1900' [for all torrents]
'DSCP value for IP packages: 0'

'Max number of connection configurations: 500' ... the tooltip says 'max number of OUTGOING simultaneous connections for KTorrent'

The upload and download speed limits are both set to pretty high ...

'Queue Manager' is OFF.

'Upload slot number: 24'

The remaining parameters seem '''reasonable''' enough ...

Maximum connection attempts per second and half-open connections limit. These do not apply to incoming traffic, but should still be tuned.

I CAN'T find anything similar to these on my configuration parameters ...




Summary:

- I will switch OFF the 'DMZ function'

- I will WAIT to configure any kind of static MAC leasing for my laptop [neither modify the DHCP range]

- I will FOCUS on downloading on to my main SSD drive [I already always do it that way ... but as soon as the download finishes, I then move everything to an external HDD , check data integrity and start seeding]. So, from now on, as soon as 'something' interesting comes up on HD-T ... I will download it ... and I will KEEP it on the main drive ... Then several days analizing the behaviour of these new items ...



Right now ... everything points to the external HDDs. Maybe ... maybe ... a leecher tries to connect, the external HDD is somewhat on 'SLEEP mode' ... and after several micro-seconds the leecher interprets some kind of 'TIMEOUT' ... so it doesn't connect. On the other hand, if ... by any chance the external HDD is in 'ACTIVE mode', it will then behaves as normaly 'connectable' ... so this might be the complex casuistry that we are facing ...




'Anon' ... thank you very, very much again ...