As I round out my Rust BitTorrent implementation, here are some thoughts on its design and the ways in which Rust has helped guide it.
The Beauty - And Frustration - Of Coding in Rust
The borrow checker, lifetimes, the limitations on when you can implement traits – these are all heavy cudgels compared to the benefits you get from them, often requiring much cajoling with the compiler to get code to work. And of the actions that they limit – particularly the borrow checker – most of them you can easily reason about the safety of, and act to prevent it yourself. I’d say that, at least in this project, they have been more of a hindrance than help. But I’m interested in seeing concrete examples of where they have worked.
The metaprogramming and type system, though, bringing high-level concepts to low-level tasks, are a thing of beauty. I experienced their possibilities mainly through the Rocket web framework and the Serde serialization library, which use traits and the Rust compiler to accomplish some pretty cool control inversion[1]. Not knowing much about PL theory, I don’t want to comment extensively on this or attempt to compare it to other languages with extensive type systems. But the possibilities with traits are vast.
On a more modest note, it took some time into the process realize I could even create something like the to_keys macro to generically convert objects to bencoded dicts. I implemented the bencoder/decoder early in the process, and they probably would be better off built off with Serde or more compiler-level integration,rather than the clumsy matches and dictionary-converting macros I created.
Concurrency
It seems that one of the key motivations for the limitations on borrowing is in enforcing safe concurrency practices. Indeed, they do, along with the Send and Sync traits, force you to consider concurrency explicitly – but since you can grant those traits to any object with Arcs and Mutex, they don’t hinder you as long as you are willing stick with safe, locking designs.
The model I chose is based on message-passing between several categories of tasks: per-connection tasks, a controller task that synchronizes writing, and a manager task (on the user’s thread) that controls adding new torrents. The controller and per-connection task serve alternatively as producers and consumers, sending and writings messages from peers. Also involved is a futures-based tracker client.
There is very likely room for improvement here: The per-connection threads should likely be made into a thread pool, there are probably unnecessary copies, and the ad-hoc sleep intervals in the controller could be optimized, or perhaps eliminated altogether. Maybe the whole task-oriented system would be better based on futures and Tokio. However, I decided after struggling a bit on the (fairly simple) task of creating the tracker client that they were more trouble than they were forth, particularly in dealing with convoluted futures return types.
In any case, any further improvement will depend on benchmark results: download/upload speeds, CPU time, memory, and perhaps tests of effects of protocol concepts like endgame performance, snubbing, or network properties like connectivity.
[1]: Perhaps a comparison a standard competitor like Spring would be worth doing