# About
{% hint style="warning" %}
Currently, Earendil is in **pre-alpha**. Not all features are ready, and some documentation pages document in-development, unreleased features.
{% endhint %}
[**Earendil**](https://earendil.network) is a decentralized, censorship-resistant, and incentive-compatible communication and value transfer network. It allows any two users of the Earendil network to communicate and transact freely, even against powerful state-level attackers.
With Earendil, you can:
* Build anonymity-protecting apps and peer-to-peer networks that cannot be taken down
* Browse normal Internet websites anonymously, hiding your IP address and location
* Earn fees by running Earendil infrastructure nodes
* Send Mel-based cryptocurrencies off-chain at very low cost
## Why Earendil?
At first sight, Earendil seems similar to existing peer-to-peer onion routing networks like I2P or mixnets like Nym. But it has several distinguishing features:
### Robust ban resistance
Earendil resists both [type-I](https://nullchinchilla.me/2023/05/two-kinds-of-censorship-resistance/) censorship (filtering content or users within the network, or **filter resistance**) and [type-II](https://nullchinchilla.me/2023/05/two-kinds-of-censorship-resistance/) censorship (blocking access to Earendil entirely, or **ban resistance**).
Strong ban resistance is rare in other projects. Even when present, it's generally limited to special-case defenses (e.g. Tor obfuscated bridges) against nation-state firewalls like the Great Firewall of China.
On the other hand, Earendil is designed to work *even* *if the GFW were deployed worldwide*. It makes no assumptions as to most of the network existing in the "free world".
### Decentralized, sybil-resistant incentives
Earendil **optionally** allows every node to set a price that their peers must pay to consume its resources, through MEL-settled cryptocurrency micropayments.
Micropayments elegantly solve sybil-resistance (preventing bad nodes from flooding the network), incentives for honest nodes, and DoS resistance. Nodes behaving badly will not be paid by their peers, while honest nodes are incentivized to compete in a free market to provide the best service to their clients. Bad actors attempting to spam Earendil must pay the network accordingly.
This has important advantages over other incentive/sybil-resistance mechanisms, such as "tax/subsidy" models based on paying fees into a central smart contract and proving contributions to the network to withdraw from it. This is explained further in this [blog post](https://nullchinchilla.me/2023/07/earendil-incentives/).
## How does Earendil work?
Earendil's architecture, in short, is *a mix network overlaid onto a ban-resistant "internet"*. This means a two-part design:
### The "ban-resistant internet" part
We overlay a peer-to-peer packet routing network onto the Internet, and this network itself works very much like the Internet --- packets have a destination, and **relays** relay packets hop by hop closer to their destination.
This layer hides from hostile ISPs and achieve ban resistance by combining two features:
* **Link-by-link obfuscation**: Earendil traffic is, by default, difficult to distinguish from "normal" network traffic. Furthermore, the protocol used for any particular node-to-node link can be switched out, using a "pluggable transport" architecture similar to those used for Tor bridges, for particular severe network environments (e.g. networks that only allow plaintext HTTP and man-in-the-middle all HTTPS traffic)
* **Invite-only, limited-information architecture**: Earendil routes traffic using a unique, invite-only [architecture](/wiki/architecture) that only reveals the IP address of the immediate peers of every participant, making it difficult for even powerful attackers to compile a list of Earendil nodes' IP addresses for surveillance or censorship. Censors blocking all nodes that they know of is likely to only block themselves from the network.
This layer *does not provide anonymity*, only reliability and ban resistance. We also do all incentives in this layer through a simple mechanism where users directly pay peers for all resources used on them.
### The "mix network" part
On top of this peer-to-peer network, we overlay a *mix network* similar in design to Nym, providing strong anonymity through onion encryption and delays. All relays must also participate in the mix network to be part of the Earendil network.
The mix network is designed to maintain anonymity even with the attacker completely monitoring all traffic on the underlying network. This prevents any information about who's communicating with whom from leaking no matter how severely the obfuscation protocols or the incentive payment system leaks information.
## Development status and roadmap
Currently, Earendil is in **pre-alpha**. You can join the Earendil network and communicate over it, but some important features are rudimentary or incomplete:
| Feature | Completion status | Notes |
| ------------------------------------------ | ---------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| Joining as a relay | :white\_check\_mark: Done | |
| Joining as a client | :white\_check\_mark: Done | |
| Onion routing | 🚧 Rudimentary | *No ability to customize route selection.* |
| Havens (anon hosting) | :white\_check\_mark: | |
| Web proxying | 🚧 Rudimentary | *You can host web proxies to help other users use the Web through Earendil. But no user authentication or access control is implemented.* |
| Debt accounting | :white\_check\_mark: | *Calculates how much to pay neighbors for their resources.* |
| Manual debt settlement | :white\_check\_mark: | *Allows settling the calculated debt out-of-band and manually resetting it in the protocol.* |
| Mixnet delays | 🚧 Rudimentary | *Delays messages to defend anonymity against large-scale attackers* |
| Auto debt settlement (MVP) | 🚧 Rudimentary | *Automatically settle MEL-denominated debt using on-chain transfers* |
| Mel-backed Sybil resistance |
❌ Not implemented (planned for 0.6.x)
| *Limit the number of relays by requiring staking assets on the Mel blockchain* |
| Auto debt settlement with payment channels |
❌ Not implemented (planned for 0.6.x)
| *Settle debt off-chain using anonymous payment channels. Earendil can then be used as an off-chain asset transfer layer.* |
# Installation
## System requirements
* For client nodes:
* At least 1 GB of free RAM and disk space, to compile the program
* Windows 10, macOS, or Linux
* For relay nodes:
* A public IP address to serve clients. Generally, you'll find this on cloud servers, VPSes, dedicated servers, etc.
* At least 1 GB of free RAM and disk space.
* Only Linux is tested, though any platform that runs Rust is likely to work
## GUI
### Windows and Mac
1. Download the latest executable for *your platform* from our [releases page](https://github.com/mel-project/earendil/releases).
2. Decompress the executable file you just downloaded.
* On Windows, you can do this by right-clicking the file and selecting "Extract All...", while on macOS, you can simply double-click the file.
3. Start the program by double-clicking on the executable file!
{% hint style="info" %}
If your executable isn't working, first make sure that you downloaded the correct file for your platform! If it still doesn't work, come to our [Discord](https://discord.gg/AVsGbhzTzx) to ask for help.
{% endhint %}
### Linux
You need an up-to-date [Rust](https://www.rust-lang.org/tools/install) installation, with tools like `cargo` and `rustup` in your $PATH. Earendil currently has no official binary distribution, so we'll be compiling it from source.
```
cargo install earendil-gui
```
## Command line
In a terminal, install `earendil` by typing:
```shell-session
rustup update # to make sure your Rust is up to date
```
```shell-session
cargo install --locked earendil
```
Check that `earendil` is successfully installed by typing:
```shell-session
earendil
```
You should see:
```shell-session
Usage: earendil
Commands:
daemon Runs an Earendil daemon
control Runs a control-protocol verb
generate-seed
help Print this message or the help of the given subcommand(s)
Options:
-h, --help Print help
-V, --version Print version
```
# Quick start
Proxying Web traffic over Earendil
In this tutorial, we'll show you how to proxy web traffic through the Earendil network using the `earendil` GUI.
If you haven't already, **install the GUI** from the [Installation](/getting-started/installation) section.
## Configure + start Earendil
All of `earendil`'s configuration options are specified in a YAML config file, located in the "Settings" tab of the GUI.
Paste this config file into your Settings tab:
```yaml
# relays to connect to
out_routes:
# arbitrary name for this relay
example-relay-free:
# IP and port where the relay is listening
connect: 62.210.93.59:12345
# relay's long-term identity
fingerprint: 4b7a641b77c2d6ceb8b3fecec2b2978dfe81ae045ed9a25ed78b828009c4967a
# obfuscation protocol to use
obfs:
# obfuscation secret, generated and provided by the relay
sosistab3: randomly-generated-cookie-lala-doodoo
```
By default, Earendil exposes a Socks5 proxy server on `localhost:30003` that will proxy traffic through Earendil.
Now, start the daemon by pressing "Start" in the bottom tray. Go to the "Logs" tab, and you should see `earendil`'s logs appearing.
You're all connected on the Earendil side!
## Connect your browser
To browse the web through Earendil, we need to configure a web browser to use our Earendil SOCKS5 proxy (at `localhost:30003`). For Firefox this looks like:
Visit any website as you normally would, except now all your traffic is going through Earendil! You can confirm this by [checking](https://bgp.he.net/) your IP address: you're properly connected if it's `62.210.93.59`. That's the IP address of this tutorial's bootstrap node.
Now you can use Earendil as a web proxy to browse the internet anonymously! Feel free to share [speed tests](https://speed.cloudflare.com/) in our [Discord](https://discord.gg/AVsGbhzTzx).
{% hint style="warning" %}
**Obtaining relay information safely**
In the configuration above, we added a *publicly available* example relay that the Mel team maintains.
It is important to note that in production, *Earendil relay information will not generally be publicly available*. You will need to personally know a relay operator to obtain contact information out-of-band, through chat, email, or offline.
This is to ensure **ban-resistance**: if any client can just request relay information, attackers can simply join the network to get a list of relays, which can let them block or identify Earendil traffic.
Thus, if you want to actually ensure ban-resistance, don't use the relay we gave you above! Instead, you can come to [our Discord](https://discord.gg/jdVuk4Qj89) to ask other users for help.
{% endhint %}
# Host a web proxy
Both client and relay Earendil nodes can host web proxies. When you host an Earendil web proxy, clients that you choose to share the proxy information with can use your node as the exit node to anonymously browse clearnet traffic.
To host a web proxy, paste this config file into the "Settings" tab of your Earendil GUI:
```yaml
out_routes:
example-relay-free:
connect: 62.210.93.59:12345
fingerprint: 4b7a641b77c2d6ceb8b3fecec2b2978dfe81ae045ed9a25ed78b828009c4967a
obfs:
sosistab3: "randomly-generated-cookie-lala-doodoo"
price_config:
inbound_price: 0
inbound_debt_limit: 0
outbound_max_price: 0
outbound_min_debt_limit: 0
exit_config: # you want to be an exit node
allowed_ports: # destination ports that your exit node will allow; to only allow browsing http and https websites through your proxy, only put 80 and 443 here
- 80
- 443
```
Note that hosting a web proxy necessarily exposes your IP address: a client can always connect to your proxy and go to an IP-checking website.
Start Earendil. Now clients will be able to automatically use your node as their web proxy!
# Visit and host havens
**Havens** are anonymously hosted services, similar to [onion services in Tor](https://community.torproject.org/onion-services/). By hosting a haven, you can serve a TCP service, like a website, IRC server, or similar. Both you and your users will then be protected by Earendil's anonymity and censorship-resistance guarantees.
This tutorial will teach you how to use and host basic Earendil havens.
## Visit havens
You can access HTTP-based havens right in your browser. Paste the following config file into the "Settings" tab of your Earendil GUI:
```yaml
out_routes:
example-relay-free:
connect: 62.210.93.59:12345
fingerprint: 4b7a641b77c2d6ceb8b3fecec2b2978dfe81ae045ed9a25ed78b828009c4967a
obfs:
sosistab3: "randomly-generated-cookie-lala-doodoo"
price_config:
inbound_price: 0
inbound_debt_limit: 0
outbound_max_price: 0
outbound_min_debt_limit: 0
```
Then, set your browser to use `localhost:30003` as a SOCKS5 proxy. For Firefox this looks like:
Try visiting
```!
http://t90bt94h01ezd75zv9rtzam60thnbkvz.haven:12345
```
like you would any ordinary website. You should be greeted with:
You just visited your first Earendil haven! With this setup, you can visit any Earendil haven you know the address to.
{% hint style="info" %}
All Earendil haven websites are HTTP only, since certificate authorities generally do not issue certificates to `.haven` domains. HTTPS is unnecessary because Earendil traffic is already encrypted.
{% endhint %}
## Host havens
As an introduction to hosting havens, let's host a website as a haven.
### Start a localhost web server
First, **set up a web server that listens on port 8000**. For our example, we'll use Nginx.
1. Install Nginx if it's not already installed.
2. In the nginx config file (most likely located at `/etc/nginx/nginx.conf`), look for a section that configures a server listening on 8000, and change that to the following:
```
server {
listen 8000;
server_name localhost;
location / {
root /usr/share/nginx/html;
index index.html;
}
}
```
3. Start your Nginx server. On Linux: `systemctl start nginx`
4. You should now be able to see your server on `localhost:8000`!
### Set up the haven
Paste this config file into your Earendil GUI's "Settings" tab:
```yaml
out_routes:
example-relay-free:
connect: 62.210.93.59:12345
fingerprint: 4b7a641b77c2d6ceb8b3fecec2b2978dfe81ae045ed9a25ed78b828009c4967a
obfs:
sosistab3: "randomly-generated-cookie-lala-doodoo"
price_config:
inbound_price: 0
inbound_debt_limit: 0
outbound_max_price: 0
outbound_min_debt_limit: 0
# havens we're hosting
havens:
- identity_file: /your/path/identity.secret # replace with a writable path for storing this haven's identity secret
listen_port: 12345
rendezvous: 4b7a641b77c2d6ceb8b3fecec2b2978dfe81ae045ed9a25ed78b828009c4967a # relay chosen as our rendezvous point for onion-routing
handler:
type: tcp_service
upstream: 127.0.0.1:8000 # where web server is listening
```
* `identity_file`: a writable path for storing your haven's identity secret
* `rendezvous` is the fingerprint of your chosen *rendezvous relay*. This is a relay node that is responsible for receiving and forwarding all the messages meant for your haven, so that your IP address can be kept private from clients of your haven. All havens must have a rendezvous relay; you can read more about the haven protocol's architecture [here](https://docs.earendil.network/wiki/protocols/haven-protocol). For this example, we’ll use the same test relay that we bootstrapped with throughout this tutorial.
* `handler` specifies how to handle traffic to the haven. Here, we use TCP [port forwarding](https://en.wikipedia.org/wiki/Port_forwarding) to forward all haven traffic to the web server on port 8000.
Start Earendil, and find your haven's address in the "Dashboard" tab:
{% hint style="info" %}
In the CLI version, you can get your haven's address with
```shell-session
earendil control havens-info
```
You should see something like:
```
TcpForward - qcmnt2mbchhanm7fzacybswzknbsw3zp:12345
```
{% endhint %}
In our example, `qcmnt2mbchhanm7fzacybswzknbsw3zp` is your haven's **fingerprint**, and `12345` is its **dock** number (analogous to TCP port number).
People can now find your haven at `http://.haven:`!
# Chat
Earendil has a minimal embedded chat tool for talking to your **immediate neighbors** in the Earendil network. You can use it to negotiate connection prices with your neighbors.
## GUI
In the "Chat" tab of the Earendil GUI, you can chat with any of your direct neighbors by selecting their fingerprint in the menu on the left side of the screen:
## CLI
To chat on the command line:
```!bash
earendil control [--connect control_listen_addr] chat
```
where `` is one of:
* `list` - prints a summary of all your conversations
* `start ` - starts an interactive chat session with a neighbor whose fingerprint starts with ``. This will fail if `` matches more than one neighbor (in that case, use a longer prefix).
Some examples:
```!bash
$ earendil control chat start zpy
<- hey alice! [2024-01-10 19:59:55]
-> hey bob! [2024-01-10 19:59:58]
<- how's the weather? [2024-01-10 20:00:04]
sunny!
-> sunny! [2024-01-10 20:01:15]
```
```!bash
$ earendil control chat list
+------------------------------------+-------------------+-----------------------------------+
| Neighbor | # of Messages | Last chat |
+------------------------------------+-------------------+-----------------------------------+
| 4b7a641b77c2d6ceb8b3fecec2b2978... | 4 | sunny! [2024-01-10 15:29:05]
+------------------------------------+-------------------+-----------------------------------+
```
# Run a relay
We currently only support running relays using the CLI version. Relays should be run on machines with public IP addresses.
Relays and clients nodes use the same `earendil` executable. The defining difference is in their config file: relay configs have a `relay_config` section that specifies `identity_file` (to store the relay's long-term identity) and `in_routes` (where and how to accept incoming connections), while client configs do not.
To run a relay, save this config file into a file named `relay-cfg.yaml`. Be sure to replace "/your/path/\` with an appropriate path:
```yaml
# neighbors, same as in client config
out_routes:
example-relay-free:
connect: 62.210.93.59:12345
fingerprint: 4b7a641b77c2d6ceb8b3fecec2b2978dfe81ae045ed9a25ed78b828009c4967a
obfs:
sosistab3: "randomly-generated-cookie-lala-doodoo"
price_config:
inbound_price: 0
inbound_debt_limit: 0
outbound_max_price: 0
outbound_min_debt_limit: 0
# relay-only settings
relay_config:
# replace with a writable path for storing identity secret
identity_file: /your/path/earendil-relay-id.secret
in_routes:
main_udp:
obfs:
# random seed for obfsudp cookie. Generate your own with `earendil generate-seed`
sosistab3:
# port where this in-route listens
listen: 0.0.0.0:19999
# price, debt limit etc. for this in-route
price_config:
inbound_price: 0
inbound_debt_limit: 0
outbound_max_price: 0
outbound_min_debt_limit: 0
```
You can learn about paying and getting paid on the Earendil network, as well as the `price_config` [here](/getting-started/pay).
Start the `earendil` daemon using this relay config:
```
earendil daemon --config relay-cfg.yaml
```
While the `earendil` daemon is running, obtain your relay's contact information for other nodes to connect with you as a neighbor with the control command `my-routes`:
```shell-session
earendil control my-routes
```
The output should look like:
```yaml
main_udp:
connect: :19999
fingerprint: 57a407e50c1f4d0cdfb16332f6a836b27cc3409941fa26d85bc2b1eca604e536
obfs:
sosistab3:
price_config:
inbound_price: 0
inbound_debt_limit: 0
outbound_max_price: 0
outbound_min_debt_limit: 0
```
Replace `` with your server's public IP address. Other nodes (both clients and relays) can simply paste this block into the `out_routes` section of their config file to add your relay as a neighbor.
{% hint style="warning" %}
To serve users in regions with internet censorship, you should *avoid* posting your relay's contact information publicly. Instead, distribute it in a way that reaches legitimate users but not censors--your relay will be blacklisted if the censor learns its IP address!
{% endhint %}
# Pay and get paid
In the Earendil network, nodes pay and get paid by their *immediate neighbors*.
This creates a free market for bandwidth --- if one neighbor is too expensive or unreliable, simply disconnect from them and find a cheaper or more reliable provider. Once you pay your neighbor, it's their responsibility to route your packets to their destinations. This is just like using the internet: you pay your ISP (like T-Mobile) and don't worry about the rest.
To learn more about Earendil's incentive system, read [this post](https://nullchinchilla.me/2023/07/earendil-incentives/).
## Price & debt limit
Two neighbors agree **out-of-band** on a price and debt limit for sending packets when they first connect to each other. They then specify this information in the `price_config` section of the `in_route` or `out_route` block:
```yaml
# every in_route and out_route has a price_config
price_config:
# how much you charge per incoming packet, in µMEL
inbound_price: 5
# debt limit for inbound packets, in µMEL
inbound_debt_limit: 50000
# max price you're willing to pay per outgoing packet, in µMEL
# this field prevents your neighbor from charging you more than the agreed amount
outbound_max_price: 10
# min debt limit you accept for outbound packets, in µMEL
# negative debt limit means prepayment is required
# this field prevents your neighbor from charging you a prepayment larger than the agreed amount
outbound_min_debt_limit: -100
```
As an example, say we have relay Alice and client Bob. Alice has this `price_config` in her in\_route for Bob:
```yaml
price_config:
inbound_price: 1
inbound_debt_limit: 5000
outbound_max_price: 0
outbound_min_debt_limit: 0
```
While Bob has this `price_config` in his out\_route for connecting to Alice:
```yaml
price_config:
inbound_price: 0
inbound_debt_limit: 0
outbound_max_price: 10
outbound_min_debt_limit: -100
```
This means Alice charges 1 µMEL for every packet Bob sends her, Bob can owe alice at most 5,000 µMELs before she disconnects from him, and Alice does not pay Bob anything to send packets to him.
## Payment methods
A node specifies all the payment methods they support in the `payment_methods` section of their config file. If two neighbors don't share any supported payment methods in common, they won't be able to connect (unless they both charge a price of 0).
We currently support 2 payment methods: on-chain payments on the Mel blockchain and proof-of-work (PoW).
```yaml
payment_methods:
- on_chain:
# no arguments required to support PoW payments
- pow
```
The `secret` field for `on_chain` is the secret key of the Mel wallet you'll use to send and receive payments. [Here's](https://docs.melproject.org/developer-guides/using-wallets) how to set up a Mel wallet. You can export the secret key from an existing wallet using:
```bash
melwallet-cli --wallet-path export-sk
```
As an example, this config means that your node only accepts on-chain payments:
```yaml
payment_methods:
- on_chain:
```
## Testing payments
The default bootstrap node we've been using throughout the tutorials is entirely free. To test payments, use this node that supports both on-chain and PoW payments:
```yaml
example-relay-paid:
connect: 62.210.93.59:19999
fingerprint: 14154070117b3c1a71fa2fc6bc7d20e5afc93fbe98a13b86b013d0a91215f74f
obfs:
sosistab3: correct-horse-battery-pink-staple-pasta-apple
price_config:
inbound_debt_limit: 0.0
inbound_price: 0.0 # do not set a price > 0; our example payments server will not pay!
outbound_max_price: 1
outbound_min_debt_limit: 0.0
# don't forget to specify payment_methods you support
payment_methods:
- pow
```
# Config file
Here's a fully annotated config file:
```yaml
# [optional] Path to the database; must be writable by the `earendil` daemon.
# If this key is not specified, `earendil` uses a default path.
# If you start more than one `earendil` daemon on the same machine, you must specify this field in all but one of the daemon configs.
# This prevents additional daemons from trying to use the same database and crashing.
db_path: ./.cache/earendil
# [optional] IP address where the daemon listens for control commands.
# If this key is not specified, `earendil` listens for control commands on a default port.
# If you start more than one `earendil` daemon on the same machine, you must specify this field in all but one of the daemon configs.
# This prevents additional daemons from trying to listen on the same port and crashing.
# Currently sending control commands to remote daemons is not supported, so this should be `127.0.0.1:`.
control_listen: 127.0.0.1:11111
# ------------------------ routing config ----------------------------
# relays to connect to as neighbors. Client configs *must* contain at least one `out_route`; optional for relays.
out_routes:
example-relay:
# IP address and port where the relay is listening for incoming connections
connect: 62.210.93.59:12345
# long-term identity of the relay
fingerprint: 4b7a641b77c2d6ceb8b3fecec2b2978dfe81ae045ed9a25ed78b828009c4967a
# obfuscation protocol to use, for resisting ISP-level censorship
# There are currently 2 obfuscation options:
# - `none`: no obfuscation. In a censored network environment, this may lead to your connection to this relay getting blocked.
# - `sosistab3`: a TCP-based obfuscated transport with a symmetric cookie, defined by the relay. This obfuscation protocol is developed as a part of [geph5](https://github.com/geph-official/geph5)
obfs:
sosistab3: shove-mistake-wish-endless-antique-citizen-filter-employ-cigar-clip-acid-defense
# price and debt config for this link
price_config:
# how much you charge per incoming packet, in µMELs
inbound_price: 0
# debt limit for inbound packets, in µMELs
inbound_debt_limit: 0
# max price you're willing to pay per outgoing packet, in µMELs
outbound_max_price: 10
# min debt limit you accept for outbound packets, in µMELs
# Negative debt limit means prepayment is required
outbound_min_debt_limit: -100
# more relays to connect to
relay-2:
connect: ...
fingerprint: ...
obfs: none
price_config: ...
# -------------------- payments + Mel blockchain access -----------------------
payment_methods:
# no arguments required to support PoW payments
- pow
- on_chain: # secret of melwallet to use for sending + receiving payments
# [optional] how to connect to the Mel blockchain, can be Earendil haven address
# If this key is not specified, then we connect to the Mel blockchain using the default bootstrap node over clearnet. This may not work in countries with internet censorship.
mel_bootstrap:
# --------------------------- havens + proxy ---------------------------
# [optional] Starts a local Socks5 server that proxies traffic through Earendil.
# This gives you access to Earendil havens.
# If this key is not specified, `earendil` starts a Socks5 proxy on **port 30003** with `fallback: pass_through`.
# If you start more than one `earendil` daemon on the same machine, you must specify this field in all but one of the daemon configs.
# This prevents additional daemons from trying to listen on the same port and crashing.
socks5:
# localhost address where the earendil Socks5 proxy listens
listen: 127.0.0.1:23456
# how to handle non-Earendil traffic (like a request to tunnel `google.com:443`). There are 3 options:
# 1) `pass_through`: let all non-Earendil traffic through as if you're not using Earendil. Requests to `google.com` will behave the same way as if you weren't connected to the Earendil proxy.
# 2) `block`: block all non-Earendil traffic. Requests to `google.com` will fail.
# 3) `simple_proxy`: proxy non-Earendil traffic via a specified exit node, similar to how you use Tor as a web proxy.
fallback:
simple_proxy: passthrough
# havens we're hosting
havens:
## a TCP haven, e.g. a website
# path to file storing long-term haven identity. Must be writable to earendil daemon
- identity_file: /your/path/identity.secret
# fingerprint of relay chosen as rendezvous point for this haven (keeps haven anonymous to visitors)
rendezvous: 4b7a641b77c2d6ceb8b3fecec2b2978dfe81ae045ed9a25ed78b828009c4967a
handler:
type: tcp_service
# dock where this TCP haven is hosted
listen_dock: 12345
# TCP address to forward all traffic for this haven to. The TCP service behind this haven (e.g., website) should be listening to this address.
upstream: 127.0.0.1:8000
## a web proxy haven
# path to file storing long-term haven identity. Must be writable to earendil daemon
- identity_file:
/your/path/identity.secret
# relay chosen as our rendezvous point. Web proxy havens cannot be anonymous, so this relay should be chosen to optimize performance.
rendezvous: 4b7a641b77c2d6ceb8b3fecec2b2978dfe81ae045ed9a25ed78b828009c4967a
handler:
# Earendil's web proxy protocol
type: simple_proxy
# dock where proxy server listens
listen_dock: 19999
# -------------------------- relay-only ----------------------------
relay_config:
# path to file for storing a long-term identity.
identity_file: /your/path/identity.secret
# where & how to accept incoming connections
in_routes:
main_obfs:
# obfuscation protocol to use, for resisting ISP-level censorship
obfs:
sosistab3: snake-before-antenna-toward-floor-stuff-frozen-power-avocado-retire-grunt-nation
# TCP port this in_route listens at
listen: 0.0.0.0:19999
# price config for this route, in µMELs
price_config:
inbound_price: 5
inbound_debt_limit: 50000
outbound_max_price: 0
outbound_min_debt_limit: 0
# another in_route, with no obfuscation
no_obfs:
obfs: none
listen: 0.0.0.0:19998
price_config:
inbound_price: 3
inbound_debt_limit: 30000
outbound_max_price: 0
outbound_min_debt_limit: 0
```
# 1+ nodes on 1 machine
# Type-I and II censorship resistance
We can distinguish between two types of censorship resistance:
* **Type-I** or *filter resistance* prevents discrimination between different users or different data use cases within the same protocol, by the infrastructure that operates the protocol (e.g. Ethereum RPC nodes)
* **Type-II** or *ban resistance* prevents the protocol itself from being banned by powerful adversaries that control *underlying* infrastructure (e.g. ISPs, nation states)
Many crypto\* projects have pretty strong Type-I censorship resistance, such as Tor, privacy coins, etc.
Some communication projects (like Tor pluggable transport) try to achieve Type-II censorship resistance at the cost of massive centralization that makes Type-I censorship resistance much harder.
Few projects try to achieve both; Earendil is an exception.
[See more here.](https://nullchinchilla.me/2023/05/two-kinds-of-censorship-resistance/)
# Network architecture
## Nodes
There are two kinds of nodes in the network:
* **Relays** form the backbone of the Earendil network, and relay messages between their **neighbors**: nodes that are directly connected to this relay.
* **Clients** do not relay any traffic, and they access the network with the help of relays. None of their neighbors can be other clients.
Earendil operates on an *invite-only* model: to bootstrap into the network, every node (relay and client) must *manually* configure at least one node as a neighbor. This means obtaining their **route secret**: a secret document that describes how to reach that node.
The secrecy of route secrets is critical to the type-II censorship resistance of the network (though not its confidentiality or its type-I censorship resistance), so nodes in censored regions would generally be very tight-lipped about these secrets except to trusted friends.
Nodes in the "free world" may publicly list their route secrets (e.g. on some website) to help other fellow "free world" netizens bootstrap, but it must be noted that these netizens' ISPs will easily tell that they are using Earendil as a result.
## Routing
### Public relay graph
Up to now, this sounds like a typical friend-to-friend or "darknet", similar to a proposed model for the Freenet project, where every node in a peer-to-peer network only knows its neighbors and is forbidden access to information about further nodes.
However, Earendil departs from this model in that all nodes do know the **relay graph** of the network: what relays there are (identified by public key fingerprint) and what relays are neighbors to which relays, even though the complete "contact information" contained in the route secrets is kept secret.
This is done by continually gossipping through the network **adjacency descriptors**: documents signed by both neighbors of a connection that contain their unique fingerprints, but no other route information.
### Packet-based onion routing
{% hint style="warning" %}
The current version of Earendil has switched to an **overlay** design, where the "peelers" of the onion routing are arbitrarily picked from the entire relay graph and decoupled from the actual path the packet takes between these peelers.
See the [design document](https://forum.melproject.org/t/decoupling-routing-from-onion-encryption-in-earendil/98).
{% endhint %}
With the entire relay topology, an arbitrary *client* can send a message to an arbitrary *relay* with **onion routing**. The basic procedure is analogous to other onion routing systems:
* Client finds a path through the relay graph, starting at itself and ending at the relay. This can be done through a shortest-path algorithm or any other heuristic, depending on the client's desired anonymity/performance tradeoff.
* Client produces a nested-encrypted [onion packet](/wiki/onion-packet-format). Given intermediate relays $$R\_1,\dots,R\_n$$, the packet is roughly the fingerprint of $$R\_n$$ attached to the message encrypted to the pubkey of $$R\_n$$, then encrypted to the pubkey of $$R\_{n-1}$$ with the fingerprint of $$R\_{n-1}$$attached, etc.
* Client sends the onion packet to $$R\_1$$. Every relay "peels off" one layer of of encryption, revealing the next hop, to whom they send the remaining layers.
* Eventually, $$R\_n$$ gets the fully "peeled off" message: a message encrypted to its pubkey, which it then decrypts and reads.
This has the nice property that no relay, including the destination, knows the entire path. Given paths picked from a high-entropy enough probability distribution, this provides strong, Tor/I2P-like anonymity to the user (modulo timing and other side-channels)
### Relay->client packets
The above method only works for unidirectional communication from a client to a relay. This is not enough for a useful network.
Fortunately, as long as the client contacts the relay first, relays can send messages back to clients while preserving the client's anonymity. This uses [reply blocks](/wiki/onion-packet-format/reply-blocks).
### Client->client packets
Client-to-client communication uses [**rendezvous routing**](/wiki/protocols/haven-protocol). In short, if Alice and Bob want to communicate, this procedure is followed (WLOG we assume that Alice is sending a packet to Bob rather than the other way around):
* (This step only needs to be done infrequently, as a "setup") Bob picks a relay, Rob, in the network to act as his **rendezvous relay**.
* Bob arranges with Rob so that when Rob receives messages with a particular unique tag, they are forwarded to Bob, using the usual relay->client reply blocks.
* Bob publishes a signed binding into a network-wide **rendezvous DHT**, binding his fingerprint to a **rendezvous descriptor**, containing Rob's fingerprint and the Bob-specific unique tag.
* When Alice needs to send a packet to Bob:
* She looks up Bob's fingerprint in the rendezvous DHT, getting Rob's fingerprint
* She sends usual client->relay packets to Rob tagged with Bob's unique tag
* Rob forwards the content to Bob using the relay->client system to Bob
Bob's choice of his "Rob" depends on his anonymity needs. If he just wants to optimize for performance, simply picking his fastest neighbor works. But if he wants to optimize for anonymity, he should pick a relay from the whole network uniformly, without taking into account his position in the network.
# Protocols
Earendil presents a pretty simple interface (basically a virtual IPv6-like network device with some convenience methods), but under the hood is many layers of protocols.
This is illustrated by the following picture. **Note that A->B means A depends on B**, not that A somehow sends data unidirectionally to B!
{% @mermaid/diagram content="flowchart TB
subgraph network\["Network layer"]
socket\["Unified socket abstraction"]
haven\["Haven protocol"]
dht\["Rendezvous DHT"]
forward\["Haven forwarding"]
rpc\["GlobalRPC"]
mixnet\["Mixnet protocol"]
onion\["Onion routing"]
end
subgraph lownet\["LowNet"]
link\["Link transport"]
end
socket --> haven
haven --> dht & forward
dht --> rpc
forward --> mixnet
rpc --> mixnet
mixnet --> onion
socks\["SOCKS5 interface"] --> sosistab\["Optionally reliable streams (sosistab2)"]
tcp\["TCP port forwarding"] --> sosistab
sosistab --> socket
tun\["tun-based VPN interface"] --> socket
onion --> link & routegraph\[("Route graph")]" %}
# N2R (node-to-relay) protocol
N2R allows communication between any two nodes, *as long as one of them is a relay*.
## Packet format
The packet format is based on the `InnerPacket` Rust enum:
```rust
/// Represents the actual end-to-end packet that is carried in the payloads. Either an application-level message, or a batch of reply blocks.
pub enum InnerPacket {
/// Normal messages
Message(Message),
/// Reply blocks, used to construct relay->anon messages
ReplyBlocks(Vec),
/// An inner packet message with corresponding UDP port-like source and destinaton docks
pub struct Message {
pub source_dock: Dock,
pub dest_dock: Dock,
pub body: Vec,
}
pub type Dock = u32;
```
An `InnerPacket` is stuffed into the 8192-byte payload by this process:
1. First, we encode the packet using [bincode](https://docs.rs/bincode/latest/bincode/), getting `box_plaintext`.
2. We then box-encrypt using an ephemeral keypair, whose public half is `box_epk`, getting `box_ciphertext`.
3. We sign `box_epk` with our identity secret key `identity_sk`, whose public half is `identity_pk`, producing `identity_signature`
4. We then package everything into a tuple `(identity_pk, identity_signature, box_ciphertext)`, bincode it, and pad it to 8192 bytes.
This picture roughly illustrates the structure of a fully encoded `InnerPacket` that stores a normal `Message`.
## Socket abstraction
The typical interface exposed by N2R is not raw functions for sending and receiving packets. Instead, we use a *socket* abstraction inspired by UDP. Each socket represents an `Endpoint`, a *local fingerprint:dock pair,* that can receive and send messages. More specifically:
* The user constructs a socket by **binding** to an identity and a dock number.
* This identity can either be the long-term identity of the node, or a temporary anonymous identity.
* The socket has a method to **send** a message to a fingerprint and dock number.
* This formats a message with:
* source identity public key and source dock number taken from the identity and dock number of the socket
* destination identity looked up by fingerprint
* If the socket is bound to a temporary identity, reply blocks must be sent to the destination fingerprint as well so that they can talk back to us.
* There's also a **receive** method, which returns a message and a source endpoint, which consists of a fingerprint and a dock number.
* This blocks until there is an incoming message addressed to the identity and dock number that the socket is bound to.
* In the implementation, there must be some sort of demultiplexing done to separate incoming messages addressed to different sockets.
# GlobalRPC
RPC protocol that any node can call any node
GlobalRPC is the primary RPC protocol exposed by all relays, that any node can call. It runs over the node-to-relay protocol (N2R).
Like all RPC protocols in Earendil, it is based on JSON-RPC 2.0.
In the following, the node hosting the RPC is called the **server**, while the node calling the RPC is called the **client**. This terminology has no bearing on their overall network roles as relay, client, haven, etc.
## Request/response transport
### Requests
Every GlobalRPC request is encoded verbatim into a N2R messages addressed to **dock 100001**. The source dock number is arbitrary and can be allocated per-request.
For instance, a call `ping(12345)` will simply be a N2R message containing something like the following string:
```json
{
"jsonrpc": "2.0",
"id": 31415926,
"method": "ping",
"params": [12345]
}
```
Some important notes:
* Requests that do not fit into one message are not supported.
* `id` must be unique for each request.
* Clients *may* choose to retransmit requests that have not received a response in a while. The retransmission must contain the same `id` as the original transmission!
* Clients *may* have multiple requests in-flight to the same destination at the same time.
### Responses
Responses are single N2R messages sent to the source fingerprint and dock number that the request comes from. For instance, the response to the above request will be:
```json
{
"jsonrpc": "2.0",
"id": 31415926,
"result": 12345
}
```
Note that:
* Responses that do not fit into one message are not supported.
* If multiple retransmissions of the same request arrive within 60 seconds, they should receive identical responses, while the logic they trigger must not run more than once. To support this, servers must maintain a cache of at least the last 60 seconds, mapping `id` to the response.
* Clients that support multiple inflight requests must use the `id` field to match responses to requests. The server may respond to the inflight requests in any order.
* Clients must silently discard responses that do not correspond to any inflight request.
## Supported methods
{% hint style="info" %}
We are in the process of finalizing the specific methods of GlobalRPC, so this may be updated in the future
{% endhint %}
```rust
pub trait GlobalRpcProtocol {
async fn ping(&self, i: u64) -> u64;
async fn dht_insert(&self, locator: HavenLocator, recurse: bool) -> Result<(), DhtError>;
async fn dht_get(
&self,
key: Fingerprint,
recurse: bool,
) -> Result, DhtError>;
/// Called by a haven server to register itself with a rendezvous node
async fn alloc_forward(&self, forward_req: RegisterHavenReq) -> Result<(), VerifyError>;
}
```
# Haven protocol
## Overview
The **haven rendezvous protocol** allows Alice and Bob to communicate without either of them being a relay or revealing their location in the network topology.
The haven protocol is built upon network-wide client-relay communication and requires first implementing the [GlobalRPC](/wiki/protocols/globalrpc) protocol. It has two main pieces:
* The **rendezvous DHT**, which maps each **haven** (publicly reachable anonymous endpoints with stable, well-known public keys, such as "darknet" websites) to their **rendezvous relay** (relay that handles traffic to and from that haven)
* The **forwarding protocol** that coordinates traffic to flow user<->relay<->haven.
## Rendezvous DHT
The rendezvous DHT maps haven fingerprints to **haven locators.** A haven locator is a document, signed by the haven's public identity key, containing:
* the haven's full public identity key
* the haven's medium-term onion key
* the fingerprint of the rendezvous relay
This is through this process:
* Every relay has RPC methods `get_dht` and `insert_dht` that can retrieve and insert keys stored locally at that relay. These keys expire after a short period of time (e.g. 10 minutes)
* Rendezvous hashing, computed over `(key, hour)`, where `hour` is the number of hours since the Unix epoch, assigns 3 relays to be responsible for a particular DHT key at a given point in time.
* Havens continually insert their own fingerprint->locator mapping into nodes. They insert them not only in the 3 relays of the current hour, but also the previous and next hour.
* When looking up locators, clients simply ask any of the 3 relays that `(key, current_hour)` map to.
{% hint style="info" %}
We use 3 relays rather than 1 for fault tolerance.
Changing the relays every hour is also for fault tolerance, so that keys that are unluckily assigned to bad relays get rotated away from them.
Inserting into three adjacent hours further increases fault-tolerance, as well as mitigating issues related to slightly out-of-sync clocks.
{% endhint %}
## Forwarding protocol
In this example, Bob is a haven, Rob is the rendezvous relay, and Alice is a client who talks to it.
There is one RPC method exposed in the global RPC: `alloc_forward` which takes in a signed request from a particular fingerprint and instructs the relay to process forwarding packets to this particular fingerprint. Bob must call this method before any of the following happens:
* Alice sends N2R messages to Rob to a special, well-known dock (**dock number 100002**). The messages are of the format `(bob_endpoint, inner)`. The inner part is separately end-to-end encrypted between Alice and Bob, using the same packet format as the [N2R protocol](/wiki/protocols/n2r-protocol), but without the padding and onion encryption.
* Rob sends to Bob the messages with the inner part verbatim, tagged as `(inner, bob_endpoint)`.
* Bob sends messages to Alice by sending messages of the format `(inner, alice_endpoint)` to Rob. Rob's internal state is something like a NAT table --- it allows anyone to send messages to Bob, and it allows Bob to send messages to anybody who has talked to Bob, but it refuses to forward any other messages, so it's not an open proxy. Rob identifies "backwards" messages because they are coming from Bob, somebody who has registered forwarding with Rob.
## Socket abstraction
Similar to sockets in the [N2R protocol](/wiki/protocols/n2r-protocol), we have a socket abstraction for haven communication.
Both havens and clients who wish to talk to havens use the same protocol:
* *Bind* which binds to a local dock, but only optionally a *haven host decriptor*.
* Without an descriptor, a random identity is generated. This socket thus is a client socket, only suitable for sending messages to havens and receiving messages from them.
* The descriptor includes info like:
* The identity keypair of the haven (must be unlinkable to the node identity)
* The fingerprint of the appointed rendezvous relay
* *Send* has as its destination a fingerprint and a dock number.
* For client sockets, we assume this fingerprint identifies a haven we want to talk to. We look up the fingerprint in the DHT to find its rendezvous relay, and send it a properly formatted N2R message that will get routed to the haven we want to talk to.
* For haven sockets, we assume that this identifies some endpoint who has talked to our haven in the past. We send the appropriately formatted forward-protocol message to our own rendezvous relay
* *Receive* returns a message, a source fingerprint, and a source dock number.
* For both client and haven sockets, we simply listen for incoming packets over N2R (at the identity and dock number indicated in *Bind*), and decode out the "true" source and contents as forwarding-packet messages.
Haven-protocol (N2H) sockets are easiest to implement simply by containing an N2R socket inside.
## Haven encryption
Haven messages are protected with an additional layer of end-to-end authenticated encryption. Unlike encryption elsewhere in Earendil, this encryption is *stateful* and session-based.
First, note that in havens we can make a distinction between the client and the haven --- havens cannot directly talk to each other. Let's say Alice is the client and Bob is a haven she talks to.
### Initial handshake
Before the first time Alice talks to Bob, Alice generates an ephemeral x25519 keypair, and sends to Bob a message that looks like this:
* 8 bytes: `0xffffffffffffffff` (indicating that this is a handshake packet)
* 32 bytes: Alice's identity public key
* 32 bytes: Alice's ephemeral x25519 public key
* 32 bytes: signature
Bob also generates an ephemeral x25519 keypair, just for communication with Alice. He replies with
* 8 bytes: `0xffffffffffffffff` (indicating that this is a handshake packet)
* 32 bytes: Bob's identity public key
* 32 bytes: Bob's ephemeral x25519 public key
* 32 bytes: signature
After this, Bob and Alice have a shared secret `shared_secret`. We can now derive "upload" and "download" symmetric keys:
* `up_key = keyed_hash(hash("haven-up"), shared_secret)`
* `dn_key = keyed_hash(hash("haven-dn"), shared_secret)`
### After the handshake
Messages are encrypted using ChaCha20-Poly1305, in the following format:
* 8 bytes: incrementing 64-bit nonce, little-endian (pad to 96 bits before using)
* n bytes: ChaCha20-Poly1305 message
Both parties must reject messages with duplicate nonces.
Bob here knows which messages come from Alice by looking at the source fingerprint and dock (tunneled through the rendezvous point).
### Closing sessions
Sessions are never explicitly closed. Instead, their state should be forgotten if not used for more than a reasonable timeout value, at least 10 minutes.
Alice should renegotiate the session at least every 30 minutes, plus if it's been more than 1 minute since she's *heard back from* Bob, to ensure forward secrecy and allow recovery if Bob forgets state.
# Link protocol
The link protocol controls every node-node link in the Earendil network, both relay-relay links and relay-client links.
The basic outline is such:
* Every link is a `picomux` session
* Each end of a link can call "LinkRPC" methods on the other end, by using JSON-RPC over `picomux` streams. These streams have label `"!rpc"`
* Onion-routed messages are passed through a stream with label `""`.
## LinkRPC
Each end of a link separately establishes one or more `sosistab2` streams labelled `"n2n_control"`. Over these streams, JSON-RPC requests are sent in a newline-delimitated manner analogous to JSON-RPC over TCP. For example, here is an example of the send and receive ends of a LinkRPC stream, while one end pings the other:
```json
--> {"jsonrpc":"2.0","method":"ping","params":[123],"id":1}
<-- {"jsonrpc":"2.0","result":123,"id":1}
```
LinkRPC methods are used for actions such as:
* Checking the liveness and performance of a link by "pinging" it
* Gossipping the relay graph
* Negotiating prices and keeping track of debt
We now discuss some of these functions in more detail.
## Relay graph gossip
The purpose of the relay graph gossip is to ensure that between two connected nodes Alice and Bob, they eventually agree on the shape of the relay graph. Since we assume that the network is ultimately globally connected, this means that *everyone*'s view is eventually consistent.
We use a simple pull-based approach. Both ends of every link expose this LinkRPC endpoint:
```
get_adjacencies(fingerprints: string[]) -> AdjacencyDescriptor[]
```
which takes a list of relay fingerprints, and returns all the adjacency descriptors where one end is one of the provided relay fingerprints.
Each side continually calls `get_adjacencies` on the other side, passing in a random subset of fingerprints they know about. The responses are then added to the local relay graph.
This ensures that relay graphs of both of the sides eventually stay in sync.
# Stream protocol
The stream protocol is built on top of the unified socket abstraction, and is used to run reliable, TCP-like communication.
Overall, we steal the same construction as the streams used in `sosistab2`, except without any multiplexing on top of the same fingerprint-dock-fingerprint-dock 4-tuple.
## Taking ownership of sockets
A socket can be passed into either a listener (for the server) or connected to a server (for the client).
## Opening streams
While connecting to the server, the client sends a SYN message to the server endpoint.
The server responds with SYN-ACK, and the connection is now open.
A random stream-ID must be chosen.
## Sending data
Data is sent as usual. The stream-ID must stay the same.
## Closing the connection
When the connection is closed, FIN or RST is sent.
The underlying socket (or forwarding table entry) of each connection must be kept alive for at least 60 seconds, even after the connection is closed. A loop then responds to every incoming packet that is not itself an RST with an RST. This is so that the dock number is kept occupied, and so that the other side realizes that the connection is closed.
# WIP: Debt accounting
{% hint style="warning" %}
This is a **design document** for an *unimplemented* feature.
{% endhint %}
Debt accounting in Earendil has three important parts:
* **Negotiating** the price paid on a link
* **Tracking** the net debt on a link
* **Settling** the debt
This is all done through GlobalRPC verbs.
## Negotiating a price
Before any data packets can be processed, both sides call `push_price` on the other side. This RPC method *pushes* a **price**, in MEL/packet, to the other side, as well as a **debt limit**, returning whether or not it is accepted. It also contains metadata about how the caller wishes to be paid.
Each sides refuses to peel/relay any packets until the other side has signaled acceptance of the price.
`push_price` may be called at any point, with the understanding that the other side may take a while to apply the changes to how debt etc is calculated.
`in_route` and `out_route` in config files specify maximum allowable prices and debt limit ranges.
## Tracking debt
Due to packet loss and delays, we cannot know *exactly* how many packets were received by the other side of the link. Continually doing two-way synchronized communication to come to consensus on the debt level is also impractical.
This means that in the steady state, we *conservatively* estimate the debt by using how much we sent the other side to estimate how much we owe them, and how much we actually received to estimate how much they owe us.
We then continually *correct* this, by both sides periodically (say, every 1 minute) calling `push_debt` on the other side. This method will provide a *signed and timestamped* claim that according to the pusher, the ledger looks a certain way.
Note that at this phase, we do not "net" the debt yet.
## Settling the net debt
At any point, either party can initiate a **settlement**. This is done by calling . This starts a process that reduces debt for both sides.
The object passed to `start_settlement` needs to contain:
* How much debt to reduce for left
* How much debt to reduce for right
* Proof of payment --- in the "net" amount. This can be in either direction!
* A signature
Note that the "proof of payment" is highly payment-method specific. It can also be a absent, in which case `start_settlement` will not automatically return until the other side presses a button the UI
## Manual debt settlement UX
There will be a GUI "node dashboard" that every node operator can access, with a chat interface for chatting with the operator's immediate neighbors. There are LinkRPC methods for pushing a message to the other side.
In each chat conversation, the UX will look something like:
```
+------------------+------------------+------------------+
| Neigbbor | Messages | Debt Status |
| List | | |
|------------------|------------------|------------------|
| Alice | Alice: Hi! | Net Debt: |
| Bob | You: Hello! | $150.00 |
| Charlie | Alice: How are | |
| Dave | you? | |
| Eve | You: I'm good, | |
|------------------| you? | |
| | Alice: I'm fine | [Manual |
| | thanks. | Settlement] |
| | | |
| | | |
| | | |
+------------------+------------------+------------------+
```
Pressing the "manual settlement" button will call `start_settlement` with an empty proof of payment.
The other person will receive a notification to either accept or reject the settlement.
# Onion packet format
The onion-packet has two, fixed-size parts:
* The **header** which includes authenticated-encrypted routing messages
* The **body** which is encrypted with layers of ChaCha20, with intentionally no integrity protection.
```
Total: 8,872 bytes
├─ Header: 680 bytes
│ ├─ Box-encrypted routing info for first hop: 68 bytes
│ └─ Onion-encrypted routing info for next hops: 612 bytes
└─ Onion-encrypted body: 8,192 bytes
```
One important thing to note is that the encoding within the fixed-size body is not specified here, but it *must* have some sort of *end-to-end* integrity protection. The onion packet format will not help with that, at all.
## Box encryption
**Box encryption** (named after the similar construction from NaCl) is a generic way of encrypting a message, with integrity protection, so that only the owner of a particular X25519 secret key can read it.
The format is:
* 32 bytes: a X25519 ephemeral public key from the sender
* ?? bytes: encrypted message, with ChaCha20Poly1305 (RFC 8439), encrypted with the key derived from blake3-hashing the X25519 shared secret computed from the sender's ephemeral key and the X25519 key of the recipient, and an all-zero nonce.
## Header format
The header for a message whose first hop is "Rob" will have:
* 69 bytes: 21 bytes box-encrypted to Rob's X25519 public key. The first byte is a flag for how to interpret the next 20 bytes:
* Byte 1 = 1 indicates we need to forward the message to the next hop. The next 20 bytes is the fingerprint for next hop.
* Byte 1 = 0 indicates the packet is addressed to us.
* If the next 20 bytes are all zeros, it means this packet is an ungarbled normal packet.
* Otherwise, it is a reply packet that is garbled through using a reply block. Bytes 1 through 8 (inclusive) is a 64-bit reply block identifier that we will use to pair this packet with the keys we stored from when we generated the reply block, which we will use to de-garble the message body.
* 621 bytes: ChaCha20-encrypted (no authentication, no nonce) with `KDF(s, "header")` where `s` is the shared-secret between the sender and our X25519 keys.
* The rest of the headers
* As much randomness to pad the entire header to 690 bytes
The padding of the header to 690 bytes is done to keep the header size fixed, which helps with anonymity and prevents traffic analysis. The maximum route length is 10 hops, so subsequent peeled header is exactly 69 bytes smaller than the previous one, but with 69 bytes more random noise added to the back. This means the intermediary nodes have no idea where they are in the route.
## Body format
The body is ChaCha20-encrypted at every layer of the onion with `KDF(s, "body")`, and is always exactly 8192 bytes in length.
## Key-derivation function
The function `KDF` is implemented by using `blake3`'s keyed-hash mode, with the "key" being padded to 32 bytes with `_`. For instance, `KDF(s, "header")` means blake3-hashing `s` with the key `"header__________________________"`.
# Reply blocks
A particular challenge in packet-based onion routing is when Alice sends a message to Bob, how can Bob reply.
The trivial solution is to attach a header to the message with a "source address". This is okay in the normal case, where anonymity against the counterparty is not desired, but not okay if Alice wants to hide her identity from Bob.
In that case, we use **reply blocks**. Let's say $$A$$ sends a message to $$B$$ over intermediary relays $$R\_1,\dots,R\_n$$. To help $$B$$ respond, $$A$$ attaches a reply block containing:
* A randomly generated fingerprint: the **source ID**
* The first hop on the return-path, $$R\_n$$. WLOG we assume there is only one path between the parties, and the reply packet follows the same path but in the other direction.
* An onion-encrypted header encoding the path $$B,R\_n,\dots,R\_1,A$$, as well as ephemeral pubkeys for each layer
* An ephemeral onion public key $$k\_{eph}$$
$$B$$ then encrypts and stuffs the response (which is an InnerPacket) into 8192 bytes using $$k\_{eph}$$and an randomly generated, ephemeral onion public key. He attaches the header verbatim, then sends the packet off to *himself*, as the first hop in Earendil routes is always the sending node itself. In $$B$$'s attempt to peel an layer of the onion while forwarding the packet, the mssage body is garbled (that is, ChaCha20-encrypted, because encryption and decryption are identical for stream ciphers), making this reply packet indistinguishable from forward packets to the next hop, $$R\_n$$.
The message will eventually pass back to $$A$$, garbled by the intermediary nodes' attempts to "decrypt" the message with layers of stream-ciphers. But $$A$$ can reconstruct the keystream that was XOR'ed against the payload, and thus reconstruct the ciphertext that $$B$$ sent.
$$A$$ is then able to decrypt the response from $$B$$.
## How this is exposed in the API
The `recv_from` function will return a packet and *source*. This source is either
* a *direct* address: a fingerprint of a relay or client
* a *reply* address: a randomly-generated fingerprint that serves as the source ID
When `send_to` processes a reply address, it looks up an unused reply block corresponding to the source ID and routes the message through it.
When a node calls `send_to` using an anonymous source address, we send a packet of 8 reply blocks in addition to the message, so that the destination can use them to respond to us.
## DoS resistance
Only the last N reply blocks received from a source, within the last T seconds are kept around. Memory usage can be tightly bound with something like a weighted `moka` cache.
Source IDs that are too inactive become dead.
# Earendil packet vs. Sphinx
### Packet header
* Similarities
* Each layer is encrypted and authenticated by a stream cipher generated from the hash of the shared secret produced by a Diffie-Hellman key exchange using an ephemeral keypair from the sender and a long-term keypair from the other mix node
* Differences
* Unlike us, Sphinx does not store a whole bunch of sender's ephemeral public keys, once for each layer, used for onion encryption inside the packet header. Instead, the sender exploits the structure of Diffie-Hellman to generate a list of keypairs `(sk_1, pk_1), ..., (sk_n, pk_n)` such that knowing `(pk_i)`, plus the `i`th shared secret, allows you to derive `pk_(i+1)`. Each hop on the mix route then derives the next sender ephemeral public key, overwriting that field in the header before forwarding.
* As such, Sphinx must use Diffie-Hellman to derive the shared secret used in making the onion header, while Earendil can be modified to use anything that has the same API as Diffie-Hellman (e.g. some sort of post-quantum scheme)
* First hop in Earendil route is the sender themselves; first hop in Sphinx corresponds to second hop in Earendil
### Forward message body
* Sphinx: body is encrypted with a block cipher, whose block size coincides with the message size, keyed with hash of shared secret for each hop on the route. A block cipher is used so that any corruption of the encrypted data makes the entire message irrecoverable
* Earendil: same, except we use a `ChaCha20` stream cipher, which is faster. We handle integrity protection by layering on a MAC (included in the ChaCha20-Poly1305 algorithm used to encrypt the message body in `InnerPacket::seal()`)
### Reply blocks
* Same everywhere, except Sphinx includes a symmetric key in reply blocks to use for encrypting the message body. Earendil does not need this because the first hop in Earendil is the sending node itself, and the dencryption (XOR with stream cipher) done during the processing of the message body encrypts the body
### Message processing by mix nodes
* Sphinx
* Compute shared secret `s`
* Perform replay protection by comparing a hash of `s` to a table of seen message tags
* Check the MAC
* Pads + decrypts the `header_body` to obtain the next destination
* IF FORWARD: Prepares the onion header for the next hop, decrypts the body with the hash of our shared secret, and sends the packet off
* IF RECEIVE MESSAGE: decrypts the body, get the plaintext msg, send off to dest
* Earendil does all the above, except that we has not yet implemented replay protection
* Note: replay protection is security-critical for mixnets. If a mixnet does not have replay protection, an attacker can duplicate packets of interest going into a mix node and see which next hop receives 2 packets to identify where the packet of interest travels
# Relay configuration
Every connection between relays is bidirectional and symmetric once established.
But how these connections are configured is not symmetric. There is a distinction between accepting a connection from a peer passively, and contacting a peer actively.
This is reflected in the configuration file, where the `in_routes` mapping specifies *ways to accept* incoming connections, while the `out_routes` mapping specifies *specific outgoing connections*.
{% code title="/etc/earendil/config.yaml" %}
```yaml
identity: /etc/earendil/identity.asc
state_cache: /etc/earendil/state_cache.db
# listeners for incoming connections
in_routes:
main_udp:
protocol: obfsudp
listen: 0.0.0.0:19999
secret: correct horse battery staple
main_http:
protocol: http-longpoll
listen: 0.0.0.0:19998
path: /correct/horse/battery/staple
tls:
domain: laboo.example.com
certificate: autoconf
# list of all outgoing connections
out_routes:
alice:
fingerprint: KCKUhWZfluAzMzwiw721CNrvyhc
protocol: obfsudp
connect: 100.1.2.3:18232
cookie: d9aeca8eb2517c18ecf6f24769161be7049187a38c7c8a3391896d502b9bc462
bob:
fingerprint: eveIb0XRU8gULsiYxPBa1aUqjy0
protocol: http-longpoll
connect: https://nala-goosha.example.com/correct/horse/battery/staple
tls-fingerprint-seed: helloworld
```
{% endcode %}
{% hint style="info" %}
Currently, two relays cannot simultaneously specify the other in their `out_routes`. We may change this in the future.
{% endhint %}
# 简介
{% hint style="warning" %}
目前，Earendil 仍处于 **Pre-alpha（预阿尔法）** 阶段，不少功能尚未准备就绪。一些文档页面描述的是尚处于开发中、未发布的功能。
{% endhint %}
[**Earendil**](https://earendil.network) 是一个去中心化的、抗审查的、奖励兼容的通信和价值传输网络。即使面对强大的国家级攻击者，Earendil 网络的任意两个用户仍然可以自由地通讯和交易，
更简单地说，Earendil 是一个**魔法互联网虫洞**：字节和金钱从一端输入，从另一端输出，没有人能叫停。
使用 Earendil 可以：
* 构建反封锁、保护匿名的应用和 P2P 网络
* 匿名浏览普通互联网网站，隐藏你的地理位置
* 通过运行 Earendil 基础设施节点来赚取费用
* 非常低成本地发送基于 Mel 的加密货币
## 为什么选择 Earendil？
乍一看，Earendil 似乎与现有的分布式洋葱路由或混淆网络（如 I2P 或 Nym） 类似。但它有几个特别独特的功能（[这篇中文博文](https://nullchinchilla.me/2023/11/qian-tan-earendil/)中有更多解释）：
### 强大的抗封锁能力
Earendil 抵抗[甲类](https://nullchinchilla.me/2023/05/two-kinds-of-censorship-resistance/)审查（在网络内过滤内容或用户，或称作**过滤**）又抵抗[乙类](https://nullchinchilla.me/2023/05/two-kinds-of-censorship-resistance/)审查（完全阻止访问 Earendil，或称作**封禁**）。
在其他项目中，抗封禁能力非常罕见。即使存在，也通常局限于针对国家级防火墙（如中国的长城防火墙）的特殊防御手段（例如 Tor 的混淆桥接）。
与此不同，Earendil 的设计目的是：**即使是 GFW（长城防火墙）在全球范围内部署**也能够运作。它不假设大部分网络存在于「自由世界」中。
### 简单、自由的点对点激励机制
Earendil **可选地**允许每个节点设置其对等点必须支付的资源消耗价格，通过使用 MEL 结算的加密货币进行微支付（micropayments）。
微支付巧妙地解决了抗女巫攻击（防止恶意节点淹没网络）、诚实节点的激励，以及抵抗拒绝服务攻击（DoS）。行为不端的节点将不会被他们的对等点支付，而诚实节点被激励在自由市场中竞争，以向客户提供最好的服务。企图刷屏 Earendil 的恶意行为者必须相应地支付网络费用。
这比其他激励/抗女巫攻击的机制有重要优势，例如用户基于向智能合约支付费用，而服务商向其证明对网络做出的贡献以取款的「统购统销」模型。在这篇[博文](https://nullchinchilla.me/2023/07/earendil-incentives/)中有进一步的解释。
## Earendil 是如何工作的？
简而言之，Earendil 的架构是一个叠加到抗封禁“互联网”上的混合网络。这意味着一个由两部分构成的设计：
### “抗封禁互联网”部分
我们在互联网之上叠加了一个点对点的数据包路由网络，而这个网络本身非常类似于互联网——数据包有一个目的地，并且中继节点将数据包一跳一跳地转发到更接近其目的地的地方。 这一层通过结合两个特性来隐藏于敌对的互联网服务提供商并实现抗封禁：
* **逐链路混淆**：Earendil 流量默认难以与“正常”网络流量区分开来。此外，任何特定节点间链接使用的协议都可以被替换，使用类似于 Tor 桥接的“可插拔传输”架构，适应特别严峻的网络环境（例如，只允许明文 HTTP 并对所有 HTTPS 流量进行中间人攻击的网络）。
* \*\*邀请制、隐藏信息的架构：\*\*Earendil 通过一种独特的，仅通过邀请的架构来路由流量，该架构仅向每个参与者的直接对等点透露 IP 地址，使即便是强大的攻击者也难以编制起 Earendil 节点的 IP 地址清单，用于监控或审查。封锁他们已知的所有节点的审查者可能最终只会将自己从网络中封锁。 这一层不提供匿名性，只提供可靠性和抗封禁性。我们也通过一个简单的机制在这一层实现所有的激励机制，即用户直接支付对端节点所有在他们身上使用的资源。
### “混淆网络”部分
在这个点对点网络之上，我们叠加了一个与 Nym 类似设计的混合网络，通过洋葱加密和延迟提供强大的匿名性。所有中继节点也必须参与混合网络才能成为 Earendil 网络的一部分。 即使在攻击者完全监视底层网络的所有流量的情况下，混合网络也被设计为保持匿名性。这防止了无论混淆协议或激励支付系统泄露信息的严重性如何，都不会泄露有关谁与谁通信的任何信息。
## 开发状态与路线图
目前，Earendil 处于 **Pre-alpha（预阿尔法）** 阶段。你可以加入 Earendil 网络并在其上通信，但一些重要功能还原始或不完整：
| 功能 | 完成状态 | 备注 |
| ---------------- | -------------------------------------------- | -------------------------------------------------- |
| 加入作为中继节点 | :white\*check\_mark: 完成 | |
| 加入作为客户端 | :white\_check\_mark: 完成 | |
| 洋葱路由 | 🚧 基本实现 | *无法自定义路由选择。* |
| 避风港（匿名托管） | :white\_check\_mark: | |
| 网页代理 | 🚧 基本实现 | *你可以托管网页代理，以帮助其他用户通过 Earendil 使用网络。但未实现用户认证或访问控制。* |
| 债务计算 | :white\_check\_mark: | *计算需支付邻居的资源费用。* |
| 手动债务结算 | :white\_check\_mark: | *允许通过线下方式结算计算出的债务，并在协议中手动重置。* |
| 混淆网络（mixnet）延迟 | 🚧 基本实现 | *延迟消息以对抗大规模攻击者保护匿名性* |
| 自动债务结算（MVP） | 🚧 基本实现 | *使用链上转账自动结算以 MEL 计价的债务* |
| 以 Mel 作后盾的女巫攻击抵抗 |