summaryrefslogtreecommitdiff
path: root/docs/core-academy/ca08.md
blob: fb61f265fe7f449bfa7182acd18cbefff2dd405d (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
---
description: "Core Academy lesson on Ames covering the encrypted peer-to-peer protocol, its packet structure, message flows, cryptographic operations, integration with Urbit ID, and the Fine remote scry protocol."
layout:
  title:
    visible: true
  description:
    visible: false
  tableOfContents:
    visible: true
  outline:
    visible: true
  pagination:
    visible: true
---

# 9. Vanes II:  Ames

*This lesson covers Ames: Urbit's networking protocol. We'll see how it relates to Urbit ID and Fine, the remote scry protocol.*

Urbit knows about three kinds of networking:  Ames and Fine over Ames and HTTP over Eyre. (That is, the network protocol over the implementing vane.)  Ames is the name of both the network protocol and the implementing vane. Ames is particularly for ship-to-ship communication, while Fine acts as a dispatcher for efficient data requests (such as desk updates). In this lesson, we will focus on Ames first and then foray into Fine, since Fine is in more flux.

Ames is a good example of a vane that is conceptually straightforward but mechanically complicated. It handles networking, but has to track peer state, message flows, individual packets, network weather, etc. as gracefully as possible.


## Network Protocol:  Ames {#network-protocol-ames}

> Ames is an encrypted peer-to-peer network running as an overlay over UDP. Ames does not have separate addressing and identity layers (like IP and DNS). An Ames address is an identity, mapped to a phonemic string to create a memorable pseudonym, and bound to a public key for encrypted communication. (Whitepaper)

From a kernel perspective, the point of Ames is to extend `move` semantics across more than one Arvo instance. Local vanes (such as Gall) pass a `%plea` request to Ames, which sends the message to the peer Ames over the wire. That peer's Ames then dispatches the message to the destination vane on the peer. One advantage of Ames is that it wraps all of the peer negotiation and message delivery details such that the calling vane need not be aware of these. Among Ames’ guarantees:

1. Messages within a flow are processed in order.
2. Messages will be delivered only once to a destination vane. (“Ames can guarantee exactly-once delivery because both ships are transactional (so if they give an ack, we know for sure they have received it permanently and won't forget about it.”  [~wicdev-wisryt](https://groups.google.com/a/urbit.org/g/dev/c/y_gaSpn9mxM/m/zNy1l9ufAgAJ))

Networking in some ways is like a duct. It requires you to keep track of the forward and reverse causal history and content. However, there are two kinds of data transmissions:  commands and facts.

> There is a categorical difference between a bus, which transports commands, and a network, which transports packets. You can drop a packet but not a command; a packet is a fact and a command is an order. Facts are inherently idempotent; learning fact _X_ twice is the same as learning it once. You can drop a packet, because you can ignore a fact. Orders are inherently sequential; if you get two commands to do thing _X_, you do thing _X_ twice. (Whitepaper)

The Ames vane is responsible for sending and receiving messages of arbitrary length. This means that it needs to know how to build and reconstruct component packets of a message, how to route, and how to encrypt and decrypt. Ames does handle some aspects of encryption and decryption but not all. For networking, Ames gets its public keys (and breach notifications) from Jael, which in turn gets them from an Azimuth userspace agent. The actual cryptographic operations may live in `/sys/zuse` but are applied by Ames as appropriate.

- [“Ames Overview”](../../urbit-os/kernel/ames)
- [Curtis Yarvin `~sorreg-namtyv`, Philip Monk `~wicdev-wisryt`, Anton Dyudin, and Raymond Pasco, “Urbit:  A Solid-State Interpreter” (“Whitepaper”)](http://media.urbit.org/whitepaper.pdf), sections 9–10
- [“Ames Security Audit and the Future of the Protocol”](https://urbit.org/blog/security-audit)

### Packet Protocol {#packet-protocol}

> Ames receives packets as Arvo events and emits packets as Arvo effects. The runtime is responsible for transferring the bytes in an Ames packet across a physical network to another ship. (Ames Tutorial)

Ames packets have a 32-bit header followed by a variable-length body.

**Header**

| Bits | Representative Value | Meaning |
| --- | --- | --- |
| 31–29 | `000` | reserved bits |
| 28 | `1` | Ames or Fine? |
| 27–25 | `000` | Ames protocol version |
| 24–23 | `11` | sender address size |
| 22–21 | `11` | receiver address size |
| 20–1 | `1000.0001.1101.0010.1111` | checksum |
| 0 | `1` | is this relayed? |

The 2-bit address size refers to the address space rank (gathering galaxies and stars together as routers).

```hoon
++  ship-meta
  |=  =ship
  ^-  [size=@ =rank]
  =/  size=@  (met 3 ship)
  ?:  (lte size 2)  [2 %0b0]
  ?:  (lte size 4)  [4 %0b1]
  ?:  (lte size 8)  [8 %0b10]
  [16 %0b11]
```

A `relay` means that the packet is not at its destination here and should be passed forward. (This is handled by `+on-hear-forward` in `/sys/ames`.)

> If a relay responds to a scry request from its cache without asking the host, the relay should include an origin containing the last known IP and port of the host. … \[The] protocol should be resilient against the origin pointing at an unreachable IP and port.

**Body**

| Number of Bits | Representative Value | Meaning |
| --- | --- | --- |
| 4 bits | `0000` | sender life (mod 16) |
| 4 bits | `0000` | receiver life (mod 16) |
| variable | `0110.0111.0110.1011` | sender address |
| variable | `1111.1100.0111.0011.0000.0101.0000.0000` | receiver address |
| 48 bits | — | `origin` (if relayed) |
| 128 bits | — | `SIV` synthetic initialization vector for AES-256 |
| 16 bits | — | ciphertext size |
| variable | — | ciphertext |

Address size is determined by the header.

Here if the relay bit is set then 32 bits of the `origin` are the last known IPv4 address and 16 bits are the port.

> The ciphertext is formed by `+jam`ming a `$shut-packet` and then encrypting using [`+en:sivc:aes:crypto`](../../hoon/cryptography.md#en).

The ciphertext results from `+jam`ming the message noun into an atom then breaking the result into 1 KB or smaller payloads. Packets are numbered so that they can be ordered upon receipt. These message fragments are then assembled into a single large atom and `+cue`d back into the noun.

Urbit messages result in raw nouns. Since Nock-derived languages are homoiconic, we could treat this noun as code directly, but instead we treat it as a cask (pair of mark and noun). We don't transmit vases over the network, but require the recipient to build the code locally.

> Ames messages are typed; the type itself is not sent, just a label (like a MIME type) that the recipient must map to a local source path. Validation failure causes a silent packet drop, because its normal cause is a recipient that has not yet received a new protocol update; we want the sender to back off. Ames also silently drops packets for encryption failure; error reports are just an attack channel.

#### UDP Packet Format

> There's a lot you can do with a stateful UDP server, especially one whose semantics are reasonably formal. (CGY)

At the host system level, the runtime communicates using the [User Datagram Protocol](https://en.wikipedia.org/wiki/User_Datagram_Protocol) (UDP) specification. UDP messages are “transaction oriented, and delivery and duplicate protection are not guaranteed.”  (To compensate for this, Ames employs a unique system of acks and nacks, covered below.)  Each UDP message has a brief header including destination, source, length, and checksum. It’s rather a “minimum viable” packet system.

> A UDP datagram consists of a datagram _header_ followed by a _data_ section (the payload data for the application). The UDP datagram header consists of 4 fields, each of which is 2 bytes (16 bits).
> UDP is faster but less reliable than TCP, another common transport protocol. In a TCP communication, the two computers begin by establishing a connection via an automated process called a ‘handshake.’ Only once this handshake has been completed will one computer actually transfer data packets to the other. ([Wikipedia](https://en.wikipedia.org/wiki/User_Datagram_Protocol))

Urbit compensates for this lower reliability by sending until receiving an appropriate ack or nack (negative acknowledgment) in reply.

> UDP is commonly used in time-sensitive communications where occasionally dropping packets is better than waiting. Voice and video traffic are sent using this protocol because they are both time-sensitive and designed to handle some level of loss. For example VOIP (voice over IP), which is used by many internet-based telephone services, operates over UDP. This is because a staticky phone conversation is preferable to one that is crystal clear but heavily delayed. ([Wikipedia](https://en.wikipedia.org/wiki/User_Datagram_Protocol))

- [RFC 768](https://tools.ietf.org/html/rfc768) (UDP specification)

### Acks and Nacks {#acks-and-nacks}

```hoon
::  $ack: positive ack, nack packet, or nack trace
::
+$  ack
  $%  [%ok ~]
      [%nack ~]
      [%naxplanation =error]
  ==
```

If every message is a transaction (or event), then what is Ames acknowledging (ack) or negatively acknowledging (nack)?  “A successful transaction has no result; a failed transaction is a negative ack and can contain an error dump.”

- An _ack_ means that a piece of information has been received successfully.
- A _nack_ means that a piece of information has been received but failed to process for some reason.

> Ames has an unusual system of acks and nacks (“negative acknowledgments”, but not like TCP’s packets of the same name; Ames nacks mean the packet was received but the message resulted in an error). In brief, each Ames packet of a message should get either an ack or a nack. In the current system, the nack may include an error message (e.g., an error code or a stack trace). ([~wicdev-wisryt](https://groups.google.com/a/urbit.org/g/dev/c/y_gaSpn9mxM/m/njlRhYZHBwAJ))

Each Ames packet will merit either an ack or a nack, in other words. A nack may optionally include an error trace with it (`[tag=@tas =tang]`). Ames will adaptively continue to send messages until the appropriate acks or nacks have been received.

(A TCP nack means that the numbered packet was never received.)

Ames will send messages and acks until a replying ack is received. “Ames guarantees that a message will only be delivered once to the destination vane.”  Thus nacks allow us to also guarantee notification that a request was completed or failed.

> 1. Always ack a dupe; never ack an ack. It's okay to ack a nack as long as you never nack a nack. ([Urbit Precepts B.1](https://urbit.org/blog/precepts))

If a remote ship sends a nack in response to a `%plea`, Ames waits until it receives a follow-up naxplanation and then delivers both to the local source vane. The flow blocks on needing to receive the naxplanation.

> When a new socket is opened, the client can resend (at-least-once delivery) or fail to resend (at-most-once). The programmer has to understand that the socket is not really a bus, and make sure the POST is actually an idempotent fact rather than an imperative command. (The idempotence problem is often punted to the human layer: “Please click only once to make your purchase.”) (Whitepaper)

Because Ames and Urbit assume several nines of uptime, sessions between ships are treated as persistent.

> The basic argument for including end-to-end acks (and by extension, nacks) is that they’re necessary for everything except those things which we don’t care whether the message was received at all. Thus, for Ames to give the guarantee that “if you give me a payload I will get it to the other side exactly once” isn’t useful in itself, because no application cares about that. They either (1) don’t care whether it gets there or (2) care whether the request itself was “completed”, in an application-defined sense. ([Phillip Monk, `~wicdev-wisryt`](https://groups.google.com/a/urbit.org/g/dev/c/y_gaSpn9mxM/m/njlRhYZHBwAJ))

Keep in mind Postel’s law, also known as the robustness principle: “Be conservative in what you send, and liberal in what you accept.”

- [Saltzer, Reed, and Clark, “End-to-End Arguments in System Design”](http://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf) on nacks

### Cryptography {#cryptography}

Almost every Ames packet is encrypted using [AES-256](https://en.wikipedia.org/wiki/Advanced_Encryption_Standard). (The exception is comet self-attestation packets.)

Urbit's cryptographic suite for jets has been organized to present a uniform interface (currently in `urcrypt/`, migrating to its own repo). This eases the development of alternative runtimes since the C functions can be utilized as an FFI (foreign function interface) with uniform call signatures and behavior.

The details of the Azimuth PKI are discussed in `ca13`, _quod vide_.

- [“Ames:  Cryptography”](../../urbit-os/kernel/ames/cryptography.md)

### Routing {#routing}

> The runtime tells Ames which physical address a packet came from, represented as an opaque atom. Ames can emit a packet effect to one of those opaque atoms or to the Urbit address of a galaxy (root node), which the runtime is responsible for translating to a physical address. (See `$lane`.)

```hoon
::  $address: opaque atomic transport address to or from unix
+$  address  @uxaddress
::  $lane: ship transport address; either opaque $address or galaxy
+$  lane  (each @pC address)
```

The `@uxaddress` value is an “opaque” address, in reality an IPv4 address for the runtime's use.

Since galaxy addresses are provided to the runtime on boot (from an RPC call to a roller or Ethereum node, see `vere/dawn.c:_dawn_eth_rpc()`), a route is always findable for any active point. Galaxy ports are hardcoded to be at 31337 or 13337 plus the galaxy numeric offset.

- What does `@pC` mean?

When we say that galaxies handle routing in Ames today (but stars will play a role later), this is the part of the system to which we refer.

### `/sys/lull` Definition {#syslull-definition}

The `/sys/lull` interface definition for Ames is quite long and should be reviewed in its entirety. A structural summary:

```hoon
++  ames  ^?
  |%
  ::  $task: job for ames
  +$  task
    $+  ames-task
    $%  [%hear =lane =blob]
        [%dear =ship =lane]
        [%heed =ship]
        [%jilt =ship]
        [%cork =ship]
        [%tame =ship]
        [%kroc bones=(list [ship bone])]
        $>(%plea vane-task)
        [%deep =deep]
    ::
        [%keen spar]
        [%yawn spar]
        [%wham spar]
    ::
        $>(%born vane-task)
        $>(%init vane-task)
        [%prod ships=(list ship)]
        [%sift ships=(list ship)]
        [%snub form=?(%allow %deny) ships=(list ship)]
        [%spew veb=(list verb)]
        [%cong msg=@ud mem=@ud]
        [%stir arg=@t]
        $>(%trim vane-task)
        $>(%vega vane-task)
    ==
  ::  $gift: effect from ames

  +$  gift
    $%  [%boon payload=*]
        [%clog =ship]
        [%done error=(unit error)]
        [%lost ~]
        [%send =lane =blob]
    ::
        [%tune spar roar=(unit roar)]
    ::
        [%turf turfs=(list turf)]
    ==
```

#### Tasks

```
::  $task: job for ames
::
::    Messaging Tasks
::
::    %hear: packet from unix
::    %dear: lane from unix
::    %heed: track peer's responsiveness; gives %clog if slow
::    %jilt: stop tracking peer's responsiveness
::    %cork: request to delete message flow
::    %tame: request to delete route for ship
::    %kroc: request to delete specific message flows, from their bones
::    %plea: request to send message
::    %deep: deferred calls to %ames, from itself
::
::    System and Lifecycle Tasks
::
::    %born: process restart notification
::    %init: vane boot
::    %prod: re-send a packet per flow, to all peers if .ships is ~
::    %sift: limit verbosity to .ships
::    %snub: set packet blocklist to .ships
::    %spew: set verbosity toggles
::    %cong: adjust congestion control parameters
::    %stir: recover from timer desync and assorted debug commands
::    %trim: release memory
::    %vega: kernel reload notification
```

Ames has a rather bohemian set of messaging names. Among others:

- `%hear` a packet
- `%heed` or `%jilt` a peer
- `%plea` to send a message (common from vanes)

Most other tasks are not used by userspace but by internal Ames state management. These are complemented by types like these:

- `$hoot` a request packet payload
- `$yowl` a serialized response packet payload

#### Notes

```
::    Messaging Gifts
::
::    %boon: response message from remote ship
::    %clog: notify vane that %boon's to peer are backing up locally
::    %done: notify vane that peer (n)acked our message
::    %lost: notify vane that we crashed on %boon
::    %send: packet to unix
::    %tune: peek result
::    %turf: domain report, relayed from jael
```

Every vane can receive a `%plea` note from Ames (except Behn, Dill, Iris, Khan, Lick). This is a redirection mechanism used to forward messages that a peer's vane passed to its own Ames en route to your peer's Ames and thence to your vane.

#### State

```
::  $ames-state: state for entire vane
+$  ames-state
  $+  ames-state
  $:  peers=(map ship ship-state)
      =unix=duct
      =life
      =rift
      crypto-core=acru:ames
      =bug
      snub=[form=?(%allow %deny) ships=(set ship)]
      cong=[msg=_5 mem=_100.000]
    ::
      $=  dead
      $:  flow=[%flow (unit dead-timer)]
          cork=[%cork (unit dead-timer)]
  ==  ==
```

- `$peers` are the state of connections to other ships, where `$ship-state` is either `%alien` or `%known`.
    - `%alien` means we have no PKI data and we must queue moves until we learn how contact that ship. The `$alien-agenda` stores messages, packets, and remote scry `keen`s.
    - `%known` means that we do have the peer state, on which more later.
- `$unix-duct` is a duct of moves to be sent to the host OS.
- `$life` is our own `life`, or how many times we rekeyed.
- `$crypto-core` is a handle to the cryptographic tools core.
- `$bug` describes the debug level (`|ames/verb`).
- `$snub` tracks a blocklist for incoming packets (`|ames/snub`).
- `$cong` tracks whether a flow should be considered clogged.
- `$dead` sets how long dead flows last and if they need to be restarted.

Ames maintains a duct (queue) of ordered messages. These are passed to and received from the runtime, and represent Arvo events. Each message is encrypted at the source and decrypted at the destination using a symmetric public-key system. A message may be a `%plea` (sent to another ship); in response, Ames can receive zero or more `%boon`s. The ack–nack system is explained above; note that nacks are in response to event crashes.

In `/sys/vane/ames`, there is a layer of versioning cruft to permit upgrades of the types (e.g. `$ames-state-5`).

#### Peer State

```hoon
::  $peer-state: state for a peer with known life and keys
::
::    route: transport-layer destination for packets to peer
::    qos: quality of service; connection status to peer
::    ossuary: bone<->duct mapper
::    snd: per-bone message pumps to send messages as fragments
::    rcv: per-bone message sinks to assemble messages from fragments
::    nax: unprocessed nacks (negative acknowledgments)
::         Each value is ~ when we've received the ack packet but not a
::         nack-trace, or an error when we've received a nack-trace but
::         not the ack packet.
::
::         When we hear a nack packet or an explanation, if there's no
::         entry in .nax, we make a new entry. Otherwise, if this new
::         information completes the packet+nack-trace, we remove the
::         entry and emit a nack to the local vane that asked us to send
::         the message.
::    heeds: listeners for %clog notifications
::    closing: bones closed on the sender side
::    corked:  bones closed on both sender and receiver
::
+$  peer-state
  $+  peer-state
  $:  $:  =symmetric-key
          =life
          =rift
          =public-key
          sponsor=ship
      ==
      route=(unit [direct=? =lane])
      =qos
      =ossuary
      snd=(map bone message-pump-state)
      rcv=(map bone message-sink-state)
      nax=(set [=bone =message-num])
      heeds=(set duct)
      closing=(set bone)
      corked=(set bone)
      keens=(map path keen-state)
  ==
```
### Structure {#structure}

Ames’ formal interface is included more than once (like Arvo) as the “external vane interface” and the “adult ames”, for instance.

```hoon
++  call  :: handle request stack
++  take  :: handle response $sign
++  stay  :: extract state before reload
++  load  :: load in old state after reload
++  scry  :: dereference namespace
```

There is a collection of `ames-helper` cores as well to handle many specific cases for unpacking and routing messages. Ames uses a more sophisticated nested core pattern than Behn did. To that end, it presents five `+abet` cores:

- `ev` event handling core
- `mi` message receiver core
- `mu` message sender core
- `pe` per-peer processing core
- `pu` packet pump

There is a substantial amount of legacy Ames state upgrade debris in the file as well.

#### Scries

As typical, scries expose internal vane state. Ames has a richer inner life than some other vanes, so you can check on peer state and snubs and message flow details.

```hoon
.^((map ship ?(%alien %known)) %ax /=//=/peers)

.^(ship-state:ames %ax /=//=/peers/~zod)

!< message-pump-state:ames .^(vase %ax /=//=/snd-bones/~zod/0)
```

Most Ames scries aren't particularly useful to us directly unless we want to do direct network negotiation. Ames is used frequently by Gall but, from the agent's perspective, incidentally.

### Messages & Flows {#messages-flows}

We've looked at the packet protocol before; now let's look at Ames' mechanics of message management.

```hoon
+$  fragment       @uwfragment
+$  fragment-num   @udfragmentnum
+$  message-blob   @udmessageblob
+$  message-num    @udmessagenum
```

Messages are separated into 1 KB (or smaller) fragments and sequentially numbered.

- Examine `+split-message` to see how messages are broken up into pieces. (There's a neat optimization therein.)

Messages are of course sent and received in fragments. The messages from a `lane` accrue for a particular `bone` using `+mi`, the message receiver core (internal alias `sink`).

- Examine `+hear` and  `+assemble-fragments`.
- Start the debug server (`|start %dbug`) and navigate to `/~debug`. Select `ames` to see message flows.

The message pump manages unsent messages, dispatching them to the packet pump when next in the queue.

> When we pop a message off .unsent-messages, we push as many fragments as we can into |packet-pump, which sends every packet it eats. Packets rejected by |packet-pump are placed in .unsent-fragments.
> When we hear a packet ack, we send it to |packet-pump to be removed from its queue of unacked packets.
> When we hear a message ack (positive or negative), we treat that as though all fragments have been acked.

There are a ton of other edge cases and consistency/sanity checks on messaging, one of the reasons that Ames is relatively complicated.

> At the end of a task, |message-pump sends a %halt task to |packet-pump, which can trigger a timer to be set or cleared based on congestion control calculations. When the timer fires, it will generally cause a packet to be re-sent.
> Message sequence numbers start at 1 so that the first message will be greater than .last-acked.message-sink-state on the receiver.

```hoon
+$  message-pump-state
  $+  message-pump-state
  $:  current=_`message-num`1
      next=_`message-num`1
      unsent-messages=(qeu message-blob)
      unsent-fragments=(list static-fragment)
      queued-message-acks=(map message-num ack)
      =packet-pump-state
  ==
::
+$  static-fragment
  $:  =message-num
      num-fragments=fragment-num
      =fragment-num
      =fragment
  ==
::
+$  partial-rcv-message
  $:  num-fragments=fragment-num
      num-received=fragment-num
      fragments=(map fragment-num fragment)
  ==
```

> A vane can pass Ames a `%heed` `task` to request Ames track a peer's responsiveness. If our `%boon`s to it start backing up locally, Ames will `give` a `%clog` back to the requesting vane containing the unresponsive peer's Urbit address.

```hoon
+$  qos
  $~  [%unborn *@da]
  [?(%live %dead %unborn) last-contact=@da]
```

To cork a flow (or “cork a bone”) closes the flow. A dangling bone refers to an incorrect bone (a message flow was closed on one side before all message fragments were received, for instance).

A message flow organizes a sequence of message fragments together. Within a flow, data order is guaranteed; however due to network traffic flows may arrive out of order.

```hoon
+$  bone           @udbone
::
+$  ossuary
  $:  =next=bone
      by-duct=(map duct bone)
      by-bone=(map bone duct)
  ==
```

A `$bone` is a duct handle, a way of identifying a particular message flow over the network.

Each `bone` increments by 4 since each flow includes a least-significant bit indicating if we send or receive pleas and a second-least-significant bit indicating if we are a diagnostic flow (naxplanation) or not.

```hoon
> *bone
0

> .^([snd=(set bone) rcv=(set bone)] %ax /=//=/bones/~nes)
[snd={0} rcv={}]
```

The `$ossuary` holds the bone↔duct bijection and the `next-bone` to map to a duct. (Thus the increment-by-four noted above.)

```hoon
::  $pump-metrics: congestion control state for a |packet-pump
::
::    This is an Ames adaptation of TCP's Reno congestion control
::    algorithm.  The information signals and their responses are
::    identical to those of the "NewReno" variant of Reno; the
::    implementation differs because Ames acknowledgments differ from
::    TCP's, because this code uses functional data structures, and
::    because TCP's sequence numbers reset when a peer becomes
::    unresponsive, whereas Ames sequence numbers only change when a
::    ship breaches.
::
::    A deviation from Reno is +fast-resend-after-ack, which re-sends
::    timed-out packets when a peer starts responding again after a
::    period of unresponsiveness.
::
::    If .skips reaches 3, we perform a fast retransmit and fast
::    recovery.  This corresponds to Reno's handling of "three duplicate
::    acks".
::
::    rto: retransmission timeout
::    rtt: roundtrip time estimate, low-passed using EWMA
::    rttvar: mean deviation of .rtt, also low-passed with EWMA
::    ssthresh: slow-start threshold
::    cwnd: congestion window; max unacked packets
::
+$  pump-metrics
  $:  rto=_~s1
      rtt=_~s1
      rttvar=_~s1
      ssthresh=_10.000
      cwnd=_1
      counter=@ud
  ==
```

### Vere I/O Driver:  `vere/io/ames.c` {#vere-io-driver-vereioamesc}

As elsewhere, the `libuv` event loop processor with callback functions responds to Ames-specific initiating events, in this case, the receipt of a UDP packet. The C side of Ames handles constructing and dispatching the UDP packets that underlie Ames communications, but perhaps surprisingly `ames.c` is actually less complicated and interesting than `ames.hoon`. (There's some serialization handling too.)

- `_ames_czar_cb()` for galaxy address resolution
- `_ames_send_cb()` for UDP transmission
- `_ames_recv_cb()` for UDP reception

## Network Protocol:  Fine {#network-protocol-fine}

A scry is a read-only request into the scry namespace. Historically, only local scries were supported, and these were instrumented synchronously using `.^` dotket. With the addition of remote scry, a new use case and use pattern emerged:  asynchronous reads over the network.

> A ship that wants to read from a remote part of the namespace will have to pass a `%keen` task to its Ames, which then cooperates with Vere to produce the desired data. In some future event when the result is available, Ames gives it back as a `%tune` gift. From the requester's perspective, this is the entire default lifecycle of a remote scry request.

```hoon
::    Remote Scry Tasks
::
::    %keen: peek: [ship /vane/care/case/spur]
::    %yawn: cancel request from arvo
::    %wham: cancels all scry request from any vane
::
```

Fine maintains its own state, but other than having its own types its operation is not so different from Ames that we need to delve into it hear.

```hoon
+$  keen-state
  $+  keen-state
  $:  wan=((mop @ud want) lte)  ::  request packets, sent
      nex=(list want)           ::  request packets, unsent
      hav=(list have)           ::  response packets, backward
      num-fragments=@ud
      num-received=@ud
      next-wake=(unit @da)
      listeners=(set duct)
      metrics=pump-metrics
  ==
+$  want
  $:  fra=@ud
      =hoot
      packet-state
  ==
+$  have
  $:  fra=@ud
      meow
  ==
::
+$  meow  ::  response fragment
  $:  sig=@ux  ::  signature
      num=@ud  ::  number of fragments
      dat=@ux  ::  contents
  ==
::
+$  peep  ::  fragment request
  $:  =path
      num=@ud
  ==
::
+$  wail  ::  tagged request fragment
  $%  [%0 peep] :: unsigned
  ==
::
+$  roar  ::  response message
  (tale:pki:jael (pair path (unit (cask))))
::
+$  purr  ::  response packet payload
  $:  peep
      meow
  ==
```

(Having worked with remote scries some in userspace, I recommend tombstoning old endpoints when they are done being used.)

### Runtime Scry Dispatch {#runtime-scry-dispatch}

Remote scries are handled by the runtime rather than generating an Arvo event.

In `vere/io/ames.c`, a scry hashtable `sac_p` is created. `ames_hear()` decides whether to inject the packet into Arvo (Ames protocol) or handle in Vere (Fine protocol).

- `_fine_hear_request()` to receive a request
- `_fine_hear_response()` to receive the response
- `_fine_get_cache()`
- `_fine_put_cache()`

There are also provisions for Fine scry path length etc. therein.

- [“Guides:  Remote Scry”](../userspace/remote-scry.md)
- [~rovnys-ricfer, “Remote Scry Protocol Proposal”](https://gist.github.com/belisarius222/d9a9c164817d3e8bbda3c45f7d2000b9)