diff options
| author | polwex <polwex@sortug.com> | 2025-10-06 01:01:41 +0700 |
|---|---|---|
| committer | polwex <polwex@sortug.com> | 2025-10-06 01:01:41 +0700 |
| commit | c4b392a179048f936c062f5ffccc2bc25627e500 (patch) | |
| tree | 09be0904be8ec4d7ea52992ef7580d42ed0c28c1 /arvo | |
working
Diffstat (limited to 'arvo')
| -rw-r--r-- | arvo/eyre.hoon | 4712 | ||||
| -rw-r--r-- | arvo/lull.hoon | 4622 |
2 files changed, 9334 insertions, 0 deletions
diff --git a/arvo/eyre.hoon b/arvo/eyre.hoon new file mode 100644 index 0000000..0abd1c3 --- /dev/null +++ b/arvo/eyre.hoon @@ -0,0 +1,4712 @@ +!: +:: lighter than eyre +:: +|= our=ship +=, eyre +:: internal data structures +:: +=> =~ +:: +:: internal data structures that won't go in zuse +:: +|% ++$ move + :: + $: :: duct: request identifier + :: + =duct + :: + :: + card=(wind note gift) + == +:: +note: private request from eyre to another vane +:: ++$ note + $% [%a $>(?(%plea %keen %yawn) task:ames)] + [%b $>(?(%rest %wait) task:behn)] + [%c $>(%warp task:clay)] + [%d $>(%flog task:dill)] + [%g $>(%deal task:gall)] + == +:: +sign: private response from another vane to eyre +:: ++$ sign + $% [%ames $>(?(%done %boon %lost %sage) gift:ames)] + [%behn $>(%wake gift:behn)] + [%gall gift:gall] + [%clay gift:clay] + == +-- +:: more structures +:: +|% +++ axle + $: :: date: date at which http-server's state was updated to this data structure + :: + date=%~2025.1.31 + :: server-state: state of inbound requests + :: + =server-state + == +:: +server-state: state relating to open inbound HTTP connections +:: ++$ server-state + $: :: bindings: actions to dispatch to when a binding matches + :: + :: Eyre is responsible for keeping its bindings sorted so that it + :: will trigger on the most specific binding first. Eyre should send + :: back an error response if an already bound binding exists. + :: + :: TODO: It would be nice if we had a path trie. We could decompose + :: the :binding into a (map (unit @t) (trie knot =action)). + :: + bindings=(list [=binding =duct =action]) + :: cache: mapping from url to versioned entry + :: + cache=(map url=@t [aeon=@ud val=(unit cache-entry)]) + :: cors-registry: state used and managed by the +cors core + :: + =cors-registry + :: connections: open http connections not fully complete + :: + connections=(map duct outstanding-connection) + :: auth: state managed by the +authentication core + :: + auth=authentication-state + :: channel-state: state managed by the +channel core + :: + =channel-state + :: domains: domain-names that resolve to us + :: + domains=(set turf) + :: http-config: our server configuration + :: + =http-config + :: ports: live servers + :: + ports=[insecure=@ud secure=(unit @ud)] + :: outgoing-duct: to unix + :: + outgoing-duct=duct + :: verb: verbosity + :: + verb=@ + :: check-session-timer: set to true for ships prior to ~2025.01.31, + :: who may have been affected by urbit/urbit#7103 + :: + check-session-timer=_| + == +:: channel-request: an action requested on a channel +:: ++$ channel-request + $% :: %ack: acknowledges that the client has received events up to :id + :: + [%ack event-id=@ud] + :: %poke: pokes an application, validating :noun against :mark + :: + [%poke request-id=@ud ship=@p app=term mark=@tas =noun] + :: %poke-json: pokes an application, translating :json to :mark + :: + [%poke-json request-id=@ud ship=@p app=term mark=@tas =json] + :: %watch: subscribes to an application path + :: + [%subscribe request-id=@ud ship=@p app=term =path] + :: %leave: unsubscribes from an application path + :: + [%unsubscribe request-id=@ud subscription-id=@ud] + :: %delete: kills a channel + :: + [%delete ~] + == +:: clog-timeout: the delay between acks after which clog-threshold kicks in +:: +++ clog-timeout ~s30 +:: clog-threshold: maximum per-subscription event buildup, after clog-timeout +:: +++ clog-threshold 50 +:: channel-timeout: the delay before a channel should be reaped +:: +++ channel-timeout ~h12 +:: session-timeout: the delay before an idle session expires +:: +++ session-timeout + |% + ++ auth ~d30 + ++ guest ~d7 + -- +:: eauth-timeout: max time we wait for remote scry response before serving 504 +:: eauth-cache-rounding: scry case rounding for cache hits & clock skew aid +:: +++ eauth-timeout ~s50 +++ eauth-cache-rounding ~m5 +-- +:: utilities +:: +|% +:: +combine-octs: combine multiple octs into one +:: +++ combine-octs + |= a=(list octs) + ^- octs + :- %+ roll a + |= [=octs sum=@ud] + (add sum p.octs) + (can 3 a) +:: +prune-events: removes all items from the front of the queue up to :id +:: +:: also produces, per request-id, the amount of events that have got acked, +:: for use with +subtract-acked-events. +:: +++ prune-events + =| acked=(map @ud @ud) + |= [q=(qeu [id=@ud @ud channel-event]) id=@ud] + ^+ [acked q] + :: if the queue is now empty, that's fine + :: + ?: =(~ q) + [acked ~] + :: + =/ next=[item=[id=@ud request-id=@ud channel-event] _q] ~(get to q) + :: if the head of the queue is newer than the acknowledged id, we're done + :: + ?: (gth id.item.next id) + [acked q] + :: otherwise, note the ack, and check next item + :: + %_ $ + q +:next + :: + acked + =, item.next + %+ ~(put by acked) request-id + +((~(gut by acked) request-id 0)) + == +:: +subtract-acked-events: update the subscription map's pending ack counts +:: +++ subtract-acked-events + |= [acked=(map @ud @ud) unacked=(map @ud @ud)] + ^+ unacked + %+ roll ~(tap by acked) + |= [[rid=@ud ack=@ud] unacked=_unacked] + ?~ sus=(~(get by unacked) rid) + unacked + %+ ~(put by unacked) rid + ?: (lte u.sus ack) 0 + (sub u.sus ack) +:: +find-channel-mode: deduce requested mode from headers +:: +++ find-channel-mode + |= [met=method:http hes=header-list:http] + ^- ?(%json %jam) + =+ ^- [hed=@t jam=@t] + ?: ?=(%'GET' met) ['x-channel-format' 'application/x-urb-jam'] + ['content-type' 'application/x-urb-jam'] + =+ typ=(bind (get-header:http hed hes) :(cork trip cass crip)) + ?:(=(`jam typ) %jam %json) +:: +parse-channel-request: parses a list of channel-requests +:: +++ parse-channel-request + |= [mode=?(%json %jam) body=octs] + ^- (each (list channel-request) @t) + ?- mode + %json + ?~ maybe-json=(de:json:html q.body) + |+'put body not json' + ?~ maybe-requests=(parse-channel-request-json u.maybe-json) + |+'invalid channel json' + &+u.maybe-requests + :: + %jam + ?~ maybe-noun=(bind (slaw %uw q.body) cue) + |+'invalid request format' + ?~ maybe-reqs=((soft (list channel-request)) u.maybe-noun) + ~& [%miss u.maybe-noun] + |+'invalid request data' + &+u.maybe-reqs + == +:: +parse-channel-request-json: parses a json list of channel-requests +:: +:: Parses a json array into a list of +channel-request. If any of the items +:: in the list fail to parse, the entire thing fails so we can 400 properly +:: to the client. +:: +++ parse-channel-request-json + |= request-list=json + ^- (unit (list channel-request)) + :: parse top + :: + =, dejs-soft:format + =- ((ar -) request-list) + :: + |= item=json + ^- (unit channel-request) + :: + ?~ maybe-key=((ot action+so ~) item) + ~ + ?: =('ack' u.maybe-key) + ((pe %ack (ot event-id+ni ~)) item) + ?: =('poke' u.maybe-key) + %. item + %+ pe %poke-json + (ot id+ni ship+(su fed:ag) app+so mark+(su sym) json+some ~) + ?: =('subscribe' u.maybe-key) + %. item + %+ pe %subscribe + (ot id+ni ship+(su fed:ag) app+so path+(su stap) ~) + ?: =('unsubscribe' u.maybe-key) + %. item + %+ pe %unsubscribe + (ot id+ni subscription+ni ~) + ?: =('delete' u.maybe-key) + `[%delete ~] + :: if we reached this, we have an invalid action key. fail parsing. + :: + ~ +:: +auth-styling: css for login and eauth pages +:: +++ auth-styling + ''' + @import url("https://rsms.me/inter/inter.css"); + @font-face { + font-family: "Source Code Pro"; + src: url("https://storage.googleapis.com/media.urbit.org/fonts/scp-regular.woff"); + font-weight: 400; + font-display: swap; + } + :root { + --red-soft: #FFEFEC; + --red: #FF6240; + --gray-100: #E5E5E5; + --gray-400: #999999; + --gray-800: #333333; + --white: #FFFFFF; + } + html { + font-family: Inter, sans-serif; + height: 100%; + margin: 0; + width: 100%; + background: var(--white); + color: var(--gray-800); + -webkit-font-smoothing: antialiased; + line-height: 1.5; + font-size: 16px; + font-weight: 600; + display: flex; + flex-flow: row nowrap; + justify-content: center; + } + body { + display: flex; + flex-flow: column nowrap; + justify-content: center; + max-width: 300px; + padding: 1rem; + width: 100%; + } + body.local #eauth, + body.eauth #local { + display: none; + min-height: 100%; + } + #eauth input { + /*NOTE dumb hack to get approx equal height with #local */ + margin-bottom: 15px; + } + body nav { + background: var(--gray-100); + border-radius: 2rem; + display: flex; + justify-content: space-around; + overflow: hidden; + margin-bottom: 1rem; + } + body nav div { + width: 50%; + padding: 0.5rem 1rem; + text-align: center; + cursor: pointer; + } + body.local nav div.local, + body.eauth nav div.eauth { + background: var(--gray-800); + color: var(--white); + cursor: default; + } + nav div.local { + border-right: none; + border-top-right-radius: 0; + border-bottom-right-radius: 0; + } + nav div.eauth { + border-left: none; + border-top-left-radius: 0; + border-bottom-left-radius: 0; + } + body > *, + form > input { + width: 100%; + } + form { + display: flex; + flex-flow: column; + align-items: flex-start; + } + input { + background: var(--gray-100); + border: 2px solid transparent; + padding: 0.5rem; + border-radius: 0.5rem; + font-size: inherit; + color: var(--gray-800); + box-shadow: none; + width: 100%; + } + input:disabled { + background: var(--gray-100); + color: var(--gray-400); + } + input:focus { + outline: none; + background: var(--white); + border-color: var(--gray-400); + } + input:invalid:not(:focus) { + background: var(--red-soft); + border-color: var(--red); + outline: none; + color: var(--red); + } + button[type=submit] { + margin-top: 1rem; + } + button[type=submit], a.button { + font-size: 1rem; + padding: 0.5rem 1rem; + border-radius: 0.5rem; + background: var(--gray-800); + color: var(--white); + border: none; + font-weight: 600; + text-decoration: none; + } + input:invalid ~ button[type=submit] { + border-color: currentColor; + background: var(--gray-100); + color: var(--gray-400); + pointer-events: none; + } + span.guest, span.guest a { + color: var(--gray-400); + } + span.failed { + display: flex; + flex-flow: row nowrap; + height: 1rem; + align-items: center; + margin-top: 0.875rem; + color: var(--red); + } + span.failed svg { + height: 1rem; + margin-right: 0.25rem; + } + span.failed path { + fill: transparent; + stroke-width: 2px; + stroke-linecap: round; + stroke: currentColor; + } + .mono { + font-family: 'Source Code Pro', monospace; + } + @media all and (prefers-color-scheme: dark) { + :root { + --white: #000000; + --gray-800: #E5E5E5; + --gray-400: #808080; + --gray-100: #333333; + --red-soft: #7F1D1D; + } + } + @media screen and (min-width: 30em) { + html { + font-size: 14px; + } + } + ''' +:: +login-page: internal page to login to an Urbit +:: +++ login-page + |= [redirect-url=(unit @t) our=@p =identity eauth=(unit ?) failed=?] + ^- octs + =+ redirect-str=?~(redirect-url "" (trip u.redirect-url)) + %- as-octs:mimes:html + %- crip + %- en-xml:html + =/ favicon %+ + weld "<svg width='10' height='10' viewBox='0 0 10 10' xmlns='http://www.w3.org/2000/svg'>" + "<circle r='3.09' cx='5' cy='5' /></svg>" + ;html + ;head + ;meta(charset "utf-8"); + ;meta(name "viewport", content "width=device-width, initial-scale=1, shrink-to-fit=no"); + ;link(rel "icon", type "image/svg+xml", href (weld "data:image/svg+xml;utf8," favicon)); + ;title:"Urbit" + ;style:"{(trip auth-styling)}" + ;style:"{?^(eauth "" "nav \{ display: none; }")}" + ;script:"our = '{(scow %p our)}';" + ;script:''' + let name, pass; + function setup(isEauth) { + name = document.getElementById('name'); + pass = document.getElementById('pass'); + if (isEauth) goEauth(); else goLocal(); + } + function goLocal() { + document.body.className = 'local'; + pass.focus(); + } + function goEauth() { + document.body.className = 'eauth'; + name.focus(); + } + function doEauth() { + if (name.value == our) { + event.preventDefault(); + goLocal(); + } + } + ''' + == + ;body + =class "{?:(=(`& eauth) "eauth" "local")}" + =onload "setup({?:(=(`& eauth) "true" "false")})" + ;div#local + ;p:"Urbit ID" + ;input(value "{(scow %p our)}", disabled "true", class "mono"); + ;+ ?: =(%ours -.identity) + ;div + ;p:"Already authenticated" + ;a.button/"{(trip (fall redirect-url '/'))}":"Continue" + == + ;form(action "/~/login", method "post", enctype "application/x-www-form-urlencoded") + ;p:"Access Key" + ;input + =type "password" + =name "password" + =id "pass" + =placeholder "sampel-ticlyt-migfun-falmel" + =class "mono" + =required "true" + =minlength "27" + =maxlength "27" + =pattern "((?:[a-z]\{6}-)\{3}(?:[a-z]\{6}))"; + ;input(type "hidden", name "redirect", value redirect-str); + ;+ ?. failed ;span; + ;span.failed + ;svg(xmlns "http://www.w3.org/2000/svg", viewBox "0 0 16 16") + ;path(d "m8 8 4-4M8 8 4 4m4 4-4 4m4-4 4 4"); + == + Key is incorrect + == + ;button(type "submit"):"Continue" + == + == + ;div#eauth + ;form(action "/~/login", method "post", onsubmit "return doEauth()") + ;p:"Urbit ID" + ;input.mono + =name "name" + =id "name" + =placeholder "{(scow %p our)}" + =required "true" + =minlength "4" + =maxlength "57" + =pattern "~((([a-z]\{6})\{1,2}-\{0,2})+|[a-z]\{3})"; + ;p + ; You will be redirected to your own web interface to authorize + ; logging in to + ;span.mono:"{(scow %p our)}" + ; . + == + ;input(type "hidden", name "redirect", value redirect-str); + ;button(name "eauth", type "submit"):"Continue" + == + == + ;* ?: ?=(%ours -.identity) ~ + =+ as="proceed as{?:(?=(%fake -.identity) " guest" "")}" + ;+ ;span.guest.mono + ; Or try to + ;a/"{(trip (fall redirect-url '/'))}":"{as}" + ; . + == + == + ;script:''' + var failSpan = document.querySelector('.failed'); + if (failSpan) { + document.querySelector("input[type=password]") + .addEventListener('keyup', function (event) { + failSpan.style.display = 'none'; + }); + } + ''' + == +:: +eauth-error-page: render an eauth error reporting page +:: +:: optionally redirects the user back to either the login page if we're +:: acting as server, or the host if we're the client. +:: +++ eauth-error-page + |= $= return + $? ~ :: no known return target + [%server last=@t] :: we are the host, return to login + [%client goal=@t] :: we are the client, return to host + == + ^- octs + %- as-octs:mimes:html + %- crip + %- en-xml:html + =/ return=(unit @t) + ?- return + ~ ~ + [%server *] %- some + %^ cat 3 '/~/login?eauth&redirect=' + (crip (en-urlt:html (trip last.return))) + [%client *] `goal.return ::TODO plus nonce? or abort? + == + =/ favicon %+ + weld "<svg width='10' height='10' viewBox='0 0 10 10' xmlns='http://www.w3.org/2000/svg'>" + "<circle r='3.09' cx='5' cy='5' /></svg>" + =/ msg=tape + ?~ return "Something went wrong!" + "Something went wrong! You will be redirected back..." + ;html + ;head + ;* ?~ return ~ + :_ ~ + ;meta(http-equiv "Refresh", content "5; url={(trip u.return)}"); + ;meta(charset "utf-8"); + ;meta(name "viewport", content "width=device-width, initial-scale=1, shrink-to-fit=no"); + ;link(rel "icon", type "image/svg+xml", href (weld "data:image/svg+xml;utf8," favicon)); + ;title:"Urbit" + ;style:''' + @import url("https://rsms.me/inter/inter.css"); + :root { + --black60: rgba(0,0,0,0.6); + --white: rgba(255,255,255,1); + } + html { + font-family: Inter, sans-serif; + height: 100%; + margin: 0; + width: 100%; + background: var(--white); + color: var(--black60); + -webkit-font-smoothing: antialiased; + line-height: 1.5; + font-size: 12px; + display: flex; + flex-flow: row nowrap; + justify-content: center; + } + body { + display: flex; + flex-flow: column nowrap; + justify-content: center; + max-width: 300px; + padding: 1rem; + width: 100%; + } + ''' + == + ;body:"{msg}" + == +:: +render-tang-to-marl: renders a tang and adds <br/> tags between each line +:: +++ render-tang-to-marl + |= [wid=@u tan=tang] + ^- marl + =/ raw=(list tape) (zing (turn tan |=(a=tank (wash 0^wid a)))) + :: + |- ^- marl + ?~ raw ~ + [;/(i.raw) ;br; $(raw t.raw)] +:: +render-tang-to-wall: renders tang as text lines +:: +++ render-tang-to-wall + |= [wid=@u tan=tang] + ^- wall + (zing (turn tan |=(a=tank (wash 0^wid a)))) +:: +wall-to-octs: text to binary output +:: +++ wall-to-octs + |= =wall + ^- (unit octs) + :: + ?: =(~ wall) + ~ + :: + :- ~ + %- as-octs:mimes:html + %- crip + %- zing ^- ^wall + %- zing ^- (list ^wall) + %+ turn wall + |= t=tape + ^- ^wall + ~[t "\0a"] +:: +internal-server-error: 500 page, with a tang +:: +++ internal-server-error + |= [authorized=? url=@t t=tang] + ^- octs + %- as-octs:mimes:html + %- crip + %- en-xml:html + ;html + ;head + ;title:"500 Internal Server Error" + == + ;body + ;h1:"Internal Server Error" + ;p:"There was an error while handling the request for {(trip url)}." + ;* ?: authorized + ;= + ;code:"*{(render-tang-to-marl 80 t)}" + == + ~ + == + == +:: +error-page: error page, with an error string if logged in +:: +++ error-page + |= [code=@ud authorized=? url=@t t=tape] + ^- octs + =/ message=tape + ?+ code "{(scow %ud code)} Error" + %400 "Bad Request" + %403 "Forbidden" + %404 "Not Found" + %405 "Method Not Allowed" + %500 "Internal Server Error" + == + :: + %- as-octs:mimes:html + %- crip + %- en-xml:html + ;html + ;head + ;title:"{(a-co:co code)} {message}" + == + ;body + ;h1:"{message}" + ;p:"There was an error while handling the request for {(trip url)}." + ;* ?: authorized + ;= + ;code:"{t}" + == + ~ + == + == +:: +host-matches: %.y if the site :binding should be used to handle :host +:: +++ host-matches + |= [binding=(unit @t) host=(unit @t)] + ^- ? + :: if the binding allows for matching anything, match + :: + ?~ binding + %.y + :: if the host is ~, that means we're trying to bind nothing to a real + :: binding. fail. + :: + ?~ host + %.n + :: otherwise, do a straight comparison + :: + =(u.binding u.host) +:: +find-suffix: returns [~ /tail] if :full is (weld :prefix /tail) +:: +++ find-suffix + |= [prefix=path full=path] + ^- (unit path) + ?~ prefix + `full + ?~ full + ~ + ?. =(i.prefix i.full) + ~ + $(prefix t.prefix, full t.full) +:: +simplified-url-parser: returns [(each @if @t) (unit port=@ud)] +:: +++ simplified-url-parser + ;~ plug + ;~ pose + %+ stag %ip + =+ tod=(ape:ag ted:ab) + %+ bass 256 + ;~(plug tod (stun [3 3] ;~(pfix dot tod))) + :: + (stag %site (cook crip (star ;~(pose dot alp)))) + == + ;~ pose + (stag ~ ;~(pfix col dim:ag)) + (easy ~) + == + == +:: +host-sans-port: strip the :<port> from a host string +:: +++ host-sans-port + ;~ sfix + %+ cook crip + %- star + ;~ less + ;~(plug col (punt dem) ;~(less next (easy ~))) + next + == + (star next) + == +:: +per-server-event: per-event server core +:: +++ per-server-event + ~% %eyre-per-server-event ..part ~ + :: gate that produces the +per-server-event core from event information + :: + |= [[eny=@ =duct now=@da rof=roof] state=server-state] + =/ eyre-id (scot %ta (cat 3 'eyre_' (scot %uv (sham duct)))) + |% + :: +request-local: bypass authentication for local lens connections + :: + ++ request-local + |= [secure=? =address =request:http] + ^- [(list move) server-state] + :: + =/ act [%app app=%lens] + :: + =/ connection=outstanding-connection + [act [& secure address request] [*@uv [%ours ~]] ~ 0] + :: + =. connections.state + %. (~(put by connections.state) duct connection) + (trace 2 |.("{<duct>} creating local")) + :: + (request-to-app [%ours ~] app.act inbound-request.connection) + :: +request: starts handling an inbound http request + :: + ++ request + |= [secure=? =address =request:http] + ^- [(list move) server-state] + =* headers header-list.request + :: for requests from localhost, respect the "forwarded" header + :: + =/ [secure=? host=(unit @t) =^address] + =/ host=(unit @t) (get-header:http 'host' headers) + =* same [secure host address] + ?. =([%ipv4 .127.0.0.1] address) same + ?~ forwards=(forwarded-params headers) same + :+ (fall (forwarded-secure u.forwards) secure) + (clap (forwarded-host u.forwards) host head) + (fall (forwarded-for u.forwards) address) + :: + =/ [=action suburl=@t] + (get-action-for-binding host url.request) + :: + ::TODO we might want to mint new identities only for requests that end + :: up going into userspace, not the ones that get handled by eyre. + :: perhaps that distinction, where userspace requests are async, but + :: eyre-handled requests are always synchronous, provides a fruitful + :: angle for refactoring... + =^ ?(invalid=@uv [suv=@uv =identity som=(list move)]) state + (session-for-request:authentication request) + ?@ - + :: the request provided a session cookie that's not (or no longer) + :: valid. to make sure they're aware, tell them 401 + :: + ::NOTE some code duplication with below, but request handling deserves + :: a refactor anyway + =. connections.state + ::NOTE required by +handle-response. + :: the session identity we provide here doesn't actually exist. + :: that's fine: we call +handle-response for this connection right + :: away, that no-ops for the non-existing session, and then + :: deletes the connection from state. + %+ ~(put by connections.state) duct + ^- outstanding-connection + [action [| secure address request] [invalid %fake *@p] ~ 0] + :: their cookie was invalid, make sure they expire it + :: + =/ bod=octs (as-octs:mimes:html 'bad session auth') + %- handle-response + :* %start + :- 401 + :~ ['set-cookie' (session-cookie-string:authentication invalid ~)] + ['content-length' (crip (a-co:co p.bod))] + == + `bod + complete=%.y + == + =; [moz=(list move) sat=server-state] + [(weld som moz) sat] + :: + =/ authenticated ?=(%ours -.identity) + :: if we have no eauth endpoint yet, and the request is authenticated, + :: deduce it from the hostname + :: + =? endpoint.auth.state + ?& authenticated + ?=(^ host) + ?=(~ auth.endpoint.auth.state) + == + %- (trace 2 |.("eauth: storing endpoint at {(trip u.host)}")) + :+ user.endpoint.auth.state + `(cat 3 ?:(secure 'https://' 'http://') u.host) + now + :: record that we started an asynchronous response + :: + =/ connection=outstanding-connection + [action [authenticated secure address request] [suv identity] ~ 0] + =. connections.state + :: NB: required by +handle-response and +handle-request:authentication. + :: XX optimize, not all requests are asynchronous + :: + (~(put by connections.state) duct connection) + :: redirect to https if insecure, redirects enabled + :: and secure port live + :: + ?: ?& !secure + redirect.http-config.state + ?=(^ secure.ports.state) + == + =/ location=@t + %+ rap 3 + :~ 'https://' + (rash (fall host '') host-sans-port) + ?: =(443 u.secure.ports.state) + '' + (crip ":{(a-co:co u.secure.ports.state)}") + ?: ?=([[~ ~] ~] (parse-request-line url.request)) + '/' + url.request + == + %- handle-response + :* %start + :- status-code=301 + headers=['location' location]~ + data=~ + complete=%.y + == + :: figure out whether this is a cors request, + :: whether the origin is approved or not, + :: and maybe add it to the "pending approval" set + :: + =/ origin=(unit origin) + (get-header:http 'origin' headers) + =^ cors-approved requests.cors-registry.state + =, cors-registry.state + ?~ origin [| requests] + ?: (~(has in approved) u.origin) [& requests] + ?: (~(has in rejected) u.origin) [| requests] + [| (~(put in requests) u.origin)] + :: if this is a cors preflight request from an approved origin + :: handle it synchronously + :: + ?: &(?=(^ origin) cors-approved ?=(%'OPTIONS' method.request)) + %- handle-response + =; =header-list:http + [%start [204 header-list] ~ &] + :: allow the method and headers that were asked for, + :: falling back to wildcard if none specified + :: + ::NOTE +handle-response will add the rest of the headers + :: + :~ :- 'Access-Control-Allow-Methods' + =- (fall - '*') + (get-header:http 'access-control-request-method' headers) + :: + :- 'Access-Control-Allow-Headers' + =- (fall - '*') + (get-header:http 'access-control-request-headers' headers) + == + :: handle HTTP scries + :: + :: TODO: ideally this would look more like: + :: + :: ?^ p=(parse-http-scry url.request) + :: (handle-http-scry authenticated p request) + :: + ?: =('/_~_/' (end [3 5] url.request)) + (handle-http-scry authenticated request) + :: handle requests to the cache, if a non-empty entry exists + :: + =/ cached=(unit [aeon=@ud val=(unit cache-entry)]) + (~(get by cache.state) url.request) + ?: &(?=([~ @ ^] cached) ?=(%'GET' method.request)) + (handle-cache-req authenticated request u.val.u.cached) + :: + ~& >> eyre-request-action=-.action + ?- -.action + %gen + =/ bek=beak [our desk.generator.action da+now] + =/ sup=spur path.generator.action + =/ ski (rof [~ ~] /eyre %ca bek sup) + =/ cag=cage (need (need ski)) + ?> =(%vase p.cag) + =/ gat=vase !<(vase q.cag) + =/ res=toon + %- mock :_ (look rof [~ ~] /eyre) + :_ [%9 2 %0 1] |. + %+ slam + %+ slam gat + !>([[now=now eny=eny bek=bek] ~ ~]) + ::TODO should get passed the requester's identity + !>([authenticated request]) + ?: ?=(%2 -.res) + =+ connection=(~(got by connections.state) duct) + %^ return-static-data-on-duct 500 'text/html' + %: internal-server-error + authenticated.inbound-request.connection + url.request.inbound-request.connection + leaf+"generator crashed" + p.res + == + ?: ?=(%1 -.res) + =+ connection=(~(got by connections.state) duct) + %^ return-static-data-on-duct 500 'text/html' + %: internal-server-error + authenticated.inbound-request.connection + url.request.inbound-request.connection + leaf+"scry blocked on" + (fall (bind (bind ((soft path) p.res) smyt) (late ~)) ~) + == + =/ result ;;(simple-payload:http +.p.res) + :: ensure we have a valid content-length header + :: + :: We pass on the response and the headers the generator produces, but + :: ensure that we have a single content-length header set correctly in + :: the returned if this has a body, and has no content-length if there + :: is no body returned to the client. + :: + =. headers.response-header.result + ?~ data.result + (delete-header:http 'content-length' headers.response-header.result) + :: + %^ set-header:http 'content-length' + (crip (a-co:co p.u.data.result)) + headers.response-header.result + :: + %- handle-response + ^- http-event:http + :* %start + response-header.result + data.result + complete=%.y + == + :: + %app + (request-to-app identity app.action inbound-request.connection) + :: + %authentication + (handle-request:authentication secure host address [suv identity] request) + :: + %eauth + (on-request:eauth:authentication [suv identity] request) + :: + %logout + (handle-logout:authentication [suv identity] request) + :: + %channel + (handle-request:by-channel [suv identity] address request) + :: + %scry + (handle-scry authenticated address request(url suburl)) + :: + %name + (handle-name identity request) + :: + %host + %^ return-static-data-on-duct 200 'text/plain' + (as-octs:mimes:html (scot %p our)) + :: + %ip + (handle-ip address request) + %boot + (handle-boot identity request) + :: + %sponsor + (handle-sponsor identity request) + :: + %four-oh-four + %^ return-static-data-on-duct 404 'text/html' + (error-page 404 authenticated url.request ~) + == + ++ ws-event + |= [wid=@ event=websocket-event] + =/ conn (~(get by connections.state) duct) + ?~ conn `state + ?. ?=(%app -.action.u.conn) `state + =/ app app.action.u.conn + ~& ws-event=[wid app] + =/ identity [%ours ~] + =/ wsid (scot %p wid) + :: TODO damn how + ?+ -.event `state + %message + :_ state + :~ %+ deal-as + /run-ws-app-request/(scot %uw (cut 3 [2 4] eny)) + :^ identity our app + :+ %poke %websocket-server-message + !>([wid message.event]) + == + %disconnect + =. connections.state (~(del by connections.state) duct) + :_ state + :~ %+ deal-as + /ws-watch-response/[wsid] + [identity our app %leave ~] + == + == + ++ ws-handshake + |= [wid=@ secure=? =address:eyre =request:http] + ^- [(list move) server-state] + ~& >>> sending-ws-handshake=[wid eyre-id] + + =/ host=(unit @t) (get-header:http 'host' header-list.request) + =/ [=action suburl=@t] + (get-action-for-binding host url.request) + :: TODO enable other actions + ?> ?=(%app -.action) + + :: TODO!! get clear what all the identity thing has to be + =/ =identity [%ours ~] + =/ app app.action + =^ ?(invalid=@uv [suv=@uv fi=^identity som=(list move)]) state + (session-for-request:authentication request) + =/ connection=outstanding-connection + ?@ - ~& invalid-session=- + [action [.n secure address request] [invalid identity] ~ 0] + [action [.n secure address request] [suv identity] ~ 0] + + + =. connections.state + (~(put by connections.state) duct connection) + :: eyre-id is assigned way up in this arm + =/ wsid (scot %ud wid) + :_ state + :~ %+ deal-as + /ws-watch-response/[wsid] + [identity our app %watch /websocket-server/[wsid]] + :: + %+ deal-as + /run-ws-app-request/[wsid] + :^ identity our app + :+ %poke %websocket-handshake + !>(`[@ inbound-request:eyre]`[wid inbound-request.connection]) + == + + :: +handle-ip: respond with the requester's ip + :: + ++ handle-ip + |= [=address =request:http] + ^- (quip move server-state) + ?. =(%'GET' method.request) + %^ return-static-data-on-duct 405 'text/html' + (error-page 405 & url.request "may only GET ip") + %^ return-static-data-on-duct 200 'text/plain' + =/ ip=@t + ?- address + [%ipv4 *] + (crip (tail ~(rend co [%$ %if +.address]))) + :: + [%ipv6 *] + %- crip + %+ scan ~(rend co [%$ %is +.address]) + ;~(pfix dot (star ;~(pose (cold ':' dot) next))) + == + (as-octs:mimes:html ip) + :: Get current sponsor of ship + :: + ++ galaxy-for + |= =ship + ^- @p + =/ next (^^sein:title rof /eyre our now ship) + ?: ?=(%czar (clan:title next)) + next + $(ship next) + :: + ++ handle-sponsor + |= [=identity =request:http] + ^- (quip move server-state) + =/ crumbs q:(rash url.request apat:de-purl:html) + ?. ?=([@t @t @t ~] crumbs) + %^ return-static-data-on-duct 400 'text/html' + %: error-page + 400 + & + url.request + "Invalid input: Expected /~/boot/<ship=@p>" + == + =/ ship + %+ slaw + %p + i.t.t.crumbs + ?~ ship + %^ return-static-data-on-duct 400 'text/html' + %: error-page + 400 + & + url.request + "Invalid input: Expected /~/boot/<ship=@p>" + == + %^ return-static-data-on-duct 200 'text/plain' + (as-octs:mimes:html (scot %p (galaxy-for u.ship))) + :: Returns peer-state data for verifying sync status between ship and network. + :: Takes two path parameters - ship=@p and optional bone=@u. + :: + :: Responds with a @uw jam of a noun containing: + :: version, sponsor, rift, life, bone as a unit (null if not provided), and + :: the last-acked message-num of the provided ship and bone as a unit (null if + :: bone is not provided). + :: + :: Responds with a 404 error-page if either: + :: - Peer is not %known + :: - Bone was not found under peer (assuming bone was provided) + :: + ++ handle-boot + |= [=identity =request:http] + ^- (quip move server-state) + ?. =(%'GET' method.request) + %^ return-static-data-on-duct 405 'text/html' + (error-page 405 & url.request "may only GET boot data") + =/ crumbs q:(rash url.request apat:de-purl:html) + => .(crumbs `(pole knot)`crumbs) + ?. ?=([%'~' %boot ship=@t req=*] crumbs) + %^ return-static-data-on-duct 400 'text/html' + %: error-page + 400 + & + url.request + "Invalid input: Expected /~/boot/<ship=@p> or /~/boot/<ship=@p>/<bone=@u>" + == + =/ ship=(unit ship) (slaw %p ship.crumbs) + =/ bone=(unit @ud) + ?. ?=([bone=@t ~] req.crumbs) ~ + (rush bone.req.crumbs dem) + ?: ?| ?=(~ ship) + &(?=([bone=@ ~] req.crumbs) ?=(~ bone)) + == + %^ return-static-data-on-duct 400 'text/html' + %: error-page + 400 + & + url.request + "Invalid input: Expected /~/boot/<ship=@p> or /~/boot/<ship=@p>/<bone=@u>" + == + :: + =/ des=(unit (unit cage)) + %: rof + [~ ~] + /eyre + %ax + [our %$ da+now] + :+ %boot (scot %p u.ship) + ?~(bone ~ [(scot %ud u.bone) ~]) :: XX + == + ?. ?=([~ ~ %noun *] des) + %^ return-static-data-on-duct 404 'text/html' + (error-page 404 & url.request "Peer {(scow %p u.ship)} not found.") + =+ !< [rift=@ud life=@ud bone=(unit @ud) last-acked=(unit @ud)] q.u.u.des + %^ return-static-data-on-duct 200 'application/octet-stream' + %- as-octs:mimes:html + %- jam + ^- boot + [%1 (galaxy-for u.ship) rift life bone last-acked] + :: +handle-name: respond with the requester's @p + :: + ++ handle-name + |= [=identity =request:http] + ^- (quip move server-state) + ?. =(%'GET' method.request) + %^ return-static-data-on-duct 405 'text/html' + (error-page 405 & url.request "may only GET name") + %^ return-static-data-on-duct 200 'text/plain' + =/ nom=@p + ?+(-.identity who.identity %ours our) + (as-octs:mimes:html (scot %p nom)) + :: +handle-http-scry: respond with scry result + :: + ++ handle-http-scry + |= [authenticated=? =request:http] + |^ ^- (quip move server-state) + ?. authenticated (error-response 403 ~) + ?. =(%'GET' method.request) + (error-response 405 "may only GET scries") + =/ req (parse-request-line url.request) + =/ fqp (fully-qualified site.req) + =/ mym (scry-mime now rof [~ ~] ext.req site.req) + ?: ?=(%| -.mym) (error-response 500 p.mym) + =* mime p.mym + %- handle-response + :* %start + :- status-code=200 + ^= headers + :~ ['content-type' (rsh 3 (spat p.mime))] + ['content-length' (crip (a-co:co p.q.mime))] + ['cache-control' ?:(fqp 'max-age=31536000' 'no-cache')] + == + data=[~ q.mime] + complete=%.y + == + :: + ++ fully-qualified + |= a=path + ^- ? + ?. ?=([%'_~_' @ @ @ *] a) %.n + =/ vez (vang | (en-beam [our %base da+now] ~)) + ?= [~ [^ ^ ^ *]] (rush (spat t.t.a) ;~(pfix fas gash:vez)) + :: + ++ error-response + |= [status=@ud =tape] + ^- (quip move server-state) + %^ return-static-data-on-duct status 'text/html' + (error-page status authenticated url.request tape) + -- + :: +handle-cache-req: respond with cached value, 404 or 500 + :: + ++ handle-cache-req + |= [authenticated=? =request:http entry=cache-entry] + |^ ^- (quip move server-state) + ?: &(auth.entry !authenticated) + (error-response 403 ~) + =* body body.entry + ?- -.body + %payload + %- handle-response + :* %start + response-header.simple-payload.body + data.simple-payload.body + complete=%.y + == + == + :: + ++ error-response + |= [status=@ud =tape] + ^- (quip move server-state) + %^ return-static-data-on-duct status 'text/html' + (error-page status authenticated url.request tape) + -- + :: +handle-scry: respond with scry result, 404 or 500 + :: + ++ handle-scry + |= [authenticated=? =address =request:http] + |^ ^- (quip move server-state) + ?. authenticated + (error-response 403 ~) + ?. =(%'GET' method.request) + (error-response 405 "may only GET scries") + :: make sure the path contains an app to scry into + :: + =+ req=(parse-request-line url.request) + ?. ?=(^ site.req) + (error-response 400 "scry path must start with app name") + :: attempt the scry that was asked for + :: + =/ res=(unit (unit cage)) + (do-scry %gx i.site.req (snoc t.site.req (fall ext.req %mime))) + ?~ res (error-response 500 "failed scry") + ?~ u.res (error-response 404 "no scry result") + =* mark p.u.u.res + =* vase q.u.u.res + ?: =(%mime mark) + =/ =mime !<(mime vase) + %^ return-static-data-on-duct 200 + (rsh 3 (spat p.mime)) q.mime + :: attempt to find conversion gate to mime + :: + =/ tub=(unit [tub=tube:clay mov=move]) + (find-tube i.site.req mark %mime) + ?~ tub (error-response 500 "no tube from {(trip mark)} to mime") + :: attempt conversion, then send results + :: + =/ mym=(each mime tang) + (mule |.(!<(mime (tub.u.tub vase)))) + =^ cards state + ?- -.mym + %| (error-response 500 "failed tube from {(trip mark)} to mime") + %& %+ return-static-data-on-duct 200 + [(rsh 3 (spat p.p.mym)) q.p.mym] + == + [[mov.u.tub cards] state] + :: + ++ find-tube + |= [dap=term from=mark to=mark] + ^- (unit [tube:clay move]) + =/ des=(unit (unit cage)) + (do-scry %gd dap /$) + ?. ?=([~ ~ *] des) ~ + =+ !<(=desk q.u.u.des) + =/ tub=(unit (unit cage)) + (do-scry %cc desk /[from]/[to]) + ?. ?=([~ ~ %tube *] tub) ~ + :- ~ + :- !<(tube:clay q.u.u.tub) + :^ duct %pass /conversion-cache/[from] + [%c %warp our desk `[%sing %c da+now /[from]/[to]]] + :: + ++ do-scry + |= [care=term =desk =path] + ^- (unit (unit cage)) + (rof [~ ~] /eyre care [our desk da+now] path) + :: + ++ error-response + |= [status=@ud =tape] + ^- (quip move server-state) + %^ return-static-data-on-duct status 'text/html' + (error-page status authenticated url.request tape) + -- + :: +request-to-app: subscribe to app and poke it with request data + :: + ++ request-to-app + |= [=identity app=term =inbound-request:eyre] + ^- (quip move server-state) + :: if the agent isn't running, we synchronously serve a 503 + :: + ?. !<(? q:(need (need (rof [~ ~] /eyre %gu [our app da+now] /$)))) + %^ return-static-data-on-duct 503 'text/html' + %: error-page + 503 + ?=(%ours -.identity) + url.request.inbound-request + "%{(trip app)} not running" + == + :: otherwise, subscribe to the agent and poke it with the request + :: + :_ state + :~ %+ deal-as + /watch-response/[eyre-id] + [identity our app %watch /http-response/[eyre-id]] + :: + %+ deal-as + /run-app-request/[eyre-id] + :^ identity our app + :+ %poke %handle-http-request + !>(`[@ta inbound-request:eyre]`[eyre-id inbound-request]) + == + :: +cancel-request: handles a request being externally aborted + :: + ++ cancel-request + ^- [(list move) server-state] + :: + ?~ connection=(~(get by connections.state) duct) + :: nothing has handled this connection + :: + [~ state] + :: + =. connections.state (~(del by connections.state) duct) + :: + ?- -.action.u.connection + %gen [~ state] + %app + :_ state + :_ ~ + =, u.connection + %- (trace 1 |.("leaving subscription to {<app.action>}")) + (deal-as /watch-response/[eyre-id] identity our app.action %leave ~) + :: + ?(%authentication %eauth %logout) + ::NOTE expiry timer will clean up cancelled eauth attempts + [~ state] + :: + %channel + on-cancel-request:by-channel + :: + ?(%scry %four-oh-four %name %host %ip %boot %sponsor) + :: it should be impossible for these to be asynchronous + :: + !! + == + :: +return-static-data-on-duct: returns one piece of data all at once + :: + ++ return-static-data-on-duct + |= [code=@ content-type=@t data=octs] + ^- [(list move) server-state] + :: + %- handle-response + :* %start + :- status-code=code + ^= headers + :~ ['content-type' content-type] + ['content-length' (crip (a-co:co p.data))] + == + data=[~ data] + complete=%.y + == + :: +authentication: per-event authentication as this Urbit's owner + :: + :: Right now this hard codes the authentication page using the old +code + :: system, but in the future should be pluggable so we can use U2F or + :: WebAuthn or whatever is more secure than passwords. + :: + ++ authentication + |% + :: +handle-request: handles an http request for the login page + :: + ++ handle-request + |= [secure=? host=(unit @t) =address [session-id=@uv =identity] =request:http] + ^- [(list move) server-state] + :: parse the arguments out of request uri + :: + =+ request-line=(parse-request-line url.request) + =/ redirect (get-header:http 'redirect' args.request-line) + =/ with-eauth=(unit ?) + ?: =(~ eauth-url:eauth) ~ + `?=(^ (get-header:http 'eauth' args.request-line)) + :: if we received a simple get: show the login page + :: + ::NOTE we never auto-redirect, to avoid redirect loops with apps that + :: send unprivileged users to the login screen + :: + ?: =('GET' method.request) + %^ return-static-data-on-duct 200 'text/html' + (login-page redirect our identity with-eauth %.n) + :: if we are not a post, return an error + :: + ?. =('POST' method.request) + %^ return-static-data-on-duct 405 'text/html' + (login-page ~ our identity with-eauth %.n) + :: we are a post, and must process the body type as form data + :: + ?~ body.request + %^ return-static-data-on-duct 400 'text/html' + (login-page ~ our identity with-eauth %.n) + :: + =/ parsed=(unit (list [key=@t value=@t])) + (rush q.u.body.request yquy:de-purl:html) + ?~ parsed + %^ return-static-data-on-duct 400 'text/html' + (login-page ~ our identity with-eauth %.n) + :: + =/ redirect=(unit @t) (get-header:http 'redirect' u.parsed) + ?^ (get-header:http 'eauth' u.parsed) + ?~ ship=(biff (get-header:http 'name' u.parsed) (cury slaw %p)) + %^ return-static-data-on-duct 400 'text/html' + (login-page redirect our identity `& %.n) + ::TODO redirect logic here and elsewhere is ugly + =/ redirect (fall redirect '') + =/ base=(unit @t) + ?~ host ~ + `(cat 3 ?:(secure 'https://' 'http://') u.host) + (start:server:eauth u.ship base ?:(=(redirect '') '/' redirect)) + :: + =. with-eauth (bind with-eauth |=(? |)) + ?~ password=(get-header:http 'password' u.parsed) + %^ return-static-data-on-duct 400 'text/html' + (login-page redirect our identity with-eauth %.n) + :: check that the password is correct + :: + ?. =(u.password code) + %^ return-static-data-on-duct 400 'text/html' + (login-page redirect our identity with-eauth %.y) + :: clean up the session they're changing out from + :: + =^ moz state + (close-session session-id |) + :: initialize the new session + :: + =^ fex state (start-session %local) + :: associate the new session with the request that caused the login + :: + :: if we don't do this here, +handle-response will include the old + :: session's cookie, confusing the client. + :: + =. connections.state + %+ ~(jab by connections.state) duct + |= o=outstanding-connection + o(session-id session.fex) + :: store the hostname used for this login, later reuse it for eauth + :: + =? endpoint.auth.state + :: avoid overwriting public domains with localhost + :: + ?& ?=(^ host) + ?| ?=(~ auth.endpoint.auth.state) + !=('localhost' (fall (rush u.host host-sans-port) '')) + == == + %- (trace 2 |.("eauth: storing endpoint at {(trip u.host)}")) + =/ new-auth=(unit @t) + `(cat 3 ?:(secure 'https://' 'http://') u.host) + =, endpoint.auth.state + :+ user new-auth + :: only update the timestamp if the derived endpoint visibly changed. + :: that is, it's not hidden behind a user-provided hardcoded url, + :: and the new value is different from the old.) + :: + ?:(|(?=(^ user) =(new-auth auth)) time now) + :: + =; out=[moves=(list move) server-state] + out(moves [give-session-tokens :(weld moz moves.fex moves.out)]) + ::NOTE that we don't provide a 'set-cookie' header here. + :: +handle-response does that for us. + ::TODO that should really also handle the content-length header for us, + :: somewhat surprising that it doesn't... + %- handle-response + =/ bod=octs + (as-octs:mimes:html (scot %uv session.fex)) + =/ col=[key=@t value=@t] + ['content-length' (crip (a-co:co p.bod))] + ?~ redirect + [%start 200^~[col] `bod &] + =/ actual-redirect ?:(=(u.redirect '') '/' u.redirect) + [%start 303^~['location'^actual-redirect col] `bod &] + :: +handle-logout: handles an http request for logging out + :: + ++ handle-logout + |= [[session-id=@uv =identity] =request:http] + ^- [(list move) server-state] + :: whatever we end up doing, we always respond with a redirect + :: + =/ response=$>(%start http-event:http) + =/ redirect=(unit @t) + %+ get-header:http 'redirect' + args:(parse-request-line url.request) + :* %start + response-header=[303 ['location' (fall redirect '/~/login')]~] + data=~ + complete=%.y + == + :: read options from the body + :: all: log out all sessions with this identity? + :: sid: which session do we log out? (defaults to requester's) + :: hos: host to log out from, for eauth logins (sid signifies the nonce) + :: + =/ arg=header-list:http + ?~ body.request ~ + (fall (rush q.u.body.request yquy:de-purl:html) ~) + =/ all=? + ?=(^ (get-header:http 'all' arg)) + =/ sid=(unit @uv) + ?. ?=(%ours -.identity) `session-id + ?~ sid=(get-header:http 'sid' arg) `session-id + :: if you provided the parameter, but it doesn't parse, we just + :: no-op. otherwise, a poorly-implemented frontend might result in + :: accidental log-outs, which would be very annoying. + :: + (slaw %uv u.sid) + =/ hos=(unit @p) + ?. ?=(%ours -.identity) ~ + (biff (get-header:http 'host' arg) (cury slaw %p)) + ?~ sid + (handle-response response) + :: if this is an eauth remote logout, send the %shut + :: + =* auth auth.state + ?: ?=(^ hos) + =^ moz state (handle-response response) + :- [(send-plea:client:eauth u.hos %0 %shut u.sid) moz] + =/ book (~(gut by visiting.auth) u.hos *logbook) + =. qeu.book (~(put to qeu.book) u.sid) + =. visiting.auth (~(put by visiting.auth) u.hos book) + state + :: if the requester is logging themselves out, make them drop the cookie + :: + =? headers.response-header.response =(u.sid session-id) + :_ headers.response-header.response + ['set-cookie' (session-cookie-string session-id ~)] + :: close the session as requested, then send the response + :: + =^ moz1 state (close-session u.sid all) + =^ moz2 state (handle-response response) + [[give-session-tokens (weld moz1 moz2)] state] + :: +session-id-from-request: attempt to find a session token + :: + :: looks in the authorization header first. if there is no such header, + :: looks in the cookie header(s) instead. + :: + ++ session-id-from-request + |= =request:http + ^- (unit @uv) + :: is there an authorization header? + :: + ?^ auth=(get-header:http 'authorization' header-list.request) + (rush u.auth ;~(pfix (jest 'Bearer 0v') viz:ag)) + :: are there cookies passed with this request? + :: + =/ cookie-header=@t + %+ roll header-list.request + |= [[key=@t value=@t] c=@t] + ?. =(key 'cookie') + c + (cat 3 (cat 3 c ?~(c 0 '; ')) value) + :: is the cookie line valid? + :: + ?~ cookies=(rush cookie-header cock:de-purl:html) + ~ + :: is there an urbauth cookie? + :: + ?~ urbauth=(get-header:http (crip "urbauth-{(scow %p our)}") u.cookies) + ~ + :: if it's formatted like a valid session cookie, produce it + :: + `(unit @)`(rush u.urbauth ;~(pfix (jest '0v') viz:ag)) + :: +request-is-logged-in: checks to see if the request has non-guest id + :: + ++ request-is-logged-in + |= =request:http + ^- ? + ?~ session-id=(session-id-from-request request) + | + ?~ session=(~(get by sessions.auth.state) u.session-id) + | + &(!?=(%fake -.identity.u.session) (lte now expiry-time.u.session)) + :: +request-is-authenticated: checks to see if the request is "us" + :: + :: We are considered authenticated if this request has an urbauth + :: Cookie for the local identity that is not expired. + :: + ++ request-is-authenticated + |= =request:http + ^- ? + :: does the request pass a session cookie? + :: + ?~ session-id=(session-id-from-request request) + %.n + :: is this a session that we know about? + :: + ?~ session=(~(get by sessions.auth.state) `@uv`u.session-id) + %.n + :: does this session have our id, and is it still valid? + :: + &(?=(%ours -.identity.u.session) (lte now expiry-time.u.session)) + :: +start-session: create a new session with %local or %guest identity + :: + ++ start-session + |= kind=?(%local %guest [%eauth who=@p]) + ^- [[session=@uv =identity moves=(list move)] server-state] + =; [key=@uv sid=identity] + =/ timeout=@dr + =, session-timeout + ?:(?=(%guest kind) guest auth) + :- :+ key sid + :: if no session existed previously, we must kick off the + :: session expiry timer + :: + ?^ sessions.auth.state ~ + [duct %pass /sessions/expire %b %wait (add now timeout)]~ + =- state(sessions.auth -) + %+ ~(put by sessions.auth.state) key + [sid (add now timeout) ~] + :: create a new session with a fake identity + :: + =/ sik=@uv new-session-key + :- sik + ?: ?=(%local kind) [%ours ~] + ?: ?=([%eauth @] kind) [%real who.kind] + :- %fake + :: pre-scramble our ship name into its displayed value, and + :: truncate it to be at most moon-length, so that we can overlay + :: it onto the end of a comet name for visual consistency. + :: to prevent escalation, make sure the guest identity isn't ours. + :: + |- + =; nom=@p + ?. =(our nom) nom + $(eny (shas %next-name eny)) + %+ end 3^16 + %^ cat 3 + (end 3^8 (fein:ob our)) + (~(raw og (shas %fake-name eny)) 128) + :: +session-for-request: get the session details for the request + :: + :: returns the @ case if an invalid session is provided. + :: creates a guest session if the request does not have any session. + :: there is no need to call +give-session-tokens after this, because + :: guest sessions do not make valid "auth session" tokens. + :: + ++ session-for-request + |= =request:http + ^- [$@(session=@uv [session=@uv =identity moves=(list move)]) server-state] + ?~ sid=(session-id-from-request request) + (start-session %guest) + ?~ ses=(~(get by sessions.auth.state) u.sid) + [u.sid state] + ?: (gth now expiry-time.u.ses) + [u.sid state] + [[u.sid identity.u.ses ~] state] + :: +close-session: delete a session and its associated channels + :: + :: if :all is true, deletes all sessions that share the same identity. + :: if this closes an %ours session, the caller is responsible for + :: also calling +give-session-tokens afterwards. + :: + ++ close-session + |= [session-id=@uv all=?] + ^- [(list move) server-state] + ?~ ses=(~(get by sessions.auth.state) session-id) + [~ state] + :: delete the session(s) and find the associated ids & channels + :: + =^ [siz=(list @uv) channels=(list @t)] sessions.auth.state + =* sessions sessions.auth.state + :: either delete just the specific session and its channels, + :: + ?. all + :- [[session-id]~ ~(tap in channels.u.ses)] + (~(del by sessions) session-id) + :: or delete all sessions with the identity from :session-id + :: + %+ roll ~(tap by sessions) + |= $: [sid=@uv s=session] + [[siz=(list @uv) caz=(list @t)] sez=(map @uv session)] + == + ^+ [[siz caz] sez] + ?. =(identity.s identity.u.ses) + :: identity doesn't match, so re-store this session + :: + [[siz caz] (~(put by sez) sid s)] + :: identity matches, so register this session as closed + :: + [[[sid siz] (weld caz ~(tap in channels.s))] sez] + :: close all affected channels and send their responses + :: + =| moves1=(list move) + |- ^- (quip move server-state) + ?^ channels + %- %+ trace 1 + |.("{(trip i.channels)} discarding channel due to closed session") + =^ moz state + (discard-channel:by-channel i.channels |) + $(moves1 (weld moves1 moz), channels t.channels) + :: lastly, %real sessions require additional cleanup + :: + ?. ?=(%real -.identity.u.ses) [moves1 state] + =^ moves2 visitors.auth.state + %+ roll ~(tap by visitors.auth.state) + |= [[nonce=@uv visa=visitor] [moz=(list move) viz=(map @uv visitor)]] + ?^ +.visa [moz (~(put by viz) nonce visa)] + :_ viz + %+ weld moz + ?~ duct.visa ~ + [(send-boon:server:eauth(duct u.duct.visa) %0 %shut nonce)]~ + [(weld `(list move)`moves1 `(list move)`moves2) state] + :: +code: returns the same as |code + :: + ++ code + ^- @ta + =/ res=(unit (unit cage)) + (rof [~ ~] /eyre %j [our %code da+now] /(scot %p our)) + (rsh 3 (scot %p ;;(@ q.q:(need (need res))))) + :: +session-cookie-string: compose session cookie + :: + ++ session-cookie-string + |= [session=@uv extend=(unit ?(%auth %guest))] + ^- @t + %- crip + =; max-age=tape + "urbauth-{(scow %p our)}={(scow %uv session)}; Path=/; Max-Age={max-age}" + %- a-co:co + ?~ extend 0 + =, session-timeout + (div (msec:milly ?-(u.extend %auth auth, %guest guest)) 1.000) + :: + :: + ++ eauth + =* auth auth.state + |% + ++ server + |% + :: +start: initiate an eauth login attempt for the :ship identity + :: + ++ start + |= [=ship base=(unit @t) last=@t] + ^- [(list move) server-state] + %- (trace 2 |.("eauth: starting eauth into {(scow %p ship)}")) + =/ nonce=@uv + |- + =+ n=(~(raw og (shas %eauth-nonce eny)) 64) + ?. (~(has by visitors.auth) n) n + $(eny (shas %try-again n)) + =/ visit=visitor [~ `[duct now] ship base last ~] + =. visitors.auth (~(put by visitors.auth) nonce visit) + :_ state + :: we delay serving an http response until we receive a scry %tune + :: + :~ (send-keen %keen ship nonce now) + (start-timeout /visitors/(scot %uv nonce)) + == + :: +on-tune: receive a client-url remote scry result + :: + ++ on-tune + |= [ship=@p nonce=@uv url=@t] + ^- [(list move) server-state] + %- (trace 2 |.("eauth: %tune from {(scow %p ship)}")) + :: guarantee the ship still controls the nonce + :: + =/ visa=visitor (~(got by visitors.auth) nonce) + ?> &(?=(^ +.visa) =(ship ship.visa)) + :: redirect the visitor to their own confirmation page + :: + =. visitors.auth (~(put by visitors.auth) nonce visa(pend ~)) + %- handle-response(duct http:(need pend.visa)) + =; url=@t [%start 303^['location' url]~ ~ &] + %+ rap 3 + :~ url + '?server=' (scot %p our) + '&nonce=' (scot %uv nonce) + == + :: +on-plea: receive an eauth network message from a client + :: + ++ on-plea + |= [=ship plea=eauth-plea] + ^- [(list move) server-state] + %- (trace 2 |.("eauth: {(trip +<.plea)} from {(scow %p ship)}")) + =; res=[(list move) server-state] + =^ moz state res + [[[duct %give %done ~] moz] state] + ?- +<.plea + %open + :: this attempt may or may not have been started in +start yet + :: + =/ visa=visitor + %+ ~(gut by visitors.auth) nonce.plea + [~ ~ ship ~ '/' ~] + ?> ?=(^ +.visa) + ?> =(ship ship.visa) + ::NOTE that token might still be empty, in which case the http + :: client will probably signal an abort when they return + :: + =. duct.visa `duct + =. toke.visa token.plea + =. visitors.auth (~(put by visitors.auth) nonce.plea visa) + :: if the eauth attempt was started on our side, we may know the + :: specific base url the user used; make sure they go back there + :: + =/ url=@t + %- need + ?~ base.visa eauth-url + eauth-url(user.endpoint.auth base.visa) + [[(send-boon %0 %okay nonce.plea url)]~ state] + :: + %shut + :: the visitor wants the associated session gone + :: + ?~ visa=(~(get by visitors.auth) nonce.plea) [~ state] + =. visitors.auth (~(del by visitors.auth) nonce.plea) + =? sessions.auth ?=(@ +.u.visa) + (~(del by sessions.auth) sesh.u.visa) + [[(send-boon %0 %shut nonce.plea)]~ state] + == + :: +cancel: the client aborted the eauth attempt, so clean it up + :: + ++ cancel + |= [nonce=@uv last=@t] + ^- [(list move) server-state] + :: if the eauth attempt doesn't exist, or it was already completed, + :: we cannot cancel it + :: + ?~ visa=(~(get by visitors.auth) nonce) [~ state] + ?@ +.u.visa [~ state] + :: delete the attempt, and go back to the login page + :: + %- (trace 2 |.("eauth: cancelling login")) + =. visitors.auth (~(del by visitors.auth) nonce) + =^ moz state + =/ url=@t + %^ cat 3 '/~/login?eauth&redirect=' + (crip (en-urlt:html (trip last))) + (handle-response %start 303^['location' url]~ ~ &) + :_ state + %+ weld moz + ?~ duct.u.visa ~ + [(send-boon(duct u.duct.u.visa) %0 %shut nonce)]~ + :: +expire: host-side cancel an eauth attempt if it's still pending + :: + ++ expire + |= nonce=@uv + ^- [(list move) server-state] + ?~ visa=(~(get by visitors.auth) nonce) + [~ state] + ?@ +.u.visa [~ state] + %- (trace 2 |.("eauth: expiring")) + =^ moz state + ?~ pend.u.visa [~ state] + %- return-static-data-on-duct(duct http.u.pend.u.visa) + [503 'text/html' (eauth-error-page %server last.u.visa)] + =? moz ?=(^ pend.u.visa) + [(send-keen %yawn ship.u.visa nonce keen.u.pend.u.visa) moz] + =. visitors.auth (~(del by visitors.auth) nonce) + :_ state + %+ weld moz + ?~ duct.u.visa ~ + [(send-boon(duct u.duct.u.visa) %0 %shut nonce)]~ + :: +finalize: eauth attempt was approved: mint the client a new session + :: + :: gives the http response on the current duct + :: + ++ finalize + |= [=plea=^duct nonce=@uv =ship last=@t] + ^- [(list move) server-state] + %- (trace 2 |.("eauth: finalizing login for {(scow %p ship)}")) + :: clean up the session they're changing out from, + :: mint the new session, + :: associate it with the nonce, + :: and the finalization request, + :: and send the visitor the cookie + final redirect + :: + =^ moz1 state + (close-session session-id:(~(got by connections.state) duct) |) + =^ [sid=@uv * moz2=(list move)] state + (start-session %eauth ship) + =. visitors.auth + %+ ~(jab by visitors.auth) nonce + |=(v=visitor v(+ sid)) + =. connections.state + %+ ~(jab by connections.state) duct + |= o=outstanding-connection + o(session-id sid) + =^ moz3 state + =; hed (handle-response %start 303^hed ~ &) + :~ ['location' last] + ['set-cookie' (session-cookie-string sid `%auth)] + == + [:(weld moz1 moz2 moz3) state] + :: +on-fail: we crashed or received an empty %tune, clean up + :: + ++ on-fail + |= [=ship nonce=@uv] + ^- [(list move) server-state] + :: if the eauth attempt doesn't exist, or it was already completed, + :: we can no-op here + :: + ?~ visa=(~(get by visitors.auth) nonce) [~ state] + ?@ +.u.visa [~ state] + :: delete the attempt, and go back to the login page + :: + %- (trace 2 |.("eauth: failed login")) + =. visitors.auth (~(del by visitors.auth) nonce) + =^ moz state + ?~ pend.u.visa [~ state] + %- return-static-data-on-duct(duct http.u.pend.u.visa) + [503 'text/html' (eauth-error-page %server last.u.visa)] + :_ state + %+ weld moz + ?~ duct.u.visa ~ + [(send-boon(duct u.duct.u.visa) %0 %shut nonce)]~ + :: + ::TODO +on-request? + :: + ++ send-keen + |= [kind=?(%keen %yawn) =ship nonce=@uv =time] + ^- move + %- (trace 2 |.("eauth: %{(trip kind)} into {(scow %p ship)}")) + :: we round down the time to make it more likely to hit cache, + :: at the expense of not working if the endpoint changed within + :: the last +eauth-cache-rounding. + :: + =/ =wire /eauth/keen/(scot %p ship)/(scot %uv nonce) + =. time (sub time (mod time eauth-cache-rounding)) + =/ =spar:ames [ship /e/x/(scot %da time)//eauth/url] + [duct %pass wire %a ?-(kind %keen keen+[~ spar], %yawn yawn+spar)] + :: + ++ send-boon + |= boon=eauth-boon + ^- move + %- (trace 2 |.("eauth: sending {(trip +<.boon)}")) + [duct %give %boon boon] + -- + :: + ++ client + |% + :: +start: as the client, approve or abort an eauth attempt + :: + :: assumes the duct is of an incoming eauth start/approve request + :: + ++ start + |= [host=ship nonce=@uv grant=?] + ^- [(list move) server-state] + =/ token=@uv (~(raw og (shas %eauth-token eny)) 128) + :: we always send an %open, because we need to redirect the user + :: back to the host. and we always set a timeout, because we may + :: not get a response quickly enough. + :: + :- :~ (send-plea host %0 %open nonce ?:(grant `token ~)) + (start-timeout /visiting/(scot %p host)/(scot %uv nonce)) + == + :: make sure we aren't attempting with this nonce already, + :: then remember the secret so we can include it in the redirect + :: + =/ book (~(gut by visiting.auth) host *logbook) + ?< (~(has by map.book) nonce) + =. visiting.auth + %+ ~(put by visiting.auth) host + :- (~(put to qeu.book) nonce) + (~(put by map.book) nonce [`duct ?:(grant `token ~)]) + state + :: +on-done: receive n/ack for plea we sent + :: + ++ on-done + |= [host=ship good=?] + ^- [(list move) server-state] + %- %- trace + ?: good + [2 |.("eauth: ack from {(scow %p host)}")] + [1 |.("eauth: nack from {(scow %p host)}")] + =/ book (~(gut by visiting.auth) host *logbook) + ?~ ~(top to qeu.book) + %. [~ state] + (trace 0 |.("eauth: done on empty queue from {(scow %p host)}")) + =^ nonce=@uv qeu.book ~(get to qeu.book) + ?: good + =. visiting.auth + ?: =([~ ~] book) + (~(del by visiting.auth) host) + (~(put by visiting.auth) host book) + [~ state] + =/ port (~(get by map.book) nonce) + ?~ port [~ state] + :: delete the attempt/session, serve response if needed + :: + =. visiting.auth + =. map.book + (~(del by map.book) nonce) + ?: =([~ ~] book) + (~(del by visiting.auth) host) + (~(put by visiting.auth) host book) + :: + ?@ u.port [~ state] + ?~ pend.u.port [~ state] + %^ return-static-data-on-duct(duct u.pend.u.port) 503 'text/html' + (eauth-error-page ~) + :: +on-boon: receive an eauth network response from a host + :: + :: crashes on unexpected circumstances, in response to which we + :: should abort the eauth attempt + :: + ++ on-boon + |= [host=ship boon=eauth-boon] + ^- [(list move) server-state] + %- (trace 2 |.("eauth: %{(trip +<.boon)} from {(scow %p host)}")) + ?- +<.boon + %okay + =/ book (~(got by visiting.auth) host) + =/ port (~(got by map.book) nonce.boon) + ?> ?=(^ port) + ?> ?=(^ pend.port) + :: update the outgoing sessions map, deleting if we aborted + :: + =. visiting.auth + ?^ toke.port + %+ ~(put by visiting.auth) host + :- qeu.book + ::NOTE optimistic + (~(put by map.book) nonce.boon now) + =. map.book + (~(del by map.book) nonce.boon) + ?: =([~ ~] book) + (~(del by visiting.auth) host) + (~(put by visiting.auth) host book) + :: always serve a redirect, with either the token, or abort signal + :: + =; url=@t + %- handle-response(duct u.pend.port) + [%start 303^['location' url]~ ~ &] + %+ rap 3 + :* url.boon + '?nonce=' (scot %uv nonce.boon) + ?~ toke.port ['&abort']~ + ~['&token=' (scot %uv u.toke.port)] + == + :: + %shut + :: the host has deleted the corresponding session + :: + =. visiting.auth + =/ book + (~(gut by visiting.auth) host *logbook) + =. map.book + (~(del by map.book) nonce.boon) + ?: =([~ ~] book) + (~(del by visiting.auth) host) + (~(put by visiting.auth) host book) + [~ state] + == + :: + ++ expire + |= [host=ship nonce=@uv] + ^- [(list move) server-state] + =/ book (~(gut by visiting.auth) host *logbook) + =/ port (~(get by map.book) nonce) + :: if the attempt was completed, we don't expire it + :: + ?~ port [~ state] + ?@ u.port [~ state] + :: delete pending attempts, serve response if needed + :: + %- %+ trace 1 + |.("eauth: attempt into {(scow %p host)} expired") + =. visiting.auth + =. map.book + (~(del by map.book) nonce) + ?: =([~ ~] book) + (~(del by visiting.auth) host) + (~(put by visiting.auth) host book) + :: + ?~ pend.u.port [~ state] + %^ return-static-data-on-duct(duct u.pend.u.port) 503 'text/html' + (eauth-error-page ~) + :: + ++ send-plea + |= [=ship plea=eauth-plea] + ^- move + ::NOTE no nonce in the wire, to avoid proliferating flows + =/ =wire /eauth/plea/(scot %p ship) + %- (trace 2 |.("eauth: {(trip +<.plea)} into {(scow %p ship)}")) + [[/eyre/eauth/synthetic]~ %pass wire %a %plea ship %e /eauth/0 plea] + :: + ++ confirmation-page + |= [server=ship nonce=@uv] + ^- octs + %- as-octs:mimes:html + %- crip + %- en-xml:html + =/ favicon %+ + weld "<svg width='10' height='10' viewBox='0 0 10 10' xmlns='http://www.w3.org/2000/svg'>" + "<circle r='3.09' cx='5' cy='5' /></svg>" + ;html + ;head + ;meta(charset "utf-8"); + ;meta(name "viewport", content "width=device-width, initial-scale=1, shrink-to-fit=no"); + ;link(rel "icon", type "image/svg+xml", href (weld "data:image/svg+xml;utf8," favicon)); + ;title:"Urbit" + ;style:"{(trip auth-styling)}" + ;style:''' + form { + border: 1px solid var(--black20); + border-radius: 4px; + padding: 1rem; + align-items: stretch; + font-size: 14px; + } + .red { + background: var(--black05) !important; + color: var(--black60) !important; + border: 1px solid var(--black60) !important; + } + code { + font-weight: bold; + font-family: "Source Code Pro", monospace; + } + button { + display: inline-block; + } + ''' + == + ;body + ;form(action "/~/eauth", method "post") + ; Hello, {(scow %p our)}. + ; You are trying to log in to: + ;code:"{(scow %p server)}" + ;input(type "hidden", name "server", value (scow %p server)); + ;input(type "hidden", name "nonce", value (scow %uv nonce)); + ;button(type "submit", name "grant", value "grant"):"approve" + ;button(type "submit", name "reject", class "red"):"reject" + == + == + == + -- + :: +on-request: http request to the /~/eauth endpoint + :: + ++ on-request + |= [[session-id=@uv =identity] =request:http] + ^- [(list move) server-state] + :: we may need the requester to log in before proceeding + :: + =* login + =; url=@t (handle-response %start 303^['location' url]~ ~ &) + %^ cat 3 '/~/login?redirect=' + (crip (en-urlt:html (trip url.request))) + :: or give them a generic, static error page in unexpected cases + :: + =* error %^ return-static-data-on-duct 400 'text/html' + (eauth-error-page ~) + :: GET requests either render the confirmation page, + :: or finalize an eauth flow + :: + ?: ?=(%'GET' method.request) + =/ args=(map @t @t) (malt args:(parse-request-line url.request)) + =/ server=(unit @p) (biff (~(get by args) 'server') (cury slaw %p)) + =/ nonce=(unit @uv) (biff (~(get by args) 'nonce') (cury slaw %uv)) + =/ token=(unit @uv) (biff (~(get by args) 'token') (cury slaw %uv)) + =/ abort=? (~(has by args) 'abort') + :: + ?~ nonce error + :: + ?^ server + :: request for confirmation page + :: + ?. ?=(%ours -.identity) login + =/ book (~(gut by visiting.auth) u.server *logbook) + =/ door (~(get by map.book) u.nonce) + ?~ door + :: nonce not yet used, render the confirmation page as normal + :: + %^ return-static-data-on-duct 200 'text/html' + (confirmation-page:client u.server u.nonce) + :: if we're still awaiting a redirect target, we choose to serve + :: this latest request instead + :: + ?@ u.door error + ?~ pend.u.door error + =. map.book (~(put by map.book) u.nonce u.door(pend `duct)) + =. visiting.auth (~(put by visiting.auth) u.server book) + %- return-static-data-on-duct(duct u.pend.u.door) + [202 'text/plain' (as-octs:mimes:html 'continued elsewhere...')] + :: important to provide an error response for unexpected states + :: + =/ visa=(unit visitor) (~(get by visitors.auth) u.nonce) + ?~ visa error + ?@ +.u.visa error + =* error %^ return-static-data-on-duct 400 'text/html' + (eauth-error-page %server last.u.visa) + :: request for finalization, must either abort or provide a token + :: + ::NOTE yes, this means that unauthenticated clients can abort + :: any eauth attempt they know the nonce for, but that should + :: be pretty benign + ?: abort (cancel:^server u.nonce last.u.visa) + ?~ token error + :: if this request provides a token, but the client didn't, complain + :: + ?~ toke.u.visa error + :: verify the request + :: + ?. =(u.token u.toke.u.visa) + %- (trace 1 |.("eauth: token mismatch")) + error + ?~ duct.u.visa error + (finalize:^server u.duct.u.visa u.nonce ship.u.visa last.u.visa) + :: + ?. ?=(%'POST' method.request) + %^ return-static-data-on-duct 405 'text/html' + (eauth-error-page ~) + ?. =(%ours -.identity) login + :: POST requests are always submissions of the confirmation page + :: + =/ args=(map @t @t) + (malt (fall (rush q:(fall body.request *octs) yquy:de-purl:html) ~)) + =/ server=(unit @p) (biff (~(get by args) 'server') (cury slaw %p)) + =/ nonce=(unit @uv) (biff (~(get by args) 'nonce') (cury slaw %uv)) + =/ grant=? =(`'grant' (~(get by args) 'grant')) + :: + =* error %^ return-static-data-on-duct 400 'text/html' + (eauth-error-page ~) + ?~ server error + ?~ nonce error + =/ book (~(gut by visiting.auth) u.server *logbook) + ?: (~(has by map.book) u.nonce) error + (start:client u.server u.nonce grant) + :: + ++ eauth-url + ^- (unit @t) + =/ end=(unit @t) (clap user.endpoint.auth auth.endpoint.auth head) + ?~ end ~ + `(cat 3 u.end '/~/eauth') + :: + ++ start-timeout + |= =path + ^- move + [duct %pass [%eauth %expire path] %b %wait (add now eauth-timeout)] + -- + -- + :: +channel: per-event handling of requests to the channel system + :: + :: Eyre offers a remote interface to your Urbit through channels, which + :: are persistent connections on the server which can be disconnected and + :: reconnected on the client. + :: + ++ by-channel + :: moves: the moves to be sent out at the end of this event, reversed + :: + =| moves=(list move) + |% + :: +handle-request: handles an http request for the subscription system + :: + ++ handle-request + |= [[session-id=@uv =identity] =address =request:http] + ^- [(list move) server-state] + :: parse out the path key the subscription is on + :: + =+ request-line=(parse-request-line url.request) + ?. ?=([@t @t @t ~] site.request-line) + :: url is not of the form '/~/channel/' + :: + %^ return-static-data-on-duct 400 'text/html' + (error-page 400 & url.request "malformed channel url") + :: channel-id: unique channel id parsed out of url + :: + =+ channel-id=i.t.t.site.request-line + :: + ?: =('PUT' method.request) + :: PUT methods starts/modifies a channel, and returns a result immediately + :: + (on-put-request channel-id identity request) + :: + ?: =('GET' method.request) + (on-get-request channel-id [session-id identity] request) + ?: =('POST' method.request) + :: POST methods are used solely for deleting channels + (on-put-request channel-id identity request) + :: + %- (trace 0 |.("session not a put")) + %^ return-static-data-on-duct 405 'text/html' + (error-page 405 & url.request "bad method for session endpoint") + :: +on-cancel-request: cancels an ongoing subscription + :: + :: One of our long lived sessions just got closed. We put the associated + :: session back into the waiting state. + :: + ++ on-cancel-request + ^- [(list move) server-state] + :: lookup the session id by duct + :: + %- (trace 1 |.("{<duct>} moving channel to waiting state")) + :: + ?~ maybe-channel-id=(~(get by duct-to-key.channel-state.state) duct) + ((trace 0 |.("{<duct>} no channel to move")) `state) + :: + =/ maybe-session + (~(get by session.channel-state.state) u.maybe-channel-id) + ?~ maybe-session + ((trace 1 |.("{<maybe-session>} session doesn't exist")) `state) + :: + =/ heartbeat-cancel=(list move) + ?~ heartbeat.u.maybe-session ~ + :~ %^ cancel-heartbeat-move + u.maybe-channel-id + date.u.heartbeat.u.maybe-session + duct.u.heartbeat.u.maybe-session + == + :: + =/ expiration-time=@da (add now channel-timeout) + :: + :- %+ weld heartbeat-cancel + [(set-timeout-move u.maybe-channel-id expiration-time) moves] + %_ state + session.channel-state + %+ ~(jab by session.channel-state.state) u.maybe-channel-id + |= =channel + :: if we are canceling a known channel, it should have a listener + :: + ?> ?=([%| *] state.channel) + channel(state [%& [expiration-time duct]], heartbeat ~) + :: + duct-to-key.channel-state + (~(del by duct-to-key.channel-state.state) duct) + == + :: +update-timeout-timer-for: sets a timeout timer on a channel + :: + :: This creates a channel if it doesn't exist, cancels existing timers + :: if they're already set (we cannot have duplicate timers), and (if + :: necessary) moves channels from the listening state to the expiration + :: state. + :: + ++ update-timeout-timer-for + |= [mode=?(%json %jam) =identity channel-id=@t] + ^+ ..update-timeout-timer-for + :: when our callback should fire + :: + =/ expiration-time=@da (add now channel-timeout) + :: if the channel doesn't exist, create it and set a timer + :: + ?~ maybe-channel=(~(get by session.channel-state.state) channel-id) + :: + %_ ..update-timeout-timer-for + session.channel-state.state + %+ ~(put by session.channel-state.state) channel-id + [mode identity [%& expiration-time duct] 0 now ~ ~ ~ ~] + :: + moves + [(set-timeout-move channel-id expiration-time) moves] + == + :: if the channel has an active listener, we aren't setting any timers + :: + ?: ?=([%| *] state.u.maybe-channel) + ..update-timeout-timer-for + :: we have a previous timer; cancel the old one and set the new one + :: + %_ ..update-timeout-timer-for + session.channel-state.state + %+ ~(jab by session.channel-state.state) channel-id + |= =channel + channel(state [%& [expiration-time duct]]) + :: + moves + :* (cancel-timeout-move channel-id p.state.u.maybe-channel) + (set-timeout-move channel-id expiration-time) + moves + == + == + :: + ++ set-heartbeat-move + |= [channel-id=@t heartbeat-time=@da] + ^- move + :^ duct %pass /channel/heartbeat/[channel-id] + [%b %wait heartbeat-time] + :: + ++ cancel-heartbeat-move + |= [channel-id=@t heartbeat-time=@da =^duct] + ^- move + :^ duct %pass /channel/heartbeat/[channel-id] + [%b %rest heartbeat-time] + :: + ++ set-timeout-move + |= [channel-id=@t expiration-time=@da] + ^- move + [duct %pass /channel/timeout/[channel-id] %b %wait expiration-time] + :: + ++ cancel-timeout-move + |= [channel-id=@t expiration-time=@da =^duct] + ^- move + :^ duct %pass /channel/timeout/[channel-id] + [%b %rest expiration-time] + :: +on-get-request: handles a GET request + :: + :: GET requests connect to a channel for the server to send events to + :: the client in text/event-stream format. + :: + ++ on-get-request + |= [channel-id=@t [session-id=@uv =identity] =request:http] + ^- [(list move) server-state] + :: if the channel doesn't exist, we cannot serve it. + :: this 404 also lets clients know if their channel was reaped since + :: they last connected to it. + :: + ?. (~(has by session.channel-state.state) channel-id) + %^ return-static-data-on-duct 404 'text/html' + (error-page 404 | url.request ~) + :: + =/ mode=?(%json %jam) + (find-channel-mode %'GET' header-list.request) + =^ [exit=? =wall moves=(list move)] state + :: the request may include a 'Last-Event-Id' header + :: + =/ maybe-last-event-id=(unit @ud) + ?~ maybe-raw-header=(get-header:http 'last-event-id' header-list.request) + ~ + (rush u.maybe-raw-header dum:ag) + =/ channel + (~(got by session.channel-state.state) channel-id) + :: we put some demands on the get request, and may need to do some + :: cleanup for prior requests. + :: + :: find the channel creator's identity, make sure it matches + :: + ?. =(identity identity.channel) + =^ mos state + %^ return-static-data-on-duct 403 'text/html' + (error-page 403 | url.request ~) + [[& ~ mos] state] + :: make sure the request "mode" doesn't conflict with a prior request + :: + ::TODO or could we change that on the spot, given that only a single + :: request will ever be listening to this channel? + ?. =(mode mode.channel) + =^ mos state + %^ return-static-data-on-duct 406 'text/html' + =; msg=tape (error-page 406 %.y url.request msg) + "channel already established in {(trip mode.channel)} mode" + [[& ~ mos] state] + :: when opening an event-stream, we must cancel our timeout timer + :: if there's no duct already bound. else, kill the old request, + :: we will replace its duct at the end of this arm + :: + =^ cancel-moves state + ?: ?=([%& *] state.channel) + :_ state + (cancel-timeout-move channel-id p.state.channel)^~ + =. duct-to-key.channel-state.state + (~(del by duct-to-key.channel-state.state) p.state.channel) + =/ cancel-heartbeat + ?~ heartbeat.channel ~ + :_ ~ + %+ cancel-heartbeat-move channel-id + [date duct]:u.heartbeat.channel + =- [(weld cancel-heartbeat -<) ->] + (handle-response(duct p.state.channel) [%cancel ~]) + :: flush events older than the passed in 'Last-Event-ID' + :: + =? state ?=(^ maybe-last-event-id) + (acknowledge-events channel-id u.maybe-last-event-id) + ::TODO that did not remove them from the channel queue though! + :: we may want to account for maybe-last-event-id, for efficiency. + :: (the client _should_ ignore events it heard previously if we do + :: end up re-sending them, but _requiring_ that feels kinda risky) + :: + :: combine the remaining queued events to send to the client + :: + =; event-replay=wall + [[| - cancel-moves] state] + %- zing + %- flop + =/ queue events.channel + =| events=(list wall) + |- + ^+ events + ?: =(~ queue) + events + =^ head queue ~(get to queue) + =, p.head + ::NOTE these will only fail if the mark and/or json types changed, + :: since conversion failure also gets caught during first receive. + :: we can't do anything about this, so consider it unsupported. + =/ said + (channel-event-to-tape channel request-id channel-event) + ?~ said $ + $(events [(event-tape-to-wall id +.u.said) events]) + ?: exit [moves state] + :: send the start event to the client + :: + =^ http-moves state + %- handle-response + :* %start + :- 200 + :~ ['content-type' 'text/event-stream'] + ['cache-control' 'no-cache'] + ['connection' 'keep-alive'] + == + :: + :: if we wouldn't otherwise send any data, send an early heartbeat + :: instead. some clients won't consider the connection established + :: until they've heard some bytes come over the wire. + :: + ?. =(~ wall) (wall-to-octs wall) + (some (as-octs:mimes:html ':\0a')) + :: + complete=%.n + == + :: associate this duct with this session key + :: + =. duct-to-key.channel-state.state + (~(put by duct-to-key.channel-state.state) duct channel-id) + :: associate this channel with the session cookie + :: + =. sessions.auth.state + %+ ~(jab by sessions.auth.state) + session-id + |= =session + session(channels (~(put in channels.session) channel-id)) + :: initialize sse heartbeat + :: + =/ heartbeat-time=@da (add now ~s20) + =/ heartbeat (set-heartbeat-move channel-id heartbeat-time) + :: record the mode & duct for future output, + :: and record heartbeat-time for possible future cancel + :: + =. session.channel-state.state + %+ ~(jab by session.channel-state.state) channel-id + |= =channel + %_ channel + mode mode + state [%| duct] + heartbeat (some [heartbeat-time duct]) + == + :: + [[heartbeat :(weld http-moves moves)] state] + :: +acknowledge-events: removes events before :last-event-id on :channel-id + :: + ++ acknowledge-events + |= [channel-id=@t last-event-id=@u] + ^- server-state + %_ state + session.channel-state + %+ ~(jab by session.channel-state.state) channel-id + |= =channel + ^+ channel + =^ acked events.channel + (prune-events events.channel last-event-id) + =. unacked.channel + (subtract-acked-events acked unacked.channel) + channel(last-ack now) + == + :: +on-put-request: handles a PUT request + :: + :: PUT requests send commands from the client to the server. We receive + :: a set of commands in JSON format in the body of the message. + :: channels don't exist until a PUT request is sent. it's valid for + :: this request to contain an empty list of commands. + :: + ++ on-put-request + |= [channel-id=@t =identity =request:http] + ^- [(list move) server-state] + :: if the channel already exists, and is not of this identity, 403 + :: + :: the creation case happens in the +update-timeout-timer-for below + :: + ?: ?~ c=(~(get by session.channel-state.state) channel-id) | + !=(identity identity.u.c) + %^ return-static-data-on-duct 403 'text/html' + (error-page 403 | url.request ~) + :: error when there's no body + :: + ?~ body.request + %^ return-static-data-on-duct 400 'text/html' + (error-page 400 %.y url.request "no put body") + :: + =/ mode=?(%json %jam) + (find-channel-mode %'PUT' header-list.request) + :: if we cannot parse requests from the body, give an error + :: + =/ maybe-requests=(each (list channel-request) @t) + (parse-channel-request mode u.body.request) + ?: ?=(%| -.maybe-requests) + %^ return-static-data-on-duct 400 'text/html' + (error-page 400 & url.request (trip p.maybe-requests)) + :: check for the existence of the channel-id + :: + :: if we have no session, create a new one set to expire in + :: :channel-timeout from now. if we have one which has a timer, update + :: that timer. + :: + =. ..on-put-request (update-timeout-timer-for mode identity channel-id) + :: for each request, execute the action passed in + :: + =+ requests=p.maybe-requests + :: gall-moves: put moves here first so we can flop for ordering + :: errors: if we accumulate any, discard the gall-moves and revert + :: + =| gall-moves=(list move) + =| errors=(map @ud @t) + =/ og-state state + =/ from=ship + ?+(-.identity who.identity %ours our) + |- + :: + ?~ requests + ?: =(~ errors) + :: everything succeeded, mark the request as completed + :: + =^ http-moves state + %- handle-response + :* %start + [status-code=204 headers=~] + data=~ + complete=%.y + == + :: + [:(weld (flop gall-moves) http-moves moves) state] + :: some things went wrong. revert all operations & give 400 + :: + %- (trace 1 |.("{<channel-id>} reverting due to errors")) + =. state og-state + =^ http-moves state + %^ return-static-data-on-duct 400 'text/html' + %- as-octs:mimes:html + %+ rap 3 + %+ turn (sort ~(tap by errors) dor) + |= [id=@ud er=@t] + (rap 3 (crip (a-co:co id)) ': ' er '<br/>' ~) + [(weld http-moves moves) state] + :: + ?- -.i.requests + %ack + :: client acknowledges that they have received up to event-id + :: + %_ $ + state (acknowledge-events channel-id event-id.i.requests) + requests t.requests + == + :: + ?(%poke %poke-json) + =, i.requests + :: + ?. |(=(from our) =(ship our)) + =+ [request-id 'non-local operation'] + $(errors (~(put by errors) -), requests t.requests) + :: + =. gall-moves + =/ =wire /channel/poke/[channel-id]/(scot %ud request-id.i.requests) + :_ gall-moves + ^- move + %+ deal-as + /channel/poke/[channel-id]/(scot %ud request-id) + :^ from ship app + ^- task:agent:gall + :+ %poke-as mark + ?- -.i.requests + %poke [%noun !>(noun)] + %poke-json [%json !>(json)] + == + :: + $(requests t.requests) + :: + %subscribe + =, i.requests + :: + ?. |(=(from our) =(ship our)) + =+ [request-id 'non-local operation'] + $(errors (~(put by errors) -), requests t.requests) + :: + ::TODO could error if the subscription is a duplicate + =. gall-moves + :_ gall-moves + ^- move + %- (trace 1 |.("subscribing to {<app>} on {<path>}")) + %+ deal-as + (subscription-wire channel-id request-id from ship app) + [from ship app %watch path] + :: + =. session.channel-state.state + %+ ~(jab by session.channel-state.state) channel-id + |= =channel + =- channel(subscriptions -) + %+ ~(put by subscriptions.channel) + request-id + [ship app path duct] + :: + $(requests t.requests) + :: + %unsubscribe + =, i.requests + :: + ?. |(=(from our) =(ship our)) + =+ [request-id 'non-local operation'] + $(errors (~(put by errors) -), requests t.requests) + :: + =/ usession (~(get by session.channel-state.state) channel-id) + ?~ usession + $(requests t.requests) + =/ subscriptions subscriptions:u.usession + :: + ?~ maybe-subscription=(~(get by subscriptions) subscription-id) + :: the client sent us a weird request referring to a subscription + :: which isn't active. + :: + %. $(requests t.requests) + =* msg=tape "{(trip channel-id)} {<subscription-id>}" + (trace 0 |.("missing subscription in unsubscribe {msg}")) + :: + =. gall-moves + :_ gall-moves + ^- move + =, u.maybe-subscription + %- (trace 1 |.("leaving subscription to {<app>}")) + %+ deal-as(duct duc) + (subscription-wire channel-id subscription-id from ship app) + [from ship app %leave ~] + :: + =. session.channel-state.state + %+ ~(jab by session.channel-state.state) channel-id + |= =channel + %_ channel + subscriptions (~(del by subscriptions.channel) subscription-id) + unacked (~(del by unacked.channel) subscription-id) + == + :: + $(requests t.requests) + :: + %delete + %- (trace 1 |.("{<channel-id>} discarding due to %delete PUT")) + =^ moves state + (discard-channel channel-id |) + =. gall-moves + (weld gall-moves moves) + $(requests t.requests) + :: + == + :: +on-gall-response: sanity-check a gall response, send as event + :: + ++ on-gall-response + |= [channel-id=@t request-id=@ud extra=wire =sign:agent:gall] + ^- [(list move) server-state] + :: if the channel doesn't exist, we should clean up subscriptions + :: + :: this is a band-aid solution. you really want eyre to have cleaned + :: these up on-channel-delete in the first place. + :: until the source of that bug is discovered though, we keep this + :: in place to ensure a slightly tidier home. + :: + ?. ?& !(~(has by session.channel-state.state) channel-id) + ?=(?(%fact %watch-ack) -.sign) + ?=([@ @ *] extra) + == + (emit-event channel-id request-id sign) + =/ =ship (slav %p i.extra) + =* app=term i.t.extra + =* msg=tape "{(trip channel-id)} {(trip app)}" + %- (trace 0 |.("removing watch for non-existent channel {msg}")) + :_ state + :_ ~ + ^- move + =/ [as=@p old=?] + ?+ t.t.extra ~|([%strange-wire extra] !!) + ~ [our &] + [@ ~] [(slav %p i.t.t.extra) |] + == + =/ =wire (subscription-wire channel-id request-id as ship app) + %+ deal-as + ::NOTE we previously used a wire format that had the local identity + :: implicit, instead of explicit at the end of the wire. if we + :: detect we used the old wire here, we must re-use that format + :: (without id in the wire) for sending the %leave. + ?:(old (snip wire) wire) + [as ship app %leave ~] + :: +emit-event: records an event occurred, possibly sending to client + :: + :: When an event occurs, we need to record it, even if we immediately + :: send it to a connected browser so in case of disconnection, we can + :: resend it. + :: + :: This function is responsible for taking the event sign and converting + :: it into a text/event-stream. The :sign then may get sent, and is + :: stored for later resending until acknowledged by the client. + :: + ++ emit-event + |= [channel-id=@t request-id=@ud =sign:agent:gall] + ^- [(list move) server-state] + :: + =/ channel=(unit channel) + (~(get by session.channel-state.state) channel-id) + ?~ channel + :_ state :_ ~ + [duct %pass /flog %d %flog %crud %eyre-no-channel >id=channel-id< ~] + :: it's possible that this is a sign emitted directly alongside a fact + :: that triggered a clog & closed the subscription. in that case, just + :: drop the sign. + :: poke-acks are not paired with subscriptions, so we can process them + :: regardless. + :: + ?: ?& !?=(%poke-ack -.sign) + !(~(has by subscriptions.u.channel) request-id) + == + [~ state] + :: attempt to convert the sign to json. + :: if conversion succeeds, we *can* send it. if the client is actually + :: connected, we *will* send it immediately. + :: + =/ maybe-channel-event=(unit channel-event) + (sign-to-channel-event sign u.channel request-id) + ?~ maybe-channel-event [~ state] + =/ =channel-event u.maybe-channel-event + =/ said=(unit (quip move tape)) + (channel-event-to-tape u.channel request-id channel-event) + =? moves ?=(^ said) + (weld moves -.u.said) + =* sending &(?=([%| *] state.u.channel) ?=(^ said)) + :: + =/ next-id next-id.u.channel + :: if we can send it, store the event as unacked + :: + =? events.u.channel ?=(^ said) + %- ~(put to events.u.channel) + [next-id request-id channel-event] + :: if it makes sense to do so, send the event to the client + :: + =? moves sending + ^- (list move) + :_ moves + ::NOTE assertions in this block because =* is flimsy + ?> ?=([%| *] state.u.channel) + :+ p.state.u.channel %give + ^- gift + :* %response %continue + :: + ^= data + %- wall-to-octs + (event-tape-to-wall next-id +:(need said)) + :: + complete=%.n + == + =? next-id ?=(^ said) +(next-id) + :: update channel's unacked counts, find out if clogged + :: + =^ clogged unacked.u.channel + :: only apply clog logic to facts. + :: and of course don't count events we can't send as unacked. + :: + ?: ?| !?=(%fact -.sign) + ?=(~ said) + == + [| unacked.u.channel] + =/ num=@ud + (~(gut by unacked.u.channel) request-id 0) + :_ (~(put by unacked.u.channel) request-id +(num)) + ?& (gte num clog-threshold) + (lth (add last-ack.u.channel clog-timeout) now) + == + :: if we're clogged, or we ran into an event we can't serialize, + :: kill this gall subscription. + :: + =* msg=tape "on {(trip channel-id)} for {(scow %ud request-id)}" + =/ kicking=? + ?: clogged + ((trace 0 |.("clogged {msg}")) &) + ?. ?=(~ said) | + ((trace 0 |.("can't serialize event, kicking {msg}")) &) + =? moves kicking + :_ moves + ::NOTE this shouldn't crash because we + :: - never fail to serialize subscriptionless signs (%poke-ack), + :: - only clog on %facts, which have a subscription associated, + :: - and already checked whether we still have that subscription. + =+ (~(got by subscriptions.u.channel) request-id) + %- (trace 1 |.("leaving subscription to {<app>}")) + %+ deal-as + (subscription-wire channel-id request-id identity.u.channel ship app) + [identity.u.channel ship app %leave ~] + :: update channel state to reflect the %kick + :: + =? u.channel kicking + %_ u.channel + subscriptions (~(del by subscriptions.u.channel) request-id) + unacked (~(del by unacked.u.channel) request-id) + events %- ~(put to events.u.channel) + :+ next-id + request-id + (need (sign-to-channel-event [%kick ~] u.channel request-id)) + == + :: if a client is connected, send the kick event to them + :: + =? moves &(kicking ?=([%| *] state.u.channel)) + :_ moves + :+ p.state.u.channel %give + ^- gift + :* %response %continue + :: + ^= data + %- wall-to-octs + %+ event-tape-to-wall next-id + +:(need (channel-event-to-tape u.channel request-id %kick ~)) + :: + complete=%.n + == + =? next-id kicking +(next-id) + :: + :- (flop moves) + %_ state + session.channel-state + %+ ~(put by session.channel-state.state) channel-id + u.channel(next-id next-id) + == + :: +sign-to-channel-event: strip the vase from a sign:agent:gall + :: + ++ sign-to-channel-event + |= [=sign:agent:gall =channel request-id=@ud] + ^- (unit channel-event) + ?. ?=(%fact -.sign) `sign + ?~ desk=(app-to-desk channel request-id) ~ + :- ~ + [%fact u.desk [p q.q]:cage.sign] + :: +app-to-desk + :: + ++ app-to-desk + |= [=channel request-id=@ud] + ^- (unit desk) + =/ sub (~(get by subscriptions.channel) request-id) + ?~ sub + ((trace 0 |.("no subscription for request-id {(scow %ud request-id)}")) ~) + =/ des=(unit (unit cage)) + (rof [~ ~] /eyre %gd [our app.u.sub da+now] /$) + ?. ?=([~ ~ *] des) + ((trace 0 |.("no desk for app {<app.u.sub>}")) ~) + `!<(=desk q.u.u.des) + :: +channel-event-to-tape: render channel-event from request-id in specified mode + :: + ++ channel-event-to-tape + |= [=channel request-id=@ud =channel-event] + ^- (unit (quip move tape)) + ?- mode.channel + %json %+ bind (channel-event-to-json channel request-id channel-event) + |=((quip move json) [+<- (trip (en:json:html +<+))]) + %jam =- `[~ (scow %uw (jam -))] + [request-id channel-event] + == + :: +channel-event-to-json: render channel event as json channel event + :: + ++ channel-event-to-json + ~% %eyre-channel-event-to-json ..part ~ + |= [=channel request-id=@ud event=channel-event] + ^- (unit (quip move json)) + :: for facts, we try to convert the result to json + :: + =/ [from=(unit [=desk =mark]) jsyn=(unit sign:agent:gall)] + ?. ?=(%fact -.event) [~ `event] + ?: ?=(%json mark.event) + ?~ jsin=((soft json) noun.event) + %. [~ ~] + (slog leaf+"eyre: dropping fake json for {(scow %ud request-id)}" ~) + [~ `[%fact %json !>(u.jsin)]] + :: find and use tube from fact mark to json + :: + :: + =* have=mark mark.event + =/ convert=(unit vase) + =/ cag=(unit (unit cage)) + (rof [~ ~] /eyre %cf [our desk.event da+now] /[have]/json) + ?. ?=([~ ~ *] cag) ~ + `q.u.u.cag + ?~ convert + ((trace 0 |.("no convert from {(trip have)} to json")) [~ ~]) + ~| "conversion failed from {(trip have)} to json" + [`[desk.event have] `[%fact %json (slym u.convert noun.event)]] + ?~ jsyn ~ + %- some + :- ?~ from ~ + :_ ~ + :^ duct %pass /conversion-cache/[mark.u.from] + [%c %warp our desk.u.from `[%sing %f da+now /[mark.u.from]/json]] + =* sign u.jsyn + =, enjs:format + %- pairs + ^- (list [@t json]) + :- ['id' (numb request-id)] + ?- -.sign + %poke-ack + :~ ['response' [%s 'poke']] + :: + ?~ p.sign + ['ok' [%s 'ok']] + ['err' (wall (render-tang-to-wall 100 u.p.sign))] + == + :: + %fact + :+ ['response' [%s 'diff']] + :- 'json' + ~| [%unexpected-fact-mark p.cage.sign] + ?> =(%json p.cage.sign) + !<(json q.cage.sign) + :: + ?~ from ~ + ['mark' [%s mark.u.from]]~ + :: + %kick + ['response' [%s 'quit']]~ + :: + %watch-ack + :~ ['response' [%s 'subscribe']] + :: + ?~ p.sign + ['ok' [%s 'ok']] + ['err' (wall (render-tang-to-wall 100 u.p.sign))] + == + == + :: + ++ event-tape-to-wall + ~% %eyre-tape-to-wall ..part ~ + |= [event-id=@ud =tape] + ^- wall + :~ (weld "id: " (a-co:co event-id)) + (weld "data: " tape) + "" + == + :: + ++ on-channel-heartbeat + |= channel-id=@t + ^- [(list move) server-state] + :: + =/ res + %- handle-response + :* %continue + data=(some (as-octs:mimes:html ':\0a')) + complete=%.n + == + =/ http-moves -.res + =/ new-state +.res + =/ heartbeat-time=@da (add now ~s20) + :_ %_ new-state + session.channel-state + %+ ~(jab by session.channel-state.state) channel-id + |= =channel + channel(heartbeat (some [heartbeat-time duct])) + == + (snoc http-moves (set-heartbeat-move channel-id heartbeat-time)) + :: +discard-channel: remove a channel from state + :: + :: cleans up state, timers, and gall subscriptions of the channel + :: + ++ discard-channel + |= [channel-id=@t expired=?] + ^- [(list move) server-state] + :: + =/ usession=(unit channel) + (~(get by session.channel-state.state) channel-id) + ?~ usession + [~ state] + =/ session=channel u.usession + :: + :_ %_ state + session.channel-state + (~(del by session.channel-state.state) channel-id) + :: + duct-to-key.channel-state + ?. ?=(%| -.state.session) duct-to-key.channel-state.state + (~(del by duct-to-key.channel-state.state) p.state.session) + == + =/ heartbeat-cancel=(list move) + ?~ heartbeat.session ~ + :~ %^ cancel-heartbeat-move + channel-id + date.u.heartbeat.session + duct.u.heartbeat.session + == + =/ expire-cancel=(list move) + ?: expired ~ + ?. ?=(%& -.state.session) ~ + =, p.state.session + [(cancel-timeout-move channel-id date duct)]~ + %+ weld heartbeat-cancel + %+ weld expire-cancel + :: produce a list of moves which cancels every gall subscription + :: + %+ turn ~(tap by subscriptions.session) + |= [request-id=@ud ship=@p app=term =path duc=^duct] + ^- move + %- (trace 1 |.("{<channel-id>} leaving subscription to {<app>}")) + %+ deal-as(duct duc) + (subscription-wire channel-id request-id identity.session ship app) + [identity.session ship app %leave ~] + -- + :: +handle-gall-error: a call to +poke-http-response resulted in a %coup + :: + ++ handle-gall-error + |= =tang + ^- [(list move) server-state] + :: + ?~ connection-state=(~(get by connections.state) duct) + %. `state + (trace 0 |.("{<duct>} error on invalid outstanding connection")) + =* connection u.connection-state + =/ moves-1=(list move) + ?. ?=(%app -.action.connection) + ~ + :_ ~ + =, connection + %- (trace 1 |.("leaving subscription to {<app.action>}")) + (deal-as /watch-response/[eyre-id] identity our app.action %leave ~) + :: + =^ moves-2 state + %^ return-static-data-on-duct 500 'text/html' + :: + %- internal-server-error :* + authenticated.inbound-request.connection + url.request.inbound-request.connection + tang + == + [(weld moves-1 moves-2) state] + :: +handle-response: check a response for correctness and send to earth + :: + :: All outbound responses including %http-server generated responses need to go + :: through this interface because we want to have one centralized place + :: where we perform logging and state cleanup for connections that we're + :: done with. + :: + ++ handle-ws-response + |= [wid=@ event=websocket-event] + ^- [(list move) server-state] + ~& eyre-handle-ws-response=event + :: TODO remove if not accepted? + =. connections.state + ?. ?=(%reject -.event) connections.state + (~(del by connections.state) duct) + + :: TODO do all verification shit that handle-response is doing + [[duct %give %websocket-response [wid event]]~ state] + + ++ handle-response + |= =http-event:http + ^- [(list move) server-state] + :: verify that this is a valid response on the duct + :: + ?~ connection-state=(~(get by connections.state) duct) + ((trace 0 |.("{<duct>} invalid outstanding connection")) `state) + :: + |^ ^- [(list move) server-state] + :: + ?- -.http-event + :: + %start + ?^ response-header.u.connection-state + ((trace 0 |.("{<duct>} error multiple start")) error-connection) + :: extend the request's session's + cookie's life + :: + =^ response-header sessions.auth.state + =, authentication + =* session-id session-id.u.connection-state + =* sessions sessions.auth.state + =* inbound inbound-request.u.connection-state + =* headers headers.response-header.http-event + :: + ?~ ses=(~(get by sessions) session-id) + :: if the session has expired since the request was opened, + :: tough luck, we don't create/revive sessions here + :: + [response-header.http-event sessions] + =/ kind ?:(?=(%fake -.identity.u.ses) %guest %auth) + =/ timeout + =, session-timeout + ?:(?=(%guest kind) guest auth) + :_ %+ ~(put by sessions) session-id + u.ses(expiry-time (add now timeout)) + =- response-header.http-event(headers -) + =/ cookie=(pair @t @t) + ['set-cookie' (session-cookie-string session-id `kind)] + |- + ?~ headers + [cookie ~] + ?: &(=(key.i.headers p.cookie) =(value.i.headers q.cookie)) + headers + [i.headers $(headers t.headers)] + :: + =* connection u.connection-state + :: + :: if the request was a simple cors request from an approved origin + :: append the necessary cors headers to the response + :: + =/ origin=(unit origin) + %+ get-header:http 'origin' + header-list.request.inbound-request.connection + =? headers.response-header + ?& ?=(^ origin) + (~(has in approved.cors-registry.state) u.origin) + == + %^ set-header:http 'Access-Control-Allow-Origin' u.origin + %^ set-header:http 'Access-Control-Allow-Credentials' 'true' + headers.response-header + :: + =. response-header.http-event response-header + =. connections.state + ?: complete.http-event + :: XX optimize by not requiring +put:by in +request + :: + (~(del by connections.state) duct) + :: + %- (trace 2 |.("{<duct>} start")) + %+ ~(put by connections.state) duct + %= connection + response-header `response-header + bytes-sent ?~(data.http-event 0 p.u.data.http-event) + == + :: + pass-response + :: + %continue + ?~ response-header.u.connection-state + %. error-connection + (trace 0 |.("{<duct>} error continue without start")) + :: + =. connections.state + ?: complete.http-event + %- (trace 2 |.("{<duct>} completed")) + (~(del by connections.state) duct) + :: + %- (trace 2 |.("{<duct>} continuing")) + ?~ data.http-event + connections.state + :: + %+ ~(put by connections.state) duct + =* size p.u.data.http-event + =* conn u.connection-state + conn(bytes-sent (add size bytes-sent.conn)) + :: + pass-response + :: + %cancel + :: todo: log this differently from an ise. + :: + ((trace 1 |.("cancel http event")) error-connection) + == + :: + ++ pass-response + ^- [(list move) server-state] + [[duct %give %response http-event]~ state] + :: + ++ error-connection + :: todo: log application error + :: + :: remove all outstanding state for this connection + :: + =. connections.state + (~(del by connections.state) duct) + :: respond to outside with %error + :: + ^- [(list move) server-state] + :_ state + :- [duct %give %response %cancel ~] + ?. ?=(%app -.action.u.connection-state) + ~ + :_ ~ + =, u.connection-state + %- %+ trace 1 + |.("leaving subscription to {<app.action>}") + (deal-as /watch-response/[eyre-id] identity our app.action %leave ~) + -- + :: +set-response: remember (or update) a cache mapping + :: + ++ set-response + |= [url=@t entry=(unit cache-entry)] + ^- [(list move) server-state] + =/ aeon ?^(prev=(~(get by cache.state) url) +(aeon.u.prev) 1) + =. cache.state (~(put by cache.state) url [aeon entry]) + :_ state + ::NOTE during boot, userspace might've sent us this before we received + :: our first %born, with which we initialize the outgoing-duct. + :: it's fine to hold off on the %grow here, we'll re-send them + :: whenever we finally receive the %born. + ?: =(~ outgoing-duct.state) ~ + [outgoing-duct.state %give %grow /cache/(scot %ud aeon)/(scot %t url)]~ + :: +add-binding: conditionally add a pairing between binding and action + :: + :: Adds =binding =action if there is no conflicting bindings. + :: + ++ add-binding + |= [=binding =action] + ^- [(list move) server-state] + =^ success bindings.state + :: prevent binding in reserved namespaces + :: + ?: ?| ?=([%'~' *] path.binding) :: eyre + ?=([%'~_~' *] path.binding) :: runtime + ?=([%'_~_' *] path.binding) :: scries + == + [| bindings.state] + [& (insert-binding [binding duct action] bindings.state)] + :_ state + [duct %give %bound & binding]~ + :: +remove-binding: removes a binding if it exists and is owned by this duct + :: + ++ remove-binding + |= =binding + :: + ^- server-state + %_ state + bindings + %+ skip bindings.state + |= [item-binding=^binding item-duct=^duct =action] + ^- ? + &(=(item-binding binding) =(item-duct duct)) + == + :: +get-action-for-binding: finds an action for an incoming web request + :: + ++ get-action-for-binding + |= [raw-host=(unit @t) url=@t] + ^- [=action suburl=@t] + :: process :raw-host + :: + :: If we are missing a 'Host:' header, if that header is a raw IP + :: address, or if the 'Host:' header refers to [our].urbit.org, we want + :: to return ~ which means we're unidentified and will match against any + :: wildcard matching. + :: + :: Otherwise, return the site given. + :: + =/ host=(unit @t) + ?~ raw-host + ~ + :: Parse the raw-host so that we can ignore ports, usernames, etc. + :: + =+ parsed=(rush u.raw-host simplified-url-parser) + ?~ parsed + ~ + :: if the url is a raw IP, assume default site. + :: + ?: ?=([%ip *] -.u.parsed) + ~ + :: if the url is "localhost", assume default site. + :: + ?: =([%site 'localhost'] -.u.parsed) + ~ + :: render our as a tape, and cut off the sig in front. + :: + =/ with-sig=tape (scow %p our) + ?> ?=(^ with-sig) + ?: =(u.raw-host (crip t.with-sig)) + :: [our].urbit.org is the default site + :: + ~ + :: + raw-host + :: url is the raw thing passed over the 'Request-Line'. + :: + :: todo: this is really input validation, and we should return a 500 to + :: the client. + :: + =/ request-line (parse-request-line url) + =/ parsed-url=(list @t) site.request-line + =? parsed-url ?=([%'~' %channel-jam *] parsed-url) + parsed-url(i.t %channel) + :: + =/ bindings bindings.state + |- + :: + ?~ bindings + [[%four-oh-four ~] url] + :: + ?. (host-matches site.binding.i.bindings raw-host) + $(bindings t.bindings) + ?~ suffix=(find-suffix path.binding.i.bindings parsed-url) + $(bindings t.bindings) + :: + :- action.i.bindings + %^ cat 3 + %+ roll + ^- (list @t) + (join '/' (flop ['' u.suffix])) + (cury cat 3) + ?~ ext.request-line '' + (cat 3 '.' u.ext.request-line) + :: +give-session-tokens: send valid local session tokens to unix + :: + ++ give-session-tokens + ^- move + :- outgoing-duct.state + :+ %give %sessions + %- sy + %+ murn ~(tap by sessions.auth.state) + |= [sid=@uv session] + ?. ?=(%ours -.identity) ~ + (some (scot %uv sid)) + :: +new-session-key + :: + ++ new-session-key + |- ^- @uv + =/ candidate=@uv (~(raw og (shas %session-key eny)) 128) + ?. (~(has by sessions.auth.state) candidate) + candidate + $(eny (shas %try-again candidate)) + :: + ++ deal-as + |= [=wire identity=$@(@p identity) =ship =dude:gall =task:agent:gall] + ^- move + =/ from=@p + ?@ identity identity + ?+(-.identity who.identity %ours our) + [duct %pass wire %g %deal [from ship /eyre] dude task] + :: + ++ trace + |= [pri=@ print=(trap tape)] + ?: (lth verb.state pri) same + (slog leaf+"eyre: {(print)}" ~) + -- +:: +++ forwarded-params + |= =header-list:http + ^- (unit (list (map @t @t))) + %+ biff + (get-header:http 'forwarded' header-list) + unpack-header:http +:: +++ forwarded-for + |= forwards=(list (map @t @t)) + ^- (unit address) + ?. ?=(^ forwards) ~ + =* forward i.forwards + ?~ for=(~(get by forward) 'for') ~ + ::NOTE per rfc7239, non-ip values are also valid. they're not useful + :: for the general case, so we ignore them here. if needed, + :: request handlers are free to inspect the headers themselves. + :: + %+ rush u.for + ;~ sfix + ;~(pose (stag %ipv4 ip4) (stag %ipv6 (ifix [sel ser] ip6))) + ;~(pose ;~(pfix col dim:ag) (easy ~)) + == +:: +++ forwarded-secure + |= forwards=(list (map @t @t)) + ^- (unit ?) + ?. ?=(^ forwards) ~ + =* forward i.forwards + ?~ proto=(~(get by forward) 'proto') ~ + ?+ u.proto ~ + %http `| + %https `& + == +:: +++ forwarded-host + |= forwards=(list (map @t @t)) + ^- (unit @t) + ?. ?=(^ forwards) ~ + (~(get by i.forwards) 'host') +:: +++ parse-request-line + |= url=@t + ^- [[ext=(unit @ta) site=(list @t)] args=(list [key=@t value=@t])] + (fall (rush url ;~(plug apat:de-purl:html yque:de-purl:html)) [[~ ~] ~]) +:: +insert-binding: add a new binding, replacing any existing at its path +:: +++ insert-binding + |= $: new=[=binding =duct =action] + bindings=(list [=binding =duct =action]) + == + ^+ bindings + ?~ bindings [new]~ + =* bid binding.i.bindings + :: replace already bound paths + :: + ?: =([site path]:bid [site path]:binding.new) + ~> %slog.[0 leaf+"eyre: replacing existing binding at {<`path`path.bid>}"] + [new t.bindings] + :: if new comes before bid, prepend it. + :: otherwise, continue our search. + :: + =; new-before-bid=? + ?: new-before-bid [new bindings] + [i.bindings $(bindings t.bindings)] + ?: =(site.binding.new site.bid) + (aor path.bid path.binding.new) + (aor (fall site.bid '') (fall site.binding.new '')) +:: +++ channel-wire + |= [channel-id=@t request-id=@ud] + ^- wire + /channel/subscription/[channel-id]/(scot %ud request-id) +:: +++ subscription-wire + |= [channel-id=@t request-id=@ud as=$@(@p identity) =ship app=term] + ^- wire + =/ from=@p + ?@ as as + ?+(-.as who.as %ours our) + %+ weld (channel-wire channel-id request-id) + ::NOTE including the originating identity is important for the band-aid + :: solution currently present in +on-gall-response, where we may + :: need to issue a %leave after we've forgotten the identity with + :: which the subscription was opened. + /(scot %p ship)/[app]/(scot %p from) +:: +++ scry-mime + |= [now=@da rof=roof =gang ext=(unit @ta) pax=path] + |^ ^- (each mime tape) + :: parse + :: + =/ u=(unit [view=term bem=beam]) + ?. ?=([@ @ @ @ *] pax) ~ + ?~ view=(slaw %tas i.t.pax) ~ + ?~ path=(expand-path t.t.pax) ~ + ?~ beam=(de-beam u.path) ~ + `[u.view u.beam] + ?~ u [%| "invalid scry path"] + :: perform scry + :: + ?~ res=(rof gang /eyre u.u) [%| "failed scry"] + ?~ u.res [%| "no scry result"] + =* mark p.u.u.res + =* vase q.u.u.res + :: convert to mime via ext + :: + =/ dysk (conversion-desk u.u) + ?: ?=(%| -.dysk) [%| p.dysk] + =/ ext (fall ext %mime) + =/ mym (convert vase mark ext p.dysk) + ?: ?=(%| -.mym) [%| p.mym] + =/ mym (convert p.mym ext %mime p.dysk) + ?: ?=(%| -.mym) [%| p.mym] + [%& !<(mime p.mym)] + :: + ++ expand-path + |= a=path + ^- (unit path) + =/ vez (vang | (en-beam [our %base da+now] ~)) + (rush (spat a) (sear plex:vez (stag %clsg ;~(pfix fas poor:vez)))) + :: + ++ conversion-desk + |= [view=term =beam] + ^- (each desk tape) + ?: =(%$ q.beam) [%& %base] + ?+ (end 3 view) [%& %base] + %c + [%& q.beam] + %g + =/ res (rof [~ ~] /eyre %gd [our q.beam da+now] /$) + ?. ?=([~ ~ *] res) + [%| "no desk for app {<q.beam>}"] + [%& !<(=desk q.u.u.res)] + == + :: + ++ convert + |= [=vase from=mark to=mark =desk] + ^- (each ^vase tape) + ?: =(from to) [%& vase] + =/ tub (rof [~ ~] /eyre %cc [our desk da+now] /[from]/[to]) + ?. ?=([~ ~ %tube *] tub) + [%| "no tube from {(trip from)} to {(trip to)}"] + =/ tube !<(tube:clay q.u.u.tub) + =/ res (mule |.((tube vase))) + ?: ?=(%| -.res) + [%| "failed tube from {(trip from)} to {(trip to)}"] + [%& +.res] + -- +-- +:: end the =~ +:: +. == +:: begin with a default +axle as a blank slate +:: +=| ax=axle +:: a vane is activated with current date, entropy, and a namespace function +:: +|= [now=@da eny=@uvJ rof=roof] +:: allow jets to be registered within this core +:: +~% %http-server ..part ~ +|% +++ call + ~/ %eyre-call + |= [=duct dud=(unit goof) wrapped-task=(hobo task)] + ^- [(list move) _http-server-gate] + :: + =/ task=task ((harden task) wrapped-task) + ~& > eyre-task=[-.task duct=duct] + :: + :: XX handle more error notifications + :: + ?^ dud + :_ http-server-gate + :: always print the error trace + :: + :- [duct %slip %d %flog %crud [-.task tang.u.dud]] + ^- (list move) + :: if a request caused the crash, respond with a 500 + :: + ?. ?=(?(%request %request-local) -.task) ~ + ^~ + =/ data (as-octs:mimes:html 'crud!') + =/ head + :~ ['content-type' 'text/html'] + ['content-length' (crip (a-co:co p.data))] + == + [duct %give %response %start 500^head `data &]~ + :: due to an error handling bug in earlier versions of +take, + :: we may need to make sure the timeout timer for sessions still exists. + :: see also urbit/urbit#7103 + :: + ?: check-session-timer.server-state.ax + :: we do this cleanup exactly once + :: + =. check-session-timer.server-state.ax | + :: if there are no sessions after running the +call, + :: we don't need a timer, so we don't need to set it + :: + ::NOTE hazard, must get state from .etc going forward + =/ [moz=(list move) etc=_http-server-gate] $ + :_ etc + ^- (list move) + ?: =(~ sessions.auth.server-state.ax.etc) moz + :: if .moz is already setting the timer, + :: we don't need to do it here + :: + ?: %+ lien moz + |= m=move + ?=([* %pass [%sessions %expire ~] *] m) + moz + :: find out from behn if there isn't already a timer set. + :: this is not ideal, but we have no other way of knowing, + :: and don't want to set duplicate timers... + :: + ?: ?~ res=(rof [~ ~] /eyre %bx [our %$ da+now] /debug/timers) | + ?~ u.res | + %+ lien !<((list [@da ^duct]) q.u.u.res) + |= [@da d=^duct] + ?=([[%eyre %sessions %expire ~] *] d) + moz + :: we need a timer, but aren't setting one, and don't have one, + :: so prepend a session expire timer to the .moz + :: + [[duct %pass /sessions/expire %b %wait now] moz] + :: %init: tells us what our ship name is + :: + ?: ?=(%init -.task) + :: initial value for the login handler + :: + =. bindings.server-state.ax + =- (roll - insert-binding) + ^- (list [binding ^duct action]) + :~ [[~ /~/login] duct [%authentication ~]] + [[~ /~/eauth] duct [%eauth ~]] + [[~ /~/logout] duct [%logout ~]] + [[~ /~/channel] duct [%channel ~]] + [[~ /~/scry] duct [%scry ~]] + [[~ /~/name] duct [%name ~]] + [[~ /~/host] duct [%host ~]] + [[~ /~/ip] duct [%ip ~]] + [[~ /~/boot] duct [%boot ~]] + [[~ /~/sponsor] duct [%sponsor ~]] + == + [~ http-server-gate] + :: %trim: in response to memory pressure + :: + :: Cancel all inactive channels + :: XX cancel active too if =(0 trim-priority) ? + :: + ?: ?=(%trim -.task) + =* event-args [[eny duct now rof] server-state.ax] + =* by-channel by-channel:(per-server-event event-args) + =* channel-state channel-state.server-state.ax + :: + =/ inactive=(list @t) + =/ full=(set @t) ~(key by session.channel-state) + =/ live=(set @t) + (~(gas in *(set @t)) ~(val by duct-to-key.channel-state)) + ~(tap in (~(dif in full) live)) + :: + ?: =(~ inactive) + [~ http-server-gate] + :: + =/ len=tape (scow %ud (lent inactive)) + ~> %slog.[0 leaf+"eyre: trim: closing {len} inactive channels"] + :: + =| moves=(list (list move)) + |- ^- [(list move) _http-server-gate] + =* channel-id i.inactive + ?~ inactive + [(zing (flop moves)) http-server-gate] + :: discard channel state, and cancel any active gall subscriptions + :: + =^ mov server-state.ax (discard-channel:by-channel channel-id |) + $(moves [mov moves], inactive t.inactive) + :: + :: %vega: notifies us of a completed kernel upgrade + :: + ?: ?=(%vega -.task) + [~ http-server-gate] + :: %born: new unix process + :: + ?: ?=(%born -.task) + :: close previously open connections + :: + :: When we have a new unix process, every outstanding open connection is + :: dead. For every duct, send an implicit close connection. + :: + =^ closed-connections=(list move) server-state.ax + =/ connections=(list [=^duct *]) + ~(tap by connections.server-state.ax) + :: + =| closed-connections=(list move) + |- + ?~ connections + [closed-connections server-state.ax] + :: + =/ event-args + [[eny duct.i.connections now rof] server-state.ax] + =/ cancel-request cancel-request:(per-server-event event-args) + =^ moves server-state.ax cancel-request + :: + $(closed-connections (weld moves closed-connections), connections t.connections) + :: save duct for future %give to unix + :: + =. outgoing-duct.server-state.ax duct + :: send all cache mappings to runtime + :: + =/ cache-moves=(list move) + %+ turn ~(tap by cache.server-state.ax) + |= [url=@t cache-val=[aeon=@ud val=(unit cache-entry)]] + [duct %give %grow /cache/(scot %u aeon.cache-val)/(scot %t url)] + :: + :_ http-server-gate + :* :: hand back default configuration for now + :: + [duct %give %set-config http-config.server-state.ax] + :: provide a list of valid auth tokens + :: + =< give-session-tokens + (per-server-event [eny duct now rof] server-state.ax) + :: + (zing ~[closed-connections cache-moves]) + == + :: + ?: ?=(%code-changed -.task) + ~> %slog.[0 leaf+"eyre: code-changed: throwing away local sessions"] + =* event-args [[eny duct now rof] server-state.ax] + :: find all the %ours sessions, we must close them + :: + =/ siz=(list @uv) + %+ murn ~(tap by sessions.auth.server-state.ax) + |= [sid=@uv session] + ?:(?=(%ours -.identity) (some sid) ~) + =| moves=(list (list move)) + |- ^- [(list move) _http-server-gate] + ?~ siz + [(zing (flop moves)) http-server-gate] + :: discard the session, clean up its channels + :: + =^ mov server-state.ax + (close-session:authentication:(per-server-event event-args) i.siz |) + $(moves [mov moves], siz t.siz) + :: + ?: ?=(%eauth-host -.task) + ?: =(user.endpoint.auth.server-state.ax host.task) + [~ http-server-gate] + =. user.endpoint.auth.server-state.ax host.task + =. time.endpoint.auth.server-state.ax now + [~ http-server-gate] + :: + :: all other commands operate on a per-server-event + :: + =/ event-args [[eny duct now rof] server-state.ax] + =/ server (per-server-event event-args) + :: + ?- -.task + :: %live: notifies us of the ports of our live http servers + :: + %live + =. ports.server-state.ax +.task + :: enable http redirects if https port live and cert set + :: + =. redirect.http-config.server-state.ax + &(?=(^ secure.task) ?=(^ secure.http-config.server-state.ax)) + [~ http-server-gate] + :: %rule: updates our http configuration + :: + %rule + ?- -.http-rule.task + :: %cert: install tls certificate + :: + %cert + =* config http-config.server-state.ax + ?: =(secure.config cert.http-rule.task) + [~ http-server-gate] + =. secure.config cert.http-rule.task + =. redirect.config + ?& ?=(^ secure.ports.server-state.ax) + ?=(^ cert.http-rule.task) + == + :_ http-server-gate + =* out-duct outgoing-duct.server-state.ax + ?~ out-duct ~ + [out-duct %give %set-config config]~ + :: %turf: add or remove domain name + :: + %turf + =* domains domains.server-state.ax + =/ mod=(set turf) + ?- -.action.http-rule.task + %put (~(put in domains) turf.action.http-rule.task) + %del (~(del in domains) turf.action.http-rule.task) + %new turfs.action.http-rule.task + == + ?: &(!?=(%new -.action.http-rule.task) =(domains mod)) + [~ http-server-gate] + =. domains mod + :_ http-server-gate + =/ cmd + [%acme %poke `cage`[%acme-order !>(mod)]] + [duct %pass /acme/order %g %deal [our our /eyre] cmd]~ + == + :: + %plea + ~| path.plea.task + ?> ?=([%eauth %'0' ~] path.plea.task) + =+ plea=;;(eauth-plea payload.plea.task) + =^ moves server-state.ax + (on-plea:server:eauth:authentication:server ship.task plea) + [moves http-server-gate] + :: + %request + =^ moves server-state.ax (request:server +.task) + [moves http-server-gate] + :: + %request-local + =^ moves server-state.ax (request-local:server +.task) + [moves http-server-gate] + :: + %cancel-request + =^ moves server-state.ax cancel-request:server + [moves http-server-gate] + :: + %connect + =^ moves server-state.ax + %+ add-binding:server binding.task + [%app app.task] + [moves http-server-gate] + :: + %serve + =^ moves server-state.ax + %+ add-binding:server binding.task + [%gen generator.task] + [moves http-server-gate] + :: + %disconnect + =. server-state.ax (remove-binding:server binding.task) + [~ http-server-gate] + :: + %approve-origin + =. cors-registry.server-state.ax + =, cors-registry.server-state.ax + :+ (~(del in requests) origin.task) + (~(put in approved) origin.task) + (~(del in rejected) origin.task) + [~ http-server-gate] + :: + %reject-origin + =. cors-registry.server-state.ax + =, cors-registry.server-state.ax + :+ (~(del in requests) origin.task) + (~(del in approved) origin.task) + (~(put in rejected) origin.task) + [~ http-server-gate] + :: + %spew + =. verb.server-state.ax veb.task + `http-server-gate + :: + %set-response + =^ moves server-state.ax (set-response:server +.task) + [moves http-server-gate] + %websocket-event + ~& websocket-event=+.task + =^ moves server-state.ax (ws-event:server +.task) + [moves http-server-gate] + + + %websocket-handshake + =^ moves server-state.ax (ws-handshake:server +.task) + [moves http-server-gate] + == +:: +++ take + ~/ %eyre-take + |= [=wire =duct dud=(unit goof) =sign] + ~& >>> duct=duct + ~& >> take-wire-eyre=wire + ^- [(list move) _http-server-gate] + => %= . + sign + ?: ?=(%gall -.sign) + ?> ?=(%unto +<.sign) + sign + sign + == + :: :wire must at least contain two parts, the type and the build + :: + ?> ?=([@ *] wire) + :: + |^ ^- [(list move) _http-server-gate] + :: + ?: ?=(%eauth i.wire) + eauth + ?^ dud + ~|(%eyre-take-dud (mean tang.u.dud)) + ?+ i.wire + ~|([%bad-take-wire wire] !!) + :: + %run-app-request run-app-request + %watch-response watch-response + %sessions sessions + %channel channel + %acme acme-ack + %conversion-cache `http-server-gate + %run-ws-app-request run-ws-app-request + %ws-watch-response watch-ws-response + == + :: + ++ watch-ws-response + =/ event-args [[eny duct now rof] server-state.ax] + ?> ?=([@ *] t.wire) + ~& >> ws-sign=[`@t`-.sign `@t`+<.sign ((soft @t) +>-.sign)] + ?+ sign `http-server-gate + [%gall %unto %watch-ack *] + ?~ p.p.sign + :: received a positive acknowledgment: take no action + :: + [~ http-server-gate] + :: we have an error; propagate it to the client + :: + ~& gall-error=u.p.p.sign + =/ handle-gall-error + handle-gall-error:(per-server-event event-args) + =^ moves server-state.ax (handle-gall-error u.p.p.sign) + [moves http-server-gate] + [%gall %unto %fact *] + =/ mark p.cage.p.sign + ~& > eyre-ws-response-fact=mark + ?. ?=(%websocket-response mark) + =/ handle-gall-error + handle-gall-error:(per-server-event event-args) + =^ moves server-state.ax + (handle-gall-error leaf+"eyre bad mark {(trip mark)}" ~) + [moves http-server-gate] + ~& websocket-vase=q.q.cage.p.sign + =/ event !<([@ websocket-event] q.cage.p.sign) + =/ handle-ws-response handle-ws-response:(per-server-event event-args) + =^ moves server-state.ax + (handle-ws-response event) + [moves http-server-gate] + + == + ++ run-ws-app-request + ~& run-ws-app-req=sign + + `http-server-gate + + ++ run-app-request + :: + ?> ?=([%gall %unto *] sign) + :: + :: + :: ~& run-app-req-eyre=p.sign + ?> ?=([%poke-ack *] p.sign) + ?> ?=([@ *] t.wire) + ?~ p.p.sign + :: received a positive acknowledgment: take no action + :: + [~ http-server-gate] + :: we have an error; propagate it to the client + :: + =/ event-args [[eny duct now rof] server-state.ax] + =/ handle-gall-error + handle-gall-error:(per-server-event event-args) + =^ moves server-state.ax + (handle-gall-error u.p.p.sign) + [moves http-server-gate] + :: + ++ watch-response + :: + =/ event-args [[eny duct now rof] server-state.ax] + :: + ?> ?=([@ *] t.wire) + ~& >> http-sign=[`@t`-.sign `@t`+<.sign ((soft @t) +>-.sign)] + ?: ?=([%gall %unto %watch-ack *] sign) + ?~ p.p.sign + :: received a positive acknowledgment: take no action + :: + [~ http-server-gate] + :: we have an error; propagate it to the client + :: + ~& eyre-watch-response-gall-error=duct + =/ handle-gall-error + handle-gall-error:(per-server-event event-args) + =^ moves server-state.ax (handle-gall-error u.p.p.sign) + [moves http-server-gate] + :: + ?: ?=([%gall %unto %kick ~] sign) + =/ handle-response handle-response:(per-server-event event-args) + =^ moves server-state.ax + (handle-response %continue ~ &) + [moves http-server-gate] + :: + ?> ?=([%gall %unto %fact *] sign) + =/ =mark p.cage.p.sign + ~& eyre-watch-response-fact=mark + =/ =vase q.cage.p.sign + ?. ?= ?(%http-response-header %http-response-data %http-response-cancel) + mark + =/ handle-gall-error + handle-gall-error:(per-server-event event-args) + =^ moves server-state.ax + (handle-gall-error leaf+"eyre bad mark {(trip mark)}" ~) + [moves http-server-gate] + :: + =/ =http-event:http + ?- mark + %http-response-header [%start !<(response-header:http vase) ~ |] + %http-response-data [%continue !<((unit octs) vase) |] + %http-response-cancel [%cancel ~] + == + =/ handle-response handle-response:(per-server-event event-args) + =^ moves server-state.ax + (handle-response http-event) + [moves http-server-gate] + :: + ++ channel + :: + =/ event-args [[eny duct now rof] server-state.ax] + :: channel callback wires are triples. + :: + ?> ?=([@ @ @t *] wire) + :: + ?+ i.t.wire + ~|([%bad-channel-wire wire] !!) + :: + %timeout + ?> ?=([%behn %wake *] sign) + ?^ error.sign + [[duct %slip %d %flog %crud %wake u.error.sign]~ http-server-gate] + =* id i.t.t.wire + %- %+ trace:(per-server-event event-args) 1 + |.("{(trip id)} cancelling channel due to timeout") + =^ moves server-state.ax + (discard-channel:by-channel:(per-server-event event-args) id &) + [moves http-server-gate] + :: + %heartbeat + =/ on-channel-heartbeat + on-channel-heartbeat:by-channel:(per-server-event event-args) + =^ moves server-state.ax + (on-channel-heartbeat i.t.t.wire) + [moves http-server-gate] + :: + ?(%poke %subscription) + ?> ?=([%gall %unto *] sign) + ~| eyre-sub=wire + ?> ?=([@ @ @t @ *] wire) + ?< ?=(%raw-fact -.p.sign) + =* channel-id i.t.t.wire + =* request-id i.t.t.t.wire + =* extra-wire t.t.t.t.wire + =/ on-gall-response + on-gall-response:by-channel:(per-server-event event-args) + :: ~& [%gall-response sign] + =^ moves server-state.ax + %- on-gall-response + [channel-id (slav %ud request-id) extra-wire p.sign] + [moves http-server-gate] + == + :: + ++ sessions + :: + ?> ?=([%behn %wake *] sign) + :: + ?^ error.sign + :_ http-server-gate + :: we must not drop the timer! so we kick the can down the road a day, + :: and hope it will run successfully later... + :: + :~ [duct %slip %d %flog %crud %wake u.error.sign] + [duct %pass /sessions/expire %b %wait (add now ~d1)] + == + ::NOTE we are not concerned with expiring channels that are still in + :: use. we require acks for messages, which bump their session's + :: timer. channels have their own expiry timer, too. + :: remove cookies that have expired + :: + =* sessions sessions.auth.server-state.ax + =. sessions.auth.server-state.ax + %- ~(gas by *(map @uv session)) + %+ skip ~(tap in sessions) + |= [cookie=@uv session] + (lth expiry-time now) + :: if there's any cookies left, set a timer for the next expected expiry + :: + ^- [(list move) _http-server-gate] + :_ http-server-gate + :- =< give-session-tokens + (per-server-event [eny duct now rof] server-state.ax) + ?: =(~ sessions) ~ + =; next-expiry=@da + [duct %pass /sessions/expire %b %wait next-expiry]~ + %+ roll ~(tap by sessions) + |= [[@uv session] next=@da] + ?: =(*@da next) expiry-time + (min next expiry-time) + :: + ++ eauth + =* auth auth.server-state.ax + =* args [[eny duct now rof] server-state.ax] + ^- [(list move) _http-server-gate] + ~| [wire +<.sign] + ?+ t.wire !! + [%plea @ ~] + =/ =ship (slav %p i.t.t.wire) + :: + ?: |(?=(^ dud) ?=([%ames %lost *] sign)) + %- %+ trace:(per-server-event args) 0 + ?~ dud |.("eauth: lost boon from {(scow %p ship)}") + |.("eauth: crashed on %{(trip +<.sign)} from {(scow %p ship)}") + ::NOTE when failing on pending attempts, we just wait for the timer + :: to clean up. when failing on live sessions, well, we should + :: just be careful not to crash when receiving %shut boons. + :: (we do not want to have the nonce in the wire, so this is the + :: best handling we can do. the alternative is tracking) + [~ http-server-gate] + :: + ?: ?=([%ames %done *] sign) + =^ moz server-state.ax + %. [ship ?=(~ error.sign)] + on-done:client:eauth:authentication:(per-server-event args) + [moz http-server-gate] + :: + ?> ?=([%ames %boon *] sign) + =/ boon ;;(eauth-boon payload.sign) + =^ moz server-state.ax + %. [ship boon] + on-boon:client:eauth:authentication:(per-server-event args) + [moz http-server-gate] + :: + [%keen @ @ ~] + =/ client=@p (slav %p i.t.t.wire) + =/ nonce=@uv (slav %uv i.t.t.t.wire) + :: + ?^ dud + =^ moz server-state.ax + %. [client nonce] + on-fail:server:eauth:authentication:(per-server-event args) + [moz http-server-gate] + :: + ?> ?& ?=([%ames %sage *] sign) + =(client ship.p.sage.sign) + == + =/ url=(unit @t) + ?~ q.sage.sign ~ + ;;((unit @t) q.q.sage.sign) + =^ moz server-state.ax + ?~ url + %. [client nonce] + on-fail:server:eauth:authentication:(per-server-event args) + %. [client nonce u.url] + on-tune:server:eauth:authentication:(per-server-event args) + [moz http-server-gate] + :: + [%expire %visiting @ @ ~] + ?> ?=([%behn %wake *] sign) + =/ server=@p (slav %p i.t.t.t.wire) + =/ nonce=@uv (slav %uv i.t.t.t.t.wire) + =^ moz server-state.ax + %. [server nonce] + expire:client:eauth:authentication:(per-server-event args) + [~ http-server-gate] + :: + [%expire %visitors @ ~] + =/ nonce=@uv (slav %uv i.t.t.t.wire) + =^ moz server-state.ax + (expire:server:eauth:authentication:(per-server-event args) nonce) + [moz http-server-gate] + == + :: + ++ acme-ack + ?> ?=([%gall %unto *] sign) + :: + ?> ?=([%poke-ack *] p.sign) + ?~ p.p.sign + :: received a positive acknowledgment: take no action + :: + [~ http-server-gate] + :: received a negative acknowledgment: XX do something + :: + [((slog u.p.p.sign) ~) http-server-gate] + -- +:: +++ http-server-gate ..$ +:: +load: migrate old state to new state (called on vane reload) +:: +++ load + => |% + +$ axle-any + $% [date=%~2020.10.18 server-state=server-state-0] + [date=%~2022.7.26 server-state=server-state-0] + [date=%~2023.2.17 server-state=server-state-1] + [date=%~2023.3.16 server-state=server-state-2] + [date=%~2023.4.11 server-state-3] + [date=%~2023.5.15 server-state-4] + [date=%~2024.8.20 server-state-4] + [date=%~2025.1.31 server-state] + == + :: + +$ server-state-0 + $: bindings=(list [=binding =duct =action]) + =cors-registry + connections=(map duct outstanding-connection-3) + auth=authentication-state-3 + channel-state=channel-state-2 + domains=(set turf) + =http-config + ports=[insecure=@ud secure=(unit @ud)] + outgoing-duct=duct + == + :: + +$ server-state-1 + $: bindings=(list [=binding =duct =action]) + =cors-registry + connections=(map duct outstanding-connection-3) + auth=authentication-state-3 + channel-state=channel-state-2 + domains=(set turf) + =http-config + ports=[insecure=@ud secure=(unit @ud)] + outgoing-duct=duct + verb=@ :: <- new + == + :: + +$ server-state-2 + $: bindings=(list [=binding =duct =action]) + cache=(map url=@t [aeon=@ud val=(unit cache-entry)]) :: <- new + =cors-registry + connections=(map duct outstanding-connection-3) + auth=authentication-state-3 + channel-state=channel-state-2 + domains=(set turf) + =http-config + ports=[insecure=@ud secure=(unit @ud)] + outgoing-duct=duct + verb=@ + == + +$ channel-state-2 + $: session=(map @t channel-2) + duct-to-key=(map duct @t) + == + +$ channel-2 + $: state=(each timer duct) + next-id=@ud + last-ack=@da + events=(qeu [id=@ud request-id=@ud channel-event=channel-event-2]) + unacked=(map @ud @ud) + subscriptions=(map @ud [ship=@p app=term =path duc=duct]) + heartbeat=(unit timer) + == + +$ channel-event-2 + $% $>(%poke-ack sign:agent:gall) + $>(%watch-ack sign:agent:gall) + $>(%kick sign:agent:gall) + [%fact =mark =noun] + == + :: + +$ server-state-3 + $: bindings=(list [=binding =duct =action]) + cache=(map url=@t [aeon=@ud val=(unit cache-entry)]) + =cors-registry + connections=(map duct outstanding-connection-3) + auth=authentication-state-3 + channel-state=channel-state-3 + domains=(set turf) + =http-config + ports=[insecure=@ud secure=(unit @ud)] + outgoing-duct=duct + verb=@ + == + +$ outstanding-connection-3 + $: =action + =inbound-request + response-header=(unit response-header:http) + bytes-sent=@ud + == + +$ authentication-state-3 sessions=(map @uv session-3) + +$ session-3 + $: expiry-time=@da + channels=(set @t) + == + +$ channel-state-3 + $: session=(map @t channel-3) + duct-to-key=(map duct @t) + == + +$ channel-3 + $: mode=?(%json %jam) + state=(each timer duct) + next-id=@ud + last-ack=@da + events=(qeu [id=@ud request-id=@ud =channel-event]) + unacked=(map @ud @ud) + subscriptions=(map @ud [ship=@p app=term =path duc=duct]) + heartbeat=(unit timer) + == + :: + +$ server-state-4 + $: bindings=(list [=binding =duct =action]) + cache=(map url=@t [aeon=@ud val=(unit cache-entry)]) + =cors-registry + connections=(map duct outstanding-connection) + auth=authentication-state + =channel-state + domains=(set turf) + =http-config + ports=[insecure=@ud secure=(unit @ud)] + outgoing-duct=duct + verb=@ + == + -- + |= old=axle-any + ^+ http-server-gate + ?- -.old + :: + :: adds /~/name + :: + %~2020.10.18 + %= $ + date.old %~2022.7.26 + :: + bindings.server-state.old + %+ insert-binding + [[~ /~/name] outgoing-duct.server-state.old [%name ~]] + bindings.server-state.old + == + :: + :: enables https redirects if certificate configured + :: inits .verb + :: + %~2022.7.26 + =. redirect.http-config.server-state.old + ?& ?=(^ secure.ports.server-state.old) + ?=(^ secure.http-config.server-state.old) + == + $(old [%~2023.2.17 server-state.old(|8 [|8 verb=0]:server-state.old)]) + :: + :: inits .cache + :: + %~2023.2.17 + $(old [%~2023.3.16 [bindings ~ +]:server-state.old]) + :: + :: inits channel mode and desks in unacked events + :: + %~2023.3.16 + :: + :: Prior to this desks were not part of events.channel. + :: When serializing we used to rely on the desk stored in + :: subscriptions.channel, but this state is deleted when we clog. + :: This migration adds the desk to events.channel, but we can not + :: scry in +load to populate the desks in the old events, + :: so we just kick all subscriptions on all channels. + %= $ + date.old %~2023.4.11 + :: + server-state.old + %= server-state.old + session.channel-state + %- ~(run by session.channel-state.server-state.old) + |= c=channel-2 + =; new-events + :- %json + c(events new-events, unacked ~, subscriptions ~) + =| events=(qeu [id=@ud request-id=@ud =channel-event]) + =/ l ~(tap in ~(key by subscriptions.c)) + |- + ?~ l events + %= $ + l t.l + next-id.c +(next-id.c) + events (~(put to events) [next-id.c i.l %kick ~]) + == + == + == + :: + :: guarantees & stores a session for each request, and a @p identity for + :: each session and channel + :: + %~2023.4.11 + %= $ + date.old %~2023.5.15 + :: + connections.old + %- ~(run by connections.old) + |= outstanding-connection-3 + ^- outstanding-connection + [action inbound-request [*@uv [%ours ~]] response-header bytes-sent] + :: + auth.old + :_ [~ ~ [~ ~ now]] + %- ~(run by sessions.auth.old) + |= s=session-3 + ^- session + [[%ours ~] s] + :: + session.channel-state.old + %- ~(run by session.channel-state.old) + |= c=channel-3 + ^- channel + [-.c [%ours ~] +.c] + :: + bindings.old + %+ insert-binding [[~ /~/host] outgoing-duct.old [%host ~]] + %+ insert-binding [[~ /~/eauth] outgoing-duct.old [%eauth ~]] + bindings.old + == + :: + :: adds /~/boot, /~/sponsor and /~/ip + :: + %~2023.5.15 + %= $ + date.old %~2024.8.20 + :: + bindings.old + %+ insert-binding + [[~ /~/boot] outgoing-duct.old [%boot ~]] + %+ insert-binding + [[~ /~/sponsor] outgoing-duct.old [%sponsor ~]] + %+ insert-binding + [[~ /~/ip] outgoing-duct.old [%ip ~]] + bindings.old + == + :: + %~2024.8.20 + %= $ + date.old %~2025.1.31 + verb.old [verb.old check-session-timer=&] + == + :: + %~2025.1.31 + http-server-gate(ax old) + :: + == +:: +stay: produce current state +:: +++ stay `axle`ax +:: +scry: request a path in the urbit namespace +:: +++ scry + ~/ %eyre-scry + ^- roon + |= [lyc=gang pov=path car=term bem=beam] + ^- (unit (unit cage)) + =* ren car + =* why=shop &/p.bem + =* syd q.bem + =/ lot=coin $/r.bem + =* tyl s.bem + :: + ?. ?=(%& -.why) + ~ + =* who p.why + :: + ?. ?=(%$ -.lot) + [~ ~] + ?. =(our who) + ?. =([%da now] p.lot) + ~ + ~& [%r %scry-foreign-host who] + ~ + :: + ?: ?=([%eauth %url ~] tyl) + ?. &(?=(%x ren) ?=(%$ syd)) ~ + =* endpoint endpoint.auth.server-state.ax + ?. ?=(%da -.p.lot) [~ ~] + :: we cannot answer for something prior to the last set time, + :: or something beyond the present moment. + :: + ?: ?| (lth q.p.lot time.endpoint) + (gth q.p.lot now) + == + ~ + :^ ~ ~ %noun + !> ^- (unit @t) + =< eauth-url:eauth:authentication + (per-server-event [eny *duct now rof] server-state.ax) + :: + ?: ?=([%cache @ @ ~] tyl) + ?. &(?=(%x ren) ?=(%$ syd)) ~ + =, server-state.ax + ?~ aeon=(slaw %ud i.t.tyl) [~ ~] + ?~ url=(slaw %t i.t.t.tyl) [~ ~] + ?~ entry=(~(get by cache) u.url) ~ + ?. =(u.aeon aeon.u.entry) ~ + ?~ val=val.u.entry ~ + ?: &(auth.u.val !=([~ ~] lyc)) ~ + ``noun+!>(u.val) + :: + ?: &(?=(%x ren) ?=([%range @ @ @ *] tyl)) + |^ + =/ beg=(unit @ud) (slaw %ud i.t.tyl) + =/ end=(unit @ud) (slaw %ud i.t.t.tyl) + =* vew i.t.t.t.tyl + =* rest t.t.t.t.tyl + =/ mym (scry-mime now rof lyc ~ [%$ vew (en-beam -.bem rest)]) + ?: ?=(%| -.mym) ~ + =* mime p.mym + ?~ range=(get-range [beg end] p.q.mime) + :^ ~ ~ %noun + !> ^- cache-entry + :- ?=(^ lyc) + :+ %payload + :- 416 + ['content-range' (cat 3 'bytes */' (crip (a-co:co p.q.mime)))]^~ + `(as-octs:mimes:html 'requested range not satisfiable') + :: + =/ =octs + %- as-octs:mimes:html + (cut 3 [p.u.range +((sub q.u.range p.u.range))] q.q.mime) + :^ ~ ~ %noun + !> ^- cache-entry + :- ?=(^ lyc) + :+ %payload + :- ?:(=(p.q.mime p.octs) 200 206) + :~ ['accept-ranges' 'bytes'] + ['content-type' (rsh 3 (spat p.mime))] + ['content-length' (crip (a-co:co p.octs))] + :- 'content-range' + %+ rap 3 + :~ 'bytes ' + (crip (a-co:co p.u.range)) '-' + (crip (a-co:co q.u.range)) '/' + (crip (a-co:co p.q.mime)) + == + == + data=[~ octs] + :: + ++ get-range + |= [req=(pair (unit @ud) (unit @ud)) len=@ud] + ^- (unit (pair @ud @ud)) + ?+ req ~ + [^ ~] + ?: (gth u.p.req (dec len)) ~ + `[u.p.req (dec len)] + :: + [~ ^] + ?. (gth u.q.req 0) ~ + `[(sub len (min len u.q.req)) (dec len)] + :: + [^ ^] + ?: |((gth u.p.req (dec len)) (gth u.p.req u.q.req)) + ~ + `[u.p.req (min (dec len) u.q.req)] + == + -- + :: private endpoints + :: + ?. ?=([~ ~] lyc) ~ + :: + ?: &(?=(%x ren) ?=(%$ syd)) + =, server-state.ax + ?+ tyl ~ + [%$ %whey ~] =- ``mass+!>(`(list mass)`-) + :~ bindings+&+bindings.server-state.ax + cache+&+cache.server-state.ax + auth+&+auth.server-state.ax + connections+&+connections.server-state.ax + channels+&+channel-state.server-state.ax + axle+&+ax + == + :: + [%cors ~] ``noun+!>(cors-registry) + [%cors %requests ~] ``noun+!>(requests.cors-registry) + [%cors %approved ~] ``noun+!>(approved.cors-registry) + [%cors %rejected ~] ``noun+!>(rejected.cors-registry) + :: + [%cors ?(%approved %rejected) @ ~] + =* kind i.t.tyl + =* orig i.t.t.tyl + ?~ origin=(slaw %t orig) [~ ~] + ?- kind + %approved ``noun+!>((~(has in approved.cors-registry) u.origin)) + %rejected ``noun+!>((~(has in rejected.cors-registry) u.origin)) + == + :: + [%authenticated %cookie @ ~] + ?~ cookies=(slaw %t i.t.t.tyl) [~ ~] + :^ ~ ~ %noun + !> ^- ? + %- =< request-is-authenticated:authentication + (per-server-event [eny *duct now rof] server-state.ax) + %*(. *request:http header-list ['cookie' u.cookies]~) + :: + [%'_~_' *] + =/ mym (scry-mime now rof lyc (deft:de-purl:html tyl)) + ?: ?=(%| -.mym) [~ ~] + ``noun+!>(p.mym) + == + ?. ?=(%$ ren) ~ + ?+ syd ~ + %bindings ``noun+!>(bindings.server-state.ax) + %cache ``noun+!>(cache.server-state.ax) + %connections ``noun+!>(connections.server-state.ax) + %authentication-state ``noun+!>(auth.server-state.ax) + %channel-state ``noun+!>(channel-state.server-state.ax) + %domains ``noun+!>(domains.server-state.ax) + %ports ``noun+!>(ports.server-state.ax) + :: + %host + %- (lift (lift |=(a=hart:eyre [%hart !>(a)]))) + ^- (unit (unit hart:eyre)) + =. p.lot ?.(=([%da now] p.lot) p.lot [%tas %real]) + ?+ p.lot + [~ ~] + :: + [%tas %fake] + ``[& [~ 8.443] %& /localhost] + :: + [%tas %real] + =* domains domains.server-state.ax + =* ports ports.server-state.ax + =/ =host:eyre [%& ?^(domains n.domains /localhost)] + =/ port=(unit @ud) + ?. ?=(^ secure.ports) + ?:(=(80 insecure.ports) ~ `insecure.ports) + ?:(=(443 u.secure.ports) ~ secure.ports) + ``[?=(^ secure.ports) port host] + == + == +-- diff --git a/arvo/lull.hoon b/arvo/lull.hoon new file mode 100644 index 0000000..110ef8d --- /dev/null +++ b/arvo/lull.hoon @@ -0,0 +1,4622 @@ +:: /sys/lull +:: %lull: arvo structures +!: +=> ..part +~% %lull ..part ~ +|% +++ lull %322 +:: :: :: +:::: :: :: (1) models + :: :: :: +:: # %misc +:: +:: miscellaneous systems types +::+| +:: +capped-queue: a +qeu with a maximum number of entries +:: +++ capped-queue + |$ [item-type] + $: queue=(qeu item-type) + size=@ud + max-size=_64 + == +:: +clock: polymorphic cache type for use with the clock replacement algorithm +:: +:: The +by-clock core wraps interface arms for manipulating a mapping from +:: :key-type to :val-type. Detailed docs for this type can be found there. +:: +++ clock + |$ :: key-type: mold of keys + :: val-type: mold of values + :: + [key-type val-type] + $: lookup=(map key-type [val=val-type fresh=@ud]) + queue=(qeu key-type) + size=@ud + max-size=_2.048 + depth=_1 + == +:: +:: $plot: composable serialization +:: +++ plot + => |% + +$ plat + $@ @ :: measure atom + $^ $% [[%c ~] (pair (pair step step) @)] :: cut slice + [[%m ~] (pair (pair step step) @)] :: measure slice + [[%s ~] p=plot] :: subslice + == :: + (pair step @) :: prefix + -- :: + =< $ + ~% %plot ..plot ~ + |% + ++ $ + $^ [l=$ r=$] :: concatenate + [a=bloq b=(list plat)] :: serialize + :: + ++ fax :: encode + ~/ %fax + |= p=$ + ^- (trel @ bloq step) + ?^ -.p + =/ l $(p l.p) + =/ r $(p r.p) + =/ s (rig +.l q.r) + [(add p.l (lsh [q.r s] p.r)) q.r (add r.r s)] + :: + ?~ b.p [0 a.p 0] + =; c=(pair @ step) + =/ d $(b.p t.b.p) + [(add p.c (lsh [a.p q.c] p.d)) a.p (add q.c r.d)] + :: + ?@ i.b.p + [i.b.p (^met a.p i.b.p)] + ?- -.i.b.p + @ [(end [a.p p.i.b.p] q.i.b.p) p.i.b.p] + [%c ~] [(cut a.p [p q]:i.b.p) q.p.i.b.p] + [%m ~] =+((cut a.p [p q]:i.b.p) [- (^met a.p -)]) + [%s ~] =/ e $(p p.i.b.p) + [p.e (rig +.e a.p)] + == + :: + ++ met :: measure + ~/ %met + |=(p=$ `(pair bloq step)`+:(fax p)) + -- +:: +:: +mop: constructs and validates ordered ordered map based on key, +:: val, and comparator gate +:: +++ mop + |* [key=mold value=mold] + |= ord=$-([key key] ?) + |= a=* + =/ b ;;((tree [key=key val=value]) a) + ?> (apt:((on key value) ord) b) + b +:: +:: +++ ordered-map on +:: +on: treap with user-specified horizontal order, ordered-map +:: +:: WARNING: ordered-map will not work properly if two keys can be +:: unequal under noun equality but equal via the compare gate +:: +++ on + ~% %on ..part ~ + |* [key=mold val=mold] + => |% + +$ item [key=key val=val] + -- + :: +compare: item comparator for horizontal order + :: + ~% %comp +>+ ~ + |= compare=$-([key key] ?) + ~% %core + ~ + |% + :: +all: apply logical AND boolean test on all values + :: + ++ all + ~/ %all + |= [a=(tree item) b=$-(item ?)] + ^- ? + |- + ?~ a + & + ?&((b n.a) $(a l.a) $(a r.a)) + :: +any: apply logical OR boolean test on all values + :: + ++ any + ~/ %any + |= [a=(tree item) b=$-(item ?)] + |- ^- ? + ?~ a + | + ?|((b n.a) $(a l.a) $(a r.a)) + :: +apt: verify horizontal and vertical orderings + :: + ++ apt + ~/ %apt + |= a=(tree item) + =| [l=(unit key) r=(unit key)] + |- ^- ? + :: empty tree is valid + :: + ?~ a %.y + :: nonempty trees must maintain several criteria + :: + ?& :: if .n.a is left of .u.l, assert horizontal comparator + :: + ?~(l %.y (compare key.n.a u.l)) + :: if .n.a is right of .u.r, assert horizontal comparator + :: + ?~(r %.y (compare u.r key.n.a)) + :: if .a is not leftmost element, assert vertical order between + :: .l.a and .n.a and recurse to the left with .n.a as right + :: neighbor + :: + ?~(l.a %.y &((mor key.n.a key.n.l.a) $(a l.a, l `key.n.a))) + :: if .a is not rightmost element, assert vertical order + :: between .r.a and .n.a and recurse to the right with .n.a as + :: left neighbor + :: + ?~(r.a %.y &((mor key.n.a key.n.r.a) $(a r.a, r `key.n.a))) + == + :: +bap: convert to list, right to left + :: + ++ bap + ~/ %bap + |= a=(tree item) + ^- (list item) + =| b=(list item) + |- ^+ b + ?~ a b + $(a r.a, b [n.a $(a l.a)]) + :: +del: delete .key from .a if it exists, producing value iff deleted + :: + ++ del + ~/ %del + |= [a=(tree item) =key] + ^- [(unit val) (tree item)] + ?~ a [~ ~] + :: we found .key at the root; delete and rebalance + :: + ?: =(key key.n.a) + [`val.n.a (nip a)] + :: recurse left or right to find .key + :: + ?: (compare key key.n.a) + =+ [found lef]=$(a l.a) + [found a(l lef)] + =+ [found rig]=$(a r.a) + [found a(r rig)] + :: +dip: stateful partial inorder traversal + :: + :: Mutates .state on each run of .f. Starts at .start key, or if + :: .start is ~, starts at the head. Stops when .f produces .stop=%.y. + :: Traverses from left to right keys. + :: Each run of .f can replace an item's value or delete the item. + :: + ++ dip + ~/ %dip + |* state=mold + |= $: a=(tree item) + =state + f=$-([state item] [(unit val) ? state]) + == + ^+ [state a] + :: acc: accumulator + :: + :: .stop: set to %.y by .f when done traversing + :: .state: threaded through each run of .f and produced by +abet + :: + =/ acc [stop=`?`%.n state=state] + =< abet =< main + |% + ++ this . + ++ abet [state.acc a] + :: +main: main recursive loop; performs a partial inorder traversal + :: + ++ main + ^+ this + :: stop if empty or we've been told to stop + :: + ?: =(~ a) this + ?: stop.acc this + :: inorder traversal: left -> node -> right, until .f sets .stop + :: + =. this left + ?: stop.acc this + =^ del this node + =? this !stop.acc right + =? a del (nip a) + this + :: +node: run .f on .n.a, updating .a, .state, and .stop + :: + ++ node + ^+ [del=*? this] + :: run .f on node, updating .stop.acc and .state.acc + :: + ?> ?=(^ a) + =^ res acc (f state.acc n.a) + ?~ res + [del=& this] + [del=| this(val.n.a u.res)] + :: +left: recurse on left subtree, copying mutant back into .l.a + :: + ++ left + ^+ this + ?~ a this + =/ lef main(a l.a) + lef(a a(l a.lef)) + :: +right: recurse on right subtree, copying mutant back into .r.a + :: + ++ right + ^+ this + ?~ a this + =/ rig main(a r.a) + rig(a a(r a.rig)) + -- + :: +gas: put a list of items + :: + ++ gas + ~/ %gas + |= [a=(tree item) b=(list item)] + ^- (tree item) + ?~ b a + $(b t.b, a (put a i.b)) + :: +get: get val at key or return ~ + :: + ++ get + ~/ %get + |= [a=(tree item) b=key] + ^- (unit val) + ?~ a ~ + ?: =(b key.n.a) + `val.n.a + ?: (compare b key.n.a) + $(a l.a) + $(a r.a) + :: +got: need value at key + :: + ++ got + |= [a=(tree item) b=key] + ^- val + (need (get a b)) + :: +has: check for key existence + :: + ++ has + ~/ %has + |= [a=(tree item) b=key] + ^- ? + !=(~ (get a b)) + :: +lot: take a subset range excluding start and/or end and all elements + :: outside the range + :: + ++ lot + ~/ %lot + |= $: tre=(tree item) + start=(unit key) + end=(unit key) + == + ^- (tree item) + |^ + ?: ?&(?=(~ start) ?=(~ end)) + tre + ?~ start + (del-span tre %end end) + ?~ end + (del-span tre %start start) + ?> (compare u.start u.end) + =. tre (del-span tre %start start) + (del-span tre %end end) + :: + ++ del-span + |= [a=(tree item) b=?(%start %end) c=(unit key)] + ^- (tree item) + ?~ a a + ?~ c a + ?- b + %start + :: found key + ?: =(key.n.a u.c) + (nip a(l ~)) + :: traverse to find key + ?: (compare key.n.a u.c) + :: found key to the left of start + $(a (nip a(l ~))) + :: found key to the right of start + a(l $(a l.a)) + :: + %end + :: found key + ?: =(u.c key.n.a) + (nip a(r ~)) + :: traverse to find key + ?: (compare key.n.a u.c) + :: found key to the left of end + a(r $(a r.a)) + :: found key to the right of end + $(a (nip a(r ~))) + == + -- + :: +nip: remove root; for internal use + :: + ++ nip + ~/ %nip + |= a=(tree item) + ^- (tree item) + ?> ?=(^ a) + :: delete .n.a; merge and balance .l.a and .r.a + :: + |- ^- (tree item) + ?~ l.a r.a + ?~ r.a l.a + ?: (mor key.n.l.a key.n.r.a) + l.a(r $(l.a r.l.a)) + r.a(l $(r.a l.r.a)) + :: + :: +pop: produce .head (leftmost item) and .rest or crash if empty + :: + ++ pop + ~/ %pop + |= a=(tree item) + ^- [head=item rest=(tree item)] + ?~ a !! + ?~ l.a [n.a r.a] + =/ l $(a l.a) + :- head.l + :: load .rest.l back into .a and rebalance + :: + ?: |(?=(~ rest.l) (mor key.n.a key.n.rest.l)) + a(l rest.l) + rest.l(r a(r r.rest.l)) + :: +pry: produce head (leftmost item) or null + :: + ++ pry + ~/ %pry + |= a=(tree item) + ^- (unit item) + ?~ a ~ + |- + ?~ l.a `n.a + $(a l.a) + :: +put: ordered item insert + :: + ++ put + ~/ %put + |= [a=(tree item) =key =val] + ^- (tree item) + :: base case: replace null with single-item tree + :: + ?~ a [n=[key val] l=~ r=~] + :: base case: overwrite existing .key with new .val + :: + ?: =(key.n.a key) a(val.n val) + :: if item goes on left, recurse left then rebalance vertical order + :: + ?: (compare key key.n.a) + =/ l $(a l.a) + ?> ?=(^ l) + ?: (mor key.n.a key.n.l) + a(l l) + l(r a(l r.l)) + :: item goes on right; recurse right then rebalance vertical order + :: + =/ r $(a r.a) + ?> ?=(^ r) + ?: (mor key.n.a key.n.r) + a(r r) + r(l a(r l.r)) + :: +ram: produce tail (rightmost item) or null + :: + ++ ram + ~/ %ram + |= a=(tree item) + ^- (unit item) + ?~ a ~ + |- + ?~ r.a `n.a + $(a r.a) + :: +run: apply gate to transform all values in place + :: + ++ run + ~/ %run + |* [a=(tree item) b=$-(val *)] + |- + ?~ a a + [n=[key.n.a (b val.n.a)] l=$(a l.a) r=$(a r.a)] + :: +tab: tabulate a subset excluding start element with a max count + :: + ++ tab + ~/ %tab + |= [a=(tree item) b=(unit key) c=@] + ^- (list item) + |^ + (flop e:(tabulate (del-span a b) b c)) + :: + ++ tabulate + |= [a=(tree item) b=(unit key) c=@] + ^- [d=@ e=(list item)] + ?: ?&(?=(~ b) =(c 0)) + [0 ~] + =| f=[d=@ e=(list item)] + |- ^+ f + ?: ?|(?=(~ a) =(d.f c)) f + =. f $(a l.a) + ?: =(d.f c) f + =. f [+(d.f) [n.a e.f]] + ?:(=(d.f c) f $(a r.a)) + :: + ++ del-span + |= [a=(tree item) b=(unit key)] + ^- (tree item) + ?~ a a + ?~ b a + ?: =(key.n.a u.b) + r.a + ?: (compare key.n.a u.b) + $(a r.a) + a(l $(a l.a)) + -- + :: +tap: convert to list, left to right + :: + ++ tap + ~/ %tap + |= a=(tree item) + ^- (list item) + =| b=(list item) + |- ^+ b + ?~ a b + $(a l.a, b [n.a $(a r.a)]) + :: +uni: unify two ordered maps + :: + :: .b takes precedence over .a if keys overlap. + :: + ++ uni + ~/ %uni + |= [a=(tree item) b=(tree item)] + ^- (tree item) + ?~ b a + ?~ a b + ?: =(key.n.a key.n.b) + [n=n.b l=$(a l.a, b l.b) r=$(a r.a, b r.b)] + ?: (mor key.n.a key.n.b) + ?: (compare key.n.b key.n.a) + $(l.a $(a l.a, r.b ~), b r.b) + $(r.a $(a r.a, l.b ~), b l.b) + ?: (compare key.n.a key.n.b) + $(l.b $(b l.b, r.a ~), a r.a) + $(r.b $(b r.b, l.a ~), a l.a) + :: +wyt: measure size + :: + ++ wyt + ~/ %wyt + |= a=(tree item) + ^- @ud + ?~(a 0 +((add $(a l.a) $(a r.a)))) + -- +:: ++$ deco ?(~ %bl %br %un) :: text decoration ++$ json :: normal json value + $@ ~ :: null + $% [%a p=(list json)] :: array + [%b p=?] :: boolean + [%o p=(map @t json)] :: object + [%n p=@ta] :: number + [%s p=@t] :: string + == :: ++$ life @ud :: ship key revision ++$ rift @ud :: ship continuity ++$ mime (pair mite octs) :: mimetyped data ++$ octs (pair @ud @) :: octet-stream ++$ sock (pair ship ship) :: outgoing [src dest] ++$ sack (trel ship ship path) :: $sock /w provenance ++$ stub (list (pair stye (list @c))) :: styled unicode ++$ stye (pair (set deco) (pair tint tint)) :: decos/bg/fg ++$ styl %+ pair (unit deco) :: cascading style + (pair (unit tint) (unit tint)) :: ++$ styx (list $@(@t (pair styl styx))) :: styled text ++$ tint $@ ?(%r %g %b %c %m %y %k %w %~) :: text color + [r=@uxD g=@uxD b=@uxD] :: 24bit true color ++$ turf (list @t) :: domain, tld first +:: :::: +:::: ++ethereum-types :: eth surs for jael + :: :::: +++ ethereum-types + |% + :: ethereum address, 20 bytes. + :: + ++ address @ux + :: event location + :: + +$ event-id [block=@ud log=@ud] + :: + ++ events (set event-id) + -- +:: :::: +:::: ++azimuth-types :: az surs for jael + :: :::: +++ azimuth-types + =, ethereum-types + |% + ++ point + $: :: ownership + :: + $= own + $: owner=address + management-proxy=address + voting-proxy=address + transfer-proxy=address + == + :: + :: networking + :: + $= net + %- unit + $: =life + =pass + continuity-number=@ud + sponsor=[has=? who=@p] + escape=(unit @p) + == + :: + :: spawning + :: + $= kid + %- unit + $: spawn-proxy=address + spawned=(set @p) ::TODO sparse range, pile, see old jael ++py + == + == + :: + +$ dnses [pri=@t sec=@t ter=@t] + :: + ++ diff-azimuth + $% [%point who=@p dif=diff-point] + [%dns dnses] + == + :: + ++ diff-point + $% [%full new=point] :: + [%owner new=address] :: OwnerChanged + [%activated who=@p] :: Activated + [%spawned who=@p] :: Spawned + [%keys =life =pass] :: ChangedKeys + [%continuity new=@ud] :: BrokeContinuity + [%sponsor new=[has=? who=@p]] :: EscapeAcc/LostSpons + [%escape new=(unit @p)] :: EscapeReq/Can + [%management-proxy new=address] :: ChangedManagementPro + [%voting-proxy new=address] :: ChangedVotingProxy + [%spawn-proxy new=address] :: ChangedSpawnProxy + [%transfer-proxy new=address] :: ChangedTransferProxy + == + -- +:: +vane-task: general tasks shared across vanes +:: ++$ vane-task + $~ [%born ~] + $% :: i/o device replaced (reset state) + :: + [%born ~] + :: boot completed (XX legacy) + :: + [%init ~] + :: trim state (in response to memory pressure) + :: + [%trim p=@ud] + :: kernel upgraded + :: + [%vega ~] + :: receive message via %ames + :: + :: TODO: move .vane from $plea to here + :: + [%plea =ship =plea:ames] + == +:: :::: +:::: ++http :: + :: :::: +:: http: shared representations of http concepts +:: +++ http ^? + |% + :: +header-list: an ordered list of http headers + :: + +$ header-list + (list [key=@t value=@t]) + :: +method: exhaustive list of http verbs + :: + +$ method + $? %'CONNECT' + %'DELETE' + %'GET' + %'HEAD' + %'OPTIONS' + %'PATCH' + %'POST' + %'PUT' + %'TRACE' + == + :: +request: a single http request + :: + +$ request + $: :: method: http method + :: + method=method + :: url: the url requested + :: + :: The url is not escaped. There is no escape. + :: + url=@t + :: header-list: headers to pass with this request + :: + =header-list + :: body: optionally, data to send with this request + :: + body=(unit octs) + == + :: +response-header: the status code and header list on an http request + :: + :: We separate these away from the body data because we may not wait for + :: the entire body before we send a %progress to the caller. + :: + +$ response-header + $: :: status: http status code + :: + status-code=@ud + :: headers: http headers + :: + headers=header-list + == + :: +http-event: packetized http + :: + :: Urbit treats Earth's HTTP servers as pipes, where Urbit sends or + :: receives one or more %http-events. The first of these will always be a + :: %start or an %error, and the last will always be %cancel or will have + :: :complete set to %.y to finish the connection. + :: + :: Calculation of control headers such as 'Content-Length' or + :: 'Transfer-Encoding' should be performed at a higher level; this structure + :: is merely for what gets sent to or received from Earth. + :: + +$ http-event + $% :: %start: the first packet in a response + :: + $: %start + :: response-header: first event information + :: + =response-header + :: data: data to pass to the pipe + :: + data=(unit octs) + :: whether this completes the request + :: + complete=? + == + :: %continue: every subsequent packet + :: + $: %continue + :: data: data to pass to the pipe + :: + data=(unit octs) + :: complete: whether this completes the request + :: + complete=? + == + :: %cancel: represents unsuccessful termination + :: + [%cancel ~] + == + :: +get-header: returns the value for :header, if it exists in :header-list + :: + ++ get-header + |= [header=@t =header-list] + ^- (unit @t) + :: + ?~ header-list + ~ + :: + ?: =(key.i.header-list header) + `value.i.header-list + :: + $(header-list t.header-list) + :: +set-header: sets the value of an item in the header list + :: + :: This adds to the end if it doesn't exist. + :: + ++ set-header + |= [header=@t value=@t =header-list] + ^- ^header-list + :: + ?~ header-list + :: we didn't encounter the value, add it to the end + :: + [[header value] ~] + :: + ?: =(key.i.header-list header) + [[header value] t.header-list] + :: + [i.header-list $(header-list t.header-list)] + :: +delete-header: removes the first instance of a header from the list + :: + ++ delete-header + |= [header=@t =header-list] + ^- ^header-list + :: + ?~ header-list + ~ + :: if we see it in the list, remove it + :: + ?: =(key.i.header-list header) + t.header-list + :: + [i.header-list $(header-list t.header-list)] + :: +unpack-header: parse header field values + :: + ++ unpack-header + |^ |= value=@t + ^- (unit (list (map @t @t))) + (rust (cass (trip value)) values) + :: + ++ values + %+ more + (ifix [. .]:(star ;~(pose ace (just '\09'))) com) + pairs + :: + ++ pairs + %+ cook + ~(gas by *(map @t @t)) + %+ most (ifix [. .]:(star ace) mic) + ;~(plug token ;~(pose ;~(pfix tis value) (easy ''))) + :: + ++ value + ;~(pose token quoted-string) + :: + ++ token :: 7230 token + %+ cook crip + ::NOTE this is ptok:de-purl:html, but can't access that here + %- plus + ;~ pose + aln zap hax buc cen pam soq tar lus + hep dot ket cab tic bar sig + == + :: + ++ quoted-string :: 7230 quoted string + %+ cook crip + %+ ifix [. .]:;~(less (jest '\\"') doq) + %- star + ;~ pose + ;~(pfix bas ;~(pose (just '\09') ace prn)) + ;~(pose (just '\09') ;~(less (mask "\22\5c\7f") (shim 0x20 0xff))) + == + -- + :: +simple-payload: a simple, one event response used for generators + :: + +$ simple-payload + $: :: response-header: status code, etc + :: + =response-header + :: data: the data returned as the body + :: + data=(unit octs) + == + -- +:: :::: +:::: ++ames :: (1a) network + :: :::: +++ ames ^? + |% + :: $task: job for ames + :: + :: Messaging Tasks + :: + :: %hear: packet from unix + :: %dear: lane from unix + :: %cork: request to delete message flow + :: %tame: request to delete route for ship + :: %kroc: request to delete specific message flows, from their bones + :: %plea: request to send message + :: %deep: deferred calls to %ames, from itself + :: %stun: STUN response (or failure), from unix + :: + :: Remote Scry Tasks + :: + :: %keen: peek: [ship /vane/care/case/spur] + :: %yawn: cancel request from arvo + :: %wham: cancels all scry request from any vane + :: + :: System and Lifecycle Tasks + :: + :: %born: process restart notification + :: %init: vane boot + :: %prod: re-send a packet per flow, to all peers if .ships is ~ + :: %sift: limit verbosity to .ships + :: %snub: set packet blocklist to .ships + :: %spew: set verbosity toggles + :: %cong: adjust congestion control parameters + :: %stir: recover from timer desync and assorted debug commands + :: %trim: release memory + :: %vega: kernel reload notification + :: + +$ task + $% [%hear =lane =blob] + [%dear =ship =lane] + [%cork =ship] + [%tame =ship] + [%kroc bones=(list [ship bone])] + $>(%plea vane-task) + [%deep =deep] + [%stun =stun] + :: + [%keen sec=(unit [idx=@ key=@]) spar] + [%chum spar] + [%yawn spar] + [%wham spar] + [%plug =path] + :: + $>(%born vane-task) + $>(%init vane-task) + [%prod ships=(list ship)] + [%sift ships=(list ship)] + [%snub form=?(%allow %deny) ships=(list ship)] + [%spew veb=(list verb)] + [%cong msg=@ud mem=@ud] + [%stir arg=@t] + $>(%trim vane-task) + $>(%vega vane-task) + :: all tasks above, if changed, would need an adapter function in the + :: larval-core +load arm, to change the events to their latest version, as + :: they exist here in %lull. + :: + :: where (i.e. from what version of the ames-state) to do the migration + :: depends on when the task was introduced (e.g. %heed and %jilt were + :: introduced in state %4, and removed in %21; %kroc was introduced in + :: state %10, modified in %17...) + :: + :: when changing any of the tasks above, please follow the same patterns + :: that exist in ames.hoon. + :: + [%mate (unit ship) dry=?] :: per-peer migration + [%rege (unit ship) dry=?] :: per-peer regression + [%load ?(%mesa %ames)] :: load core for new peers; XX [... term] + :: + [%heer =lane:pact p=@] :: receive a packet + [%mess =mess] :: receive a message + [%moke =space =spar =path] :: initiate %poke request + [%meek =space =spar] :: initiate %peek request + [%mage =space =spar] :: send %page of data; intended for acks + [%rate =spar rate] :: get rate progress for +peeks, from unix + $: %prog :: subscribe to progress %rate + =spar :: if ?=(^ task), use it to modify path.spar + $= task + $@(~ ?([%chum ~] [%keen kid=(unit @)])) + feq=@ud + == + [%whey =spar boq=@ud] :: weight of noun bounded at .path.spar + :: as measured by .boq + [%gulp path] :: like %plug, but for |mesa + == + :: + :: $gift: effect from ames + :: + :: Messaging Gifts + :: + :: %boon: response message from remote ship + :: %noon: boon with duct for clog tracking + :: %clog: notify vane that %boon's to peer are backing up locally + :: %done: notify vane that peer (n)acked our message + :: %lost: notify vane that we crashed on %boon + :: %send: packet to unix + :: %nail: lanes to unix + :: + :: Remote Scry Gifts + :: + :: %tune: peek result + :: + :: System and Lifecycle Gifts + :: + :: %turf: domain report, relayed from jael + :: %saxo: our sponsor list report + :: + +$ gift + $~ lost/~ + $% [%boon payload=*] + [%noon id=* payload=*] + [%done error=(unit error)] + [%lost ~] + [%send =lane =blob] + [%nail =ship lanes=(list lane)] + :: + [%stub num=@ud key=@] + [%near spar dat=(unit (unit page))] + [%tune spar roar=(unit roar)] + :: + [%turf turfs=(list turf)] + [%saxo sponsors=(list ship)] + :: + [%push p=(list lane:pact) q=@] :: send a request/response packet + [%sage =sage:mess] :: give deserialized/open payload + $>(%page mess) :: give serialized/sealed payload + $>(%rate task) + == + :: + :::: :: (1a2) + :: + ++ acru $_ ^? :: asym cryptosuite + |% :: opaque object + ++ as ^? :: asym ops + |% ++ seal |~([a=pass b=@] *@) :: encrypt to a + ++ sign |~(a=@ *@) :: certify as us + ++ sigh |~(a=@ *@) :: certification only + ++ sure |~(a=@ *(unit @)) :: authenticate from us + ++ safe |~([a=@ b=@] *?) :: authentication only + ++ tear |~([a=pass b=@] *(unit @)) :: accept from a + -- ::as :: + ++ de |~([a=@ b=@] *(unit @)) :: symmetric de, soft + ++ dy |~([a=@ b=@] *@) :: symmetric de, hard + ++ en |~([a=@ b=@] *@) :: symmetric en + ++ ex ^? :: export + |% ++ fig *@uvH :: fingerprint + ++ pac *@uvG :: default passcode + ++ pub *pass :: public key + ++ sec *ring :: private key + -- ::ex :: + ++ nu ^? :: reconstructors + |% ++ pit |~([a=@ b=@] ^?(..nu)) :: from [width seed] + ++ nol |~(a=ring ^?(..nu)) :: from ring + ++ com |~(a=pass ^?(..nu)) :: from pass + -- ::nu :: + -- ::acru :: + :: +protocol-version: current version of the ames wire protocol + :: + ++ protocol-version `?(%0 %1 %2 %3 %4 %5 %6 %7)`%0 + :: $address: opaque atomic transport address to or from unix + :: + +$ address @uxaddress + :: $verb: verbosity flag for ames + :: + +$ verb ?(%snd %rcv %odd %msg %ges %for %rot %kay %fin %sun) + :: $bug: debug printing configuration + :: + :: veb: verbosity toggles + :: ships: identity filter; if ~, print for all + :: + :: veb: verbosity flags + :: + +$ veb-all-off + $: snd=_`?`%.n :: sending packets + rcv=_`?`%.n :: receiving packets + odd=_`?`%.n :: unusual events + msg=_`?`%.n :: message-level events (including flows) + ges=_`?`%.n :: congestion control + for=_`?`%.n :: packet forwarding + rot=_`?`%.n :: routing attempts + kay=_`?`%.n :: is ok/not responding + fin=_`?`%.n :: remote-scry + sun=_`?`%.n :: system level logs (STUN, keys, lanes...) + == + :: + +$ bug + $: veb=veb-all-off + ships=(set ship) + == + :: $blob: raw atom to or from unix, representing a packet + :: + +$ blob @uxblob + :: $error: tagged diagnostic trace + :: + +$ error [tag=@tas =tang] + :: $lane: ship transport address; either opaque $address or galaxy + :: + :: The runtime knows how to look up galaxies, so we don't need to + :: know their transport addresses. + :: + +$ lane (each @pC address) + :: $plea: application-level message, as a %pass + :: + :: vane: destination vane on remote ship + :: path: internal route on the receiving ship + :: payload: semantic message contents + :: + +$ plea [vane=@tas =path payload=*] + :: + +$ message + $% [%plea plea] + [%boon payload=*] + [%naxplanation =message-num =error] + == + :: $spar: pair of $ship and $path + :: + :: Instead of fully qualifying a scry path, ames infers rift and + :: life based on the ship. + :: + +$ spar [=ship =path] + :: $deep: deferred %ames call, from self, to keep +abet cores pure + :: + +$ deep + $% [%nack =ship =nack=bone =message] + [%sink =ship =target=bone naxplanation=[=message-num =error]] + [%drop =ship =nack=bone =message-num] + [%cork =ship =bone] + [%kill =ship =bone] + [%ahoy =ship =bone] :: XX remove bone; it's just next-bone.ossuary + [%prun =ship =user=path =duct =ames=path] + == + :: $stun: STUN notifications, from unix + :: + :: .lane is the latest cached lane in vere, from the point of view of .ship + :: + +$ stun + $% [%stop =ship =lane] :: succesful STUN response, stop %ping app + [%fail =ship =lane] :: failure to STUN, re-enable %ping app + [%once =ship =lane] :: new lane discovered, notify ping %app + == + :: +| %atomics + :: + +$ bone @udbone + +$ fragment @uwfragment + +$ fragment-num @udfragmentnum + +$ message-blob @udmessageblob + +$ message-num @udmessagenum + +$ public-key @uwpublickey + +$ private-key @uwprivatekey + +$ symmetric-key @uwsymmetrickey + :: + :: $hoot: request packet payload + :: $yowl: serialized response packet payload + :: $hunk: a slice of $yowl fragments + :: $lock: keys for remote scry + :: + +$ hoot @uxhoot + +$ yowl @uxyowl + +$ hunk [lop=@ len=@] + +$ lock [idx=@ key=@] + :: + :: +| %kinetics + :: $dyad: pair of sender and receiver ships + :: + +$ dyad [sndr=ship rcvr=ship] + :: $shot: noun representation of an ames datagram packet + :: + :: Roundtrips losslessly through atom encoding and decoding. + :: + :: .origin is ~ unless the packet is being forwarded. If present, + :: it's an atom that encodes a route to another ship, such as an IPv4 + :: address. Routes are opaque to Arvo and only have meaning in the + :: interpreter. This enforces that Ames is transport-agnostic. + :: + :: req: is a request + :: sam: is using the ames protocol (not fine or another protocol) + :: + +$ shot + $: dyad + req=? + sam=? + sndr-tick=@ubC + rcvr-tick=@ubC + origin=(unit @uxaddress) + content=@uxcontent + == + :: $ack: positive ack, nack packet, or nack trace + :: + +$ ack + $% [%ok ~] + [%nack ~] + [%naxplanation =error] + == + :: + :: +| %statics + :: $ship-state: all we know about a peer + :: + :: %alien: no PKI data, so enqueue actions to perform once we learn it + :: %known: we know their life and public keys, so we have a channel + :: + +$ ship-state + $+ ship-state + $% [%alien alien-agenda] + [%known peer-state] + == + :: $alien-agenda: what to do when we learn a peer's life and keys + :: + :: messages: pleas local vanes have asked us to send + :: packets: packets we've tried to send + :: + +$ alien-agenda + $+ alien-agenda + $: messages=(list [=duct =plea]) + packets=(set =blob) + keens=(jug [path ints] duct) + chums=(jug [path ints] duct) + == + +$ chain ((mop ,@ ,[key=@ =path]) lte) + :: $peer-state: state for a peer with known life and keys + :: + :: route: transport-layer destination for packets to peer + :: qos: quality of service; connection status to peer + :: ossuary: bone<->duct mapper + :: snd: per-bone message pumps to send messages as fragments + :: rcv: per-bone message sinks to assemble messages from fragments + :: nax: unprocessed nacks (negative acknowledgments) + :: Each value is ~ when we've received the ack packet but not a + :: nack-trace, or an error when we've received a nack-trace but + :: not the ack packet. + :: + :: When we hear a nack packet or an explanation, if there's no + :: entry in .nax, we make a new entry. Otherwise, if this new + :: information completes the packet+nack-trace, we remove the + :: entry and emit a nack to the local vane that asked us to send + :: the message. + :: closing: bones closed on the sender side + :: corked: bones closed on both sender and receiver + :: + +$ peer-state + $+ peer-state + $: $: =symmetric-key + =life + =rift + =public-key + sponsor=ship + == + route=(unit [direct=? =lane]) :: XX (list) + =qos + =ossuary + snd=(map bone message-pump-state) + rcv=(map bone message-sink-state) + nax=(set [=bone =message-num]) + closing=(set bone) + corked=(set bone) + keens=(map path keen-state) + =chain + tip=(jug =user=path [duct =ames=path]) + == + +$ keen-state + $+ keen-state + $: wan=((mop @ud want) lte) :: request packets, sent + nex=(list want) :: request packets, unsent + hav=(list have) :: response packets, backward + num-fragments=@ud + num-received=@ud + next-wake=(unit @da) + listeners=(jug duct ints) + metrics=pump-metrics + == + +$ want + $: fra=@ud + =hoot + packet-state + == + +$ have + $: fra=@ud + meow + == + :: + +$ meow :: response fragment + $: sig=@ux :: signature + num=@ud :: number of fragments + dat=@ux :: contents + == + :: + +$ peep :: fragment request + $: =path + num=@ud + == + :: + +$ wail :: tagged request fragment + $% [%0 peep] :: unsigned + == + :: + +$ roar :: response message + (tale:pki:jael (pair path (unit (cask)))) + :: + +$ purr :: response packet payload + $: peep + meow + == + :: + :: $qos: quality of service; how is our connection to a peer doing? + :: + :: .last-contact: last time we heard from peer, or if %unborn, when + :: we first started tracking time + :: + +$ qos + $~ [%unborn *@da] + [?(%live %dead %unborn) last-contact=@da] + :: $ossuary: bone<->duct bijection and .next-bone to map to a duct + :: + :: The first bone is 0. They increment by 4, since each flow includes + :: a bit for each message determining forward vs. backward and a + :: second bit for whether the message is on the normal flow or the + :: associated diagnostic flow (for naxplanations). + :: + :: The least significant bit of a $bone is: + :: 1 if "forward", i.e. we send %plea's on this flow, or + :: 0 if "backward", i.e. we receive %plea's on this flow. + :: + :: The second-least significant bit is 1 if the bone is a + :: naxplanation bone, and 0 otherwise. Only naxplanation + :: messages can be sent on a naxplanation bone, as %boon's. + :: + +$ ossuary + $: =next=bone + by-duct=(map duct bone) + by-bone=(map bone duct) + == + :: $message-pump-state: persistent state for |message-pump + :: + :: Messages queue up in |message-pump's .unsent-messages until they + :: can be packetized and fed into |packet-pump for sending. When we + :: pop a message off .unsent-messages, we push as many fragments as + :: we can into |packet-pump, which sends every packet it eats. + :: Packets rejected by |packet-pump are placed in .unsent-fragments. + :: + :: When we hear a packet ack, we send it to |packet-pump to be + :: removed from its queue of unacked packets. + :: + :: When we hear a message ack (positive or negative), we treat that + :: as though all fragments have been acked. If this message is not + :: .current, then this ack is for a future message and .current has + :: not yet been acked, so we place the ack in .queued-message-acks. + :: + :: If we hear a message ack before we've sent all the fragments for + :: that message, clear .unsent-fragments and have |packet-pump delete + :: all sent fragments from the message. If this early message ack was + :: positive, print it out because it indicates the peer is not + :: behaving properly. + :: + :: If the ack is for the current message, have |packet-pump delete + :: all packets from the message, give the message ack back + :: to the client vane, increment .current, and check if this next + :: message is in .queued-message-acks. If it is, emit the message + :: (n)ack, increment .current, and check the next message. Repeat + :: until .current is not fully acked. + :: + :: The following equation is always true: + :: .next - .current == number of messages in flight + :: + :: At the end of a task, |message-pump sends a %halt task to + :: |packet-pump, which can trigger a timer to be set or cleared based + :: on congestion control calculations. When the timer fires, it will + :: generally cause a packet to be re-sent. + :: + :: Message sequence numbers start at 1 so that the first message will + :: be greater than .last-acked.message-sink-state on the receiver. + :: + :: current: sequence number of earliest message sent or being sent + :: next: sequence number of next message to send + :: unsent-messages: messages to be sent after current message + :: unsent-fragments: fragments of current message waiting for sending + :: queued-message-acks: future message acks to be applied after current + :: packet-pump-state: state of corresponding |packet-pump + :: + +$ message-pump-state + $+ message-pump-state + $: current=_`message-num`1 + next=_`message-num`1 + unsent-messages=(qeu message) + unsent-fragments=(list static-fragment) + queued-message-acks=(map message-num ack) + =packet-pump-state + == + +$ static-fragment + $: =message-num + num-fragments=fragment-num + =fragment-num + =fragment + == + :: $packet-pump-state: persistent state for |packet-pump + :: + :: next-wake: last timer we've set, or null + :: live: packets in flight; sent but not yet acked + :: metrics: congestion control information + :: + +$ packet-pump-state + $+ packet-pump-state + $: next-wake=(unit @da) + live=((mop live-packet-key live-packet-val) lte-packets) + metrics=pump-metrics + == + :: +lte-packets: yes if a is before b + :: + ++ lte-packets + |= [a=live-packet-key b=live-packet-key] + ^- ? + :: + ?: (lth message-num.a message-num.b) + %.y + ?: (gth message-num.a message-num.b) + %.n + (lte fragment-num.a fragment-num.b) + :: $pump-metrics: congestion control state for a |packet-pump + :: + :: This is an Ames adaptation of TCP's Reno congestion control + :: algorithm. The information signals and their responses are + :: identical to those of the "NewReno" variant of Reno; the + :: implementation differs because Ames acknowledgments differ from + :: TCP's, because this code uses functional data structures, and + :: because TCP's sequence numbers reset when a peer becomes + :: unresponsive, whereas Ames sequence numbers only change when a + :: ship breaches. + :: + :: A deviation from Reno is +fast-resend-after-ack, which re-sends + :: timed-out packets when a peer starts responding again after a + :: period of unresponsiveness. + :: + :: If .skips reaches 3, we perform a fast retransmit and fast + :: recovery. This corresponds to Reno's handling of "three duplicate + :: acks". + :: + :: rto: retransmission timeout + :: rtt: roundtrip time estimate, low-passed using EWMA + :: rttvar: mean deviation of .rtt, also low-passed with EWMA + :: ssthresh: slow-start threshold + :: cwnd: congestion window; max unacked packets + :: + +$ pump-metrics + $: rto=_~s1 + rtt=_~s1 + rttvar=_~s1 + ssthresh=_10.000 + cwnd=_1 + counter=@ud + == + +$ live-packet + $: key=live-packet-key + val=live-packet-val + == + +$ live-packet-key + $: =message-num + =fragment-num + == + +$ live-packet-val + $: packet-state + num-fragments=fragment-num + =fragment + == + +$ packet-state + $: last-sent=@da + tries=_1 + skips=@ud + == + :: $message-sink-state: state of |message-sink to assemble messages + :: + :: last-acked: highest $message-num we've fully acknowledged + :: last-heard: highest $message-num we've heard all fragments on + :: pending-vane-ack: heard but not processed by local vane + :: live-messages: partially received messages + :: + +$ message-sink-state + $+ message-sink-state + $: last-acked=message-num + last-heard=message-num + pending-vane-ack=(qeu [=message-num message=*]) + live-messages=(map message-num partial-rcv-message) + nax=(set message-num) + == + :: $partial-rcv-message: message for which we've received some fragments + :: + :: num-fragments: total number of fragments in this message + :: num-received: how many fragments we've received so far + :: fragments: fragments we've received, eventually producing a $message + :: + +$ partial-rcv-message + $: num-fragments=fragment-num + num-received=fragment-num + fragments=(map fragment-num fragment) + == + :: $rank: which kind of ship address, by length + :: + :: 0b0: galaxy or star -- 2 bytes + :: 0b1: planet -- 4 bytes + :: 0b10: moon -- 8 bytes + :: 0b11: comet -- 16 bytes + :: + +$ rank ?(%0b0 %0b1 %0b10 %0b11) + :: + :: +| %coding + :: +sift-ship-size: decode a 2-bit ship type specifier into a byte width + :: + :: Type 0: galaxy or star -- 2 bytes + :: Type 1: planet -- 4 bytes + :: Type 2: moon -- 8 bytes + :: Type 3: comet -- 16 bytes + :: + ++ sift-ship-size + |= rank=@ubC + ^- @ + :: + ?+ rank !! + %0b0 2 + %0b1 4 + %0b10 8 + %0b11 16 + == + :: +is-valid-rank: does .ship match its stated .size? + :: + ++ is-valid-rank + |= [=ship size=@ubC] + ^- ? + .= size + =/ wid (met 3 ship) + ?: (lte wid 1) 2 + ?: =(2 wid) 2 + ?: (lte wid 4) 4 + ?: (lte wid 8) 8 + ?> (lte wid 16) 16 + :: +sift-shot: deserialize packet from bytestream or crash + :: + ++ sift-shot + |= =blob + ^- shot + ~| %sift-shot-fail + :: first 32 (2^5) bits are header; the rest is body + :: + =/ header (end 5 blob) + =/ body (rsh 5 blob) + :: read header; first two bits are reserved + :: + =/ req =(& (cut 0 [2 1] header)) + =/ sam =(& (cut 0 [3 1] header)) + :: + =/ version (cut 0 [4 3] header) + ?. =(protocol-version version) + ~& [%ames-protocol-version protocol-version version] + ~| ames-protocol-version+version !! + :: + =/ sndr-size (sift-ship-size (cut 0 [7 2] header)) + =/ rcvr-size (sift-ship-size (cut 0 [9 2] header)) + =/ checksum (cut 0 [11 20] header) + =/ relayed (cut 0 [31 1] header) + :: origin, if present, is 6 octets long, at the end of the body + :: + =^ origin=(unit @) body + ?: =(| relayed) + [~ body] + =/ len (sub (met 3 body) 6) + [`(end [3 6] body) (rsh [3 6] body)] + :: .checksum does not apply to the origin + :: + ?. =(checksum (end [0 20] (mug body))) + ~& >>> %ames-checksum + ~| %ames-checksum !! + :: read fixed-length sndr and rcvr life data from body + :: + :: These represent the last four bits of the sender and receiver + :: life fields, to be used for quick dropping of honest packets to + :: or from the wrong life. + :: + =/ sndr-tick (cut 0 [0 4] body) + =/ rcvr-tick (cut 0 [4 4] body) + :: read variable-length .sndr and .rcvr addresses + :: + =/ off 1 + =^ sndr off [(cut 3 [off sndr-size] body) (add off sndr-size)] + ?. (is-valid-rank sndr sndr-size) + ~& >>> [%ames-sender-imposter sndr sndr-size] + ~| ames-sender-impostor+[sndr sndr-size] !! + :: + =^ rcvr off [(cut 3 [off rcvr-size] body) (add off rcvr-size)] + ?. (is-valid-rank rcvr rcvr-size) + ~& >>> [%ames-receiver-imposter rcvr rcvr-size] + ~| ames-receiver-impostor+[rcvr rcvr-size] !! + :: read variable-length .content from the rest of .body + :: + =/ content (cut 3 [off (sub (met 3 body) off)] body) + [[sndr rcvr] req sam sndr-tick rcvr-tick origin content] + :: + ++ sift-wail + |= =hoot + ^- wail + ?> =(0 (end 3 hoot)) + [%0 +:(sift-peep (rsh 3 hoot))] + :: + ++ sift-purr + |= =hoot + ^- purr + =+ [wid peep]=(sift-peep hoot) + [peep (sift-meow (rsh [3 wid] hoot))] + :: + ++ sift-peep + |= =hoot + ^- [wid=@ =peep] + =+ num=(cut 3 [0 4] hoot) + =+ len=(cut 3 [4 2] hoot) + =+ pat=(cut 3 [6 len] hoot) + ~| pat=pat + :- (add 6 len) + :_ num + (rash pat ;~(pfix fas (most fas (cook crip (star ;~(less fas prn)))))) + :: + ++ sift-meow + |= =yowl + :* sig=(cut 3 [0 64] yowl) + num=(cut 3 [64 4] yowl) + dat=(rsh 3^68 yowl) + == + :: +etch-shot: serialize a packet into a bytestream + :: + ++ etch-shot + |= shot + ^- blob + :: + =/ sndr-meta (ship-meta sndr) + =/ rcvr-meta (ship-meta rcvr) + :: + =/ body=@ + ;: mix + sndr-tick + (lsh 2 rcvr-tick) + (lsh 3 sndr) + (lsh [3 +(size.sndr-meta)] rcvr) + (lsh [3 +((add size.sndr-meta size.rcvr-meta))] content) + == + =/ checksum (end [0 20] (mug body)) + =? body ?=(^ origin) (mix u.origin (lsh [3 6] body)) + :: + =/ header=@ + %+ can 0 + :~ [2 reserved=0] + [1 req] + [1 sam] + [3 protocol-version] + [2 rank.sndr-meta] + [2 rank.rcvr-meta] + [20 checksum] + [1 relayed=.?(origin)] + == + (mix header (lsh 5 body)) + :: + :: +ship-meta: produce size (in bytes) and address rank for .ship + :: + :: 0: galaxy or star + :: 1: planet + :: 2: moon + :: 3: comet + :: + ++ ship-meta + |= =ship + ^- [size=@ =rank] + :: + =/ size=@ (met 3 ship) + :: + ?: (lte size 2) [2 %0b0] + ?: (lte size 4) [4 %0b1] + ?: (lte size 8) [8 %0b10] + [16 %0b11] + :: + +$ axle + $: peers=(map ship ship-state) + =unix=duct :: [//ames/0v0 ~] + =life + =rift + =bug + snub=[form=?(%allow %deny) ships=(set ship)] + cong=[msg=_5 mem=_100.000] + $= dead :: dead-flow consolidation timers + $: flow=[%flow (unit dead-timer)] :: ... for |ames + chum=[%chum (unit dead-timer)] :: ... for |mesa + cork=[%cork (unit dead-timer)] :: ... for %nacked corks + rots=[%rots (unit dead-timer)] :: ... fir expiring direct routes + == + :: + =server=chain :: for serving %shut requests + priv=private-key + chums=(map ship chum-state) :: XX migrated peers + core=_`?(%ames %mesa)`%ames :: XX use migrated core by default + :: TODOs + :: XX tmp=(map @ux page) :: temporary hash-addressed bindings + == + :: + +$ dead-timer [=duct =wire date=@da] + :: + +$ space + $~ none/~ + $% [%none ~] + [%publ =life] + [%shut kid=@ key=@uxI] + [%chum =server=life client=ship =client=life key=@] + == + :: [0 %for] => %poke: %plea %watch => [0 %bak] + :: [0 %for] <= %poke: %boon <= [0 %bak] + :: + :: +load: payloads bounded in the namespace + :: + +$ load ?(%poke %ack %naxp %cork) + :: +dire: side of the flow (%bak: %boon sender; %for: %plea sender) + :: + +$ dire ?(%bak %for) + +$ side [=bone =dire] + +$ azimuth-state [=symmetric-key =life =rift =public-key sponsor=ship] + +$ chum-state + $+ chum-state + $% [%known fren-state] + [%alien ovni-state] + == + :: + +$ ovni-state + $+ ovni-state + $: pokes=(list [=duct message=mesa-message]) + peeks=(jug [path ints] duct) + chums=(jug [path ints] duct) + == + :: + +$ fren-state + $: azimuth-state + lane=(unit [hop=@ =lane:pact]) :: XX (list) + =qos + corked=(set side) :: can be +peeked in the namespace + :: XX how many flows to keep here? + =ossuary :: XX redefine ossuary in terms of bone^side + flows=(map side flow-state) + pit=(map path request-state) :: active +peek namespace paths + =client=chain :: stores keys for %shut requests + tip=(jug =user=path [duct =ames=path]) :: reverse .pit lookup map + :: a migrated flow in a weird state is tagged with a $term, and data + :: + weir=(jug side [tag=term data=*]) + == + :: + :: interest gifts per path in the pith + :: %sage used by |mesa + :: %tune used by |fine + :: %rate XX give $rate every .feq of bloq size .boq + :: + +$ ints ?(%sage %tune [%rate boq=@ud feq=@ud]) + +$ rate $@(~ [boq=@ud fag=@ud tot=@ud]) + +$ request-state + $: for=(jug duct ints) + pay=(unit path) + ps=(unit pact-state) + == + :: + ++ pact-state + :: XX duplicated in %zuse + :: + => |% + ++ lss + |% + ++ verifier + |% + +$ state + $: leaves=@ud + counter=@ud + pairs=(map @ud [l=@ux r=@ux]) + == + -- + -- + -- + $: los=state:verifier:lss + fags=(list @) + == + :: + +$ mesa-message + $~ [%plea *plea] + $>(?(%plea %boon) message) + :: + +$ flow-state + $: :: a flow switches to closing when: + :: - forward: a %cork $plea %poke request is sent + :: - backward: a %cork $plea %poke request is received + :: + :: the flow is deleted first on the forward side when it can read the + :: %ack for the %cork, and then on the backward side when it can +peek + :: the corked flow from the forward namespace + :: + closing=?(%.y %.n) + :: line: high-water mark for the last-acked message before migration + :: + line=@ud + :: outbound %poke payloads, bounded in the ship's namespace + :: always and only for requests + :: + $= snd + $: %outbound + :: as soon as we can read the ack for the %poke we remove it from + :: the queue since that proof that they have processed the message + :: + :: (n)acks are considered payload responses, and are part of + :: received pokes, so we track them in the nax map + :: + :: both for boons and pleas, and per (seq)message + :: the ordered map guarantees that we receive the acks in ordered + :: if (dec received-ack=@ud) has not been acked, we drop it + :: + :: payloads can be +peek'ed via a well-formed path with the format: + :: e.g. /flow/[bone=0]/[load]/?[%for %bak]/[ship]/[seq=1] + :: + :: XX option to include messages that won't be bounded into the + :: namespace (two-layer queue) + :: + loads=((mop ,@ud mesa-message) lte) :: all unacked + next=_1 :: =(next +(last-acked)) + :: + send-window-max=_1 :: how many pleas to send + send-window=_1 :: XX + acks=((mop ,@ud ack) lte) :: out-of-order acks + == + :: incoming %pokes, pending their ack from the vane + :: + $= rcv + $: %incoming + :: acks can be +peek'ed via a well-formed path with a known structure + :: (as stored in the producer of the ack) + :: e.g. /flow/bone=0/ack-{plea-boon}/~zod/seq=1 + :: + last-acked=@ud :: for acking old duplicates (only 10) + :: and dropping future acks + :: only +(last-acked) messages are handled + :: + pending-ack=_`?`%.n :: there's only one pending ack to guarantee + :: that messages are delivered in order + :: and to only send the ack to the vane once + nax=(map seq=@ud error) :: messages you have nacked, + :: (last-acked - 10 <= ack <= last-acked) + == == + :: atom ops + :: + :: +nac: reverse +can + :: + ++ nac + |= [a=bloq b=(list (pair step step)) c=@] + ^- (list @) + ?~ b ~ + [(cut a [i.b] c) $(b t.b)] + :: + :: +dew: dry +hew + :: + ++ dew + |= [a=bite b=* c=@] + ^- * + =< - + =/ [d=bloq e=step] ?^(a a [a 0]) + |- ^- (pair * step) + ?@ b + [(cut d [e b] c) (add e b)] + =^ f e $(b -.b) + =^ g e $(b +.b) + [[f g] e] + :: + :: binary tree ops + :: + :: +| %arboric + :: + ++ bao + |= n=@ud + =| i=@ud + =| s=(list) + |- ^- * + ?: =(i n) + =^ d s s + |-(?~(s d $(d [i.s d], s t.s))) + :: + =/ d=* i + =. i +(i) + =/ j (ctz i) + |- ^- * + ?: =(0 j) + ^$(s [d s]) + =^ e s s + $(d [e d], j (dec j)) + :: + ++ unroll + |= d=* + =| s=(list [axe=@ d=*]) + =/ a 1 + |- ^+ s + ?@ d + ?~ s ~ + $(d d.i.s, a axe.i.s, s t.s) + :- [a d] + $(d -.d, a (peg a 2), s [[(peg a 3) +.d] s]) + :: + :: +| %messages + :: + ++ mess + => |% + +$ auth (each @uxJ @uxH) :: &+sig, |+hmac + +$ gage $@(~ page) + +$ sage (pair spar gage) + -- + $% [%page (trel spar auth @)] + [%peek spar] + [%poke (pair spar sage)] + == + :: + :: packet de/serialization + :: + :: +| %packets + :: + :: > :(add 8 305 1.159) + :: 1.472 + :: + ++ pact + => |% + +$ frag @udG + +$ ship @pH + +$ rift @udF + +$ bloq @D + +$ name + $: [her=ship rif=rift] + [boq=bloq wan=$@(~ [typ=?(%auth %data) fag=frag])] + pat=path + == + +$ auth + :: %& for auth packet + :: %| for data packets + :: + (each auth:mess (unit (pair @uxI @uxI))) + +$ data [tob=@ud aut=auth:pact dat=@] + +$ lane $@ @ux + $% [%if p=@ifF q=@udE] + [%is p=@isH q=@udE] + == + +$ next (list lane) + +$ pact $: hop=@ud + $% [%page =name =data =next] + [%poke ack=name pok=name =data] + [%peek =name] + == == + -- + :: + |% + ++ en + |= pak=pact + ^- plot + =* typ +<.pak + =/ bod=plot + ?- typ + %page [(en:^name name.pak) (en:^data data.pak) (en:^next next.pak)] + %peek (en:^name name.pak) + %poke [(en:^name ack.pak) (en:^name pok.pak) (en:^data data.pak)] + == + =/ hed=plot + =/ nex=@B + ?. ?=(%page typ) 0b0 + ?~ next.pak 0b0 + ?^ t.next.pak 0b11 + ?:(?=([%if *] i.next.pak) 0b1 0b10) + (en:head nex typ hop.pak (mug p:(fax:plot bod))) + [hed bod] + :: + ++ de + |= a=bite + |= dat=@ + ^- [pact boq=bloq sep=step] + =+ ^= [hed b] ((de:head a) dat) + =+ ^= [pac c] + ?- typ.hed + %page =^ nam b ((de:^name b) dat) + =^ dat b ((de:^data b) dat) + =^ nex b ((de:^next b nex.hed) ^dat) + [[hop.hed [typ.hed nam dat nex]] b] + :: + %peek =^ nam b ((de:^name b) dat) + [[hop.hed [typ.hed nam]] b] + :: + %poke =^ nam b ((de:^name b) dat) + =^ dam b ((de:^name b) dat) + =^ dat b ((de:^data b) dat) + [[hop.hed [typ.hed nam dam dat]] b] + == + =/ gum + (end [0 20] (mug (cut -.c [(rig b -.c) +.c] dat))) + ~| gum.hed^gum + ?> =(gum.hed gum) :: XX jumbo fragments have wrong mug; fixed? + [pac c] + -- + :: + ++ head + |% + ++ en + |= [nex=@B typ=?(%page %peek %poke) hop=@ gum=@F] + ^- plot + =/ tip ?-(typ %page 0b1, %peek 0b10, %poke 0b11) + =. hop (min 7 hop) + =* tok [32 0x67e0.0200] + :- bloq=0 + [[2 0] [2 nex] [3 ver=1] [2 tip] [3 hop] [20 gum] tok ~] + :: + ++ de + |= a=bite + =/ b=[bloq step] [0 (rig a 0)] + |= dat=@ + ^- [[nex=@B typ=?(%page %peek %poke) hop=@ gum=@F] bloq step] + =^ c b + ((hew b dat) [res=2 nex=2 ver=3 tip=2 hop=3 gum=20 tok=32]) + ?> =(0 res.c) + ?> =(1 ver.c) + ?> =(0x67e0.0200 tok.c) + =/ typ ?+(tip.c !! %0b1 %page, %0b10 %peek, %0b11 %poke) + [[nex.c typ hop.c gum.c] b] + -- + :: + ++ next + |% + ++ en + |= nex=next:pact + ^- plot + :- bloq=3 + ?~ nex ~ + ?: ?=([[%if *] ~] nex) + [[4 p] [2 q] ~]:i.nex + |- ^- (list plat:plot) + =; one=(list plat:plot) + ?~(t.nex one (weld one $(nex t.nex))) + ?- i.nex + @ =/ l (met 3 i.nex) + ?> (lth l 255) + [[1 +(l)] [1 2] [l i.nex] ~] + [%if *] [[1 7] [1 0] [4 p] [2 q] ~]:i.nex + [%is *] =/ l (met 3 p.i.nex) + ?> (lth l 253) + [[1 (add 3 l)] [1 1] [l p.i.nex] [2 q.i.nex] ~] + == + :: + ++ de + |= [a=bite b=@B] + =/ c=[bloq step] [3 (rig a 3)] + |= dat=@ + ^- [next:pact bloq step] + =< ?+ b !! + %0b0 [~ c] + :: + %0b1 =^ if=[@ @] c ((hew c dat) 4 2) + [[if+if ~] c] + :: + %0b10 =^ la c (need one) + [[la ~] c] + :: + %0b11 =| nex=next:pact + |- ^- [next:pact bloq step] + =/ a one + ?~ a [(flop nex) c] + $(nex [-.u.a nex], c +.u.a) + == + |% + ++ one + ^- (unit [lane:pact bloq step]) + =^ raw c ((hew c dat) 1 1) + ?: =(0 -.raw) ~ + ?+ +.raw !! + %0 ?> =(7 -.raw) + =^ if=[@ @] c ((hew c dat) 4 2) + `[if+if c] + :: + %1 ?> (gte -.raw 3) + =^ is=[@ @] c ((hew c dat) (sub -.raw 3) 2) + `[is+is c] + :: + %2 =^ la c ((hew c dat) (dec -.raw)) + `[la c] + == + -- + -- + :: + :: +name: encoded-path + :: + :: range: { meta[1], her[2^1-4], rif[1-4], boq[1], fag[1-4], len[2], pat[2^0-16 - 1] } + :: max: { 1, 16, 4, 1, 4, 2, 65.535 } + :: actual: { 1, 16, 4, 1, 4, 2, 277 } + :: + :: XX increment path-len by one, exclude zero-length paths + :: + :: > :(add 1 16 4 1 4 2 277) + :: 305 + :: + :: for %poke: + :: > (div (sub 305 (mul 2 (sub 305 277))) 2) + :: 124 + :: + ++ name + |% + ++ en + |= name:pact + ^- plot + =/ ran ?~(her 0 (xeb (dec (met 4 (end 7 her))))) + =/ ryf ?~(rif 0 (dec (met 3 rif))) :: XX is rift always non-zero? + =/ [nit=@ tau=@ gaf=@ gyf=@ fag=@] + ?~ wan + [0b1 0b0 0b0 0 0] + =/ gaf (xeb (dec (met 3 (max 1 fag.wan)))) + [0b0 ?:(?=(%auth typ.wan) 0b1 0b0) gaf (bex gaf) fag.wan] + :: + =/ tap =-([p=(met 3 -) q=-] `@t`(rap 3 (join '/' pat))) + ?> &(?=(^ pat) (lth p.tap ^~((bex 16)))) :: XX truncate instead? + :+ bloq=3 + [s+~ 0 [2 ran] [2 ryf] [1 nit] [1 tau] [2 gaf] ~] + [[(bex +(ran)) her] [+(ryf) rif] [1 boq] [gyf fag] [2 p.tap] tap ~] + :: + ++ de + |= a=bite + =/ b=[boq=bloq sep=step] [0 (rig a 0)] + |= pat=@ + ^- [name:pact _b] + =^ c b + ((hew b pat) [ran=2 ryf=2 nit=1 tau=1 gaf=2]) + :: + =. b [3 (rig b 3)] + =^ d b + %- (hew b pat) + :^ who=[her=(bex +(ran.c)) rif=+(ryf.c)] + boq=1 + fag=?:(=(0b1 nit.c) 0 (bex gaf.c)) + tap=2 + :: + :: XX ?< =(0 tap.d) + =/ pat + %+ rash (cut boq.b [sep.b tap.d] pat) + (more fas (cook crip (star ;~(less fas prn)))) + =/ wan + ?. =(0b1 nit.c) + [?:(=(1 tau.c) %auth %data) fag.d] + ?>(&(=(0 tau.c) =(0 fag.d)) ~) + :: + =. sep.b (add sep.b tap.d) + [[who.d [boq.d wan] pat] b] + -- + :: + :: +data: response data + :: + :: range: { meta[1], tot[1-4], aum[32*0-2], aup[32*0-2], len-len[0-1], len[0-255], dat[0-2^252-1] } + :: max: { 1, 4, 64, 64, 1, 255, 2^252-1 } + :: actual: { 1, 4, 64, 64, 0, 2, 1.024 } + :: + :: XX increment len-len by 3, recalculate max limits + :: XX max len-len s/b 32 to align with max bloq size + :: XX move tot after auth to avoid trailing zeros? + :: + :: > :(add 1 4 64 64 2 1.024) + :: 1.159 + :: + ++ data + |% + ++ en + |= [tob=@ud aut=auth:pact dat=@] + ^- plot + =/ lot (xeb (dec (met 3 (max 1 tob)))) + ?> (lte lot 3) + :: + =/ [aub=@ubB aum=plat:plot] + ?- aut + [%& %& *] ?> (lte (met 3 +.p.aut) 64) + [0b0 64 +.p.aut] + :: + [%| ~] [0b1 0] + :: + [%& %| *] ?> (lte (met 3 +.p.aut) 16) + [0b10 16 +.p.aut] + :: + [%| ^] ?> (lte (met 3 p.u.p.aut) 32) + ?> (lte (met 3 q.u.p.aut) 32) + [0b11 s+~ 3 [32 p] [32 q] ~]:u.p.aut + == + :: + =/ len (met 3 dat) + =/ nel (met 3 len) + =/ men=(pair @B @A) + ?:((lth nel 3) [nel 0] [0b11 1]) + :+ bloq=3 + [s+~ 0 [2 lot] [2 aub] [2 0] [2 p.men] ~] + [[(bex lot) tob] aum [q.men nel] [nel len] [len dat] ~] + :: + ++ de + |= a=bite + =/ b=[boq=bloq sep=step] [0 (rig a 0)] + |= dat=@ + ^- [data:pact boq=bloq sep=step] + =^ c b + ((hew b dat) [lot=2 [aub=2 nil=2] men=2]) + =. b [3 (rig b 3)] + =^ d b + %- (hew b dat) + :+ tob=(bex lot.c) + ^= aub + ?+ aub.c !! + %0b0 `@`64 + %0b10 `@`16 + %0b1 `@`0 + %0b11 [`@`32 `@`32] + == + nel=?.(=(3 men.c) 0 1) + :: + =/ aut=auth:pact + ?+ aub.c !! + %0b0 [%& %& ?>(?=(@ aub.d) aub.d)] + %0b10 [%& %| ?>(?=(@ aub.d) aub.d)] + %0b1 [%| ~] + %0b11 [%| ?>(?=(^ aub.d) `aub.d)] + == + :: + =/ nel ?.(=(3 men.c) men.c nel.d) + =^ len sep.b [(cut 3 [sep.b nel] dat) (add sep.b nel)] + =^ dat sep.b [(cut 3 [sep.b len] dat) (add sep.b len)] + [[tob.d aut dat] b] + -- + :: + ++ name-to-beam + |= name:pact + ^- beam + :* [her %$ ud+1] + %mess (scot %ud rif) + %pact (scot %ud boq) %etch + ?~ wan [%init pat] + [typ.wan (scot %ud fag.wan) pat] + == + :: + :: XX to lib? + :: + ++ roundtrip + |* [dat=* en=$-(* plot) de=$-(@ [* boq=@ sep=@])] + ^- (unit _dat) + =/ pol (en dat) + =/ ser (fax:plot pol) + =/ ron (de p.ser) + ?. =(dat -.ron) + ~& %roundtrip-fail-a + `-.ron + ?. =(q.ser boq.ron) + ~& [%roundtrip-fail-b q.ser boq.ron] + `-.ron + ?. =(r.ser sep.ron) + ~& [%roundtrip-fail-c r.ser sep.ron] + `-.ron + ~ + :: + ++ generator + |% + +$ gen $-(@uvJ [* @uvJ]) + ++ just |*(a=* |=(eny=@uvJ [a (shaz eny)])) + :: + ++ cook + |* [a=$-(* *) b=gen] + |= eny=@uvJ + =^ c eny (b eny) + [(a c) eny] + :: + ++ flag + |= eny=@uvJ + =+ [b og]=(~(raws og eny) 1) + [=(b 0) a.og] + :: + ++ rand + |= top=@u + |= eny=@uvJ + =/ og ~(. og eny) + |- ^- [@ @uvJ] + =^ a og (rads:og (met 0 top)) + =^ b og (raws:og a) + ?:((gte b top) $ [b a.og]) + :: + ++ bits + |= [zer=? top=@ud] + |= eny=@uvJ + ^- [@ @uvJ] + =^ a eny ((rand top) eny) + =+ [b og]=(~(raws og eny) a) + ?: &(!zer =(0 b)) $ + [b a.og] + :: + ++ char + |= [boq=bloq src=@] + =/ wyd (met boq src) + |= eny=@uvJ + ^- [@ @uvJ] + =^ a eny ((rand wyd) eny) + [(cut boq [a 1] src) eny] + :: + ++ many + |* [[min=@ud max=@ud] =gen] + |= eny=@uvJ + =^ a eny ((rand max) eny) + ?: (lth a min) $ + =| [i=@ud lit=(list _-:$:gen)] + |- ^+ [lit eny] + ?: =(i a) [lit eny] + =^ b eny (gen eny) + $(lit [b lit], i +(i)) + :: + ++ both + |* [l=gen r=gen] + |= eny=@uvJ + =^ a eny (l eny) + =^ b eny (r eny) + [[a b] eny] + :: + ++ pick + |* [l=gen r=gen] + |= eny=@uvJ + =^ a eny (flag eny) + (?:(a l r) eny) + :: + ++ aura + =| zer=? + |= yaz=@t + ~+ + ^- $-(@uvJ [@ @uvJ]) + =/ [max=@ud aur=@ta] + =/ len (met 3 yaz) + ?: =(0 len) + [0 %$] + =. len (dec len) + =/ tyl (rsh [3 len] yaz) + ?. &((gte tyl 'A') (lte tyl 'Z')) + [0 yaz] + [(sub tyl 'A') (end [3 len] yaz)] + :: + =- ?@ - (just -) + ?: ?=(%| -<) -> + (bits zer ?:(=(0 max) -> (bex max))) + :: + =+ yed=(rip 3 aur) + ?+ yed &+256 + [%c *] :- %| + %+ cook (cury rep 5) + (many [0 256] (rand 0x11.0000)) :: XX nonzero? + :: + [%d ?(%a %r) *] &+128 + [%f *] &+1 + [%n *] `@`0 + [%i %f *] &+32 + [%i %s *] &+128 + [?(%p %q) *] &+128 + [%r %h *] &+16 + [%r %s *] &+32 + [%r %d *] &+64 + [%r %q *] &+128 + :: + [%u %c *] |+(cook enc:fa (bits & 256)) + :: + [%t %a %s *] :- %| + %+ cook crip + %+ both + (char 3 ^~((crip (gulf 'a' 'z')))) + %+ many [0 32] + %+ char 3 + ^~((crip (weld (gulf '0' '9') ['-' (gulf 'a' 'z')]))) + :: + [%t %a *] :- %| + %+ cook crip + %+ many [0 64] + %+ char 3 + ^~((crip :(weld "-~_." (gulf '0' '9') (gulf 'a' 'z')))) + :: + [%t *] :- %| + %+ cook (corl tuft (cury rep 5)) + (many [0 256] (rand 0x11.0000)) :: XX nonzero? + == + :: + ++ name + => |% + ++ ship (aura 'pH') + ++ rift (aura 'udF') + ++ bloq (aura 'udD') + ++ frag rift + ++ want %+ pick (just ~) + (both (pick (just %auth) (just %data)) frag) + ++ path (many [1 10] (aura %ta)) + -- + ^- $-(@uvJ [name:pact @uvJ]) + %+ both (both ship rift) + (both (both bloq want) path) + :: + ++ data + => |% + ++ frag (=+(aura -(zer |)) 'udF') + ++ hash (aura 'uxI') + ++ mess-auth + (pick (both (just %&) (aura 'uxJ')) (both (just %|) (aura 'uxH'))) + ++ auth + ;: pick + :(both (just %|) (pick (just ~) :(both (just ~) hash hash))) + :(both (just %&) mess-auth) + == + -- + ^- $-(@uvJ [data:pact @uvJ]) + :(both frag auth (bits & ^~((bex 16)))) + :: + ++ lane + => |% + ++ port (aura 'udE') + -- + ;: pick + (bits & 256) + :(both (just %if) (aura 'ifF') port) + :(both (just %is) (aura 'isH') port) + == + :: + ++ pactt + ^- $-(@uvJ [pact:pact @uvJ]) + =/ hop (rand 7) + ;: pick + :(both hop (just %peek) name) + :(both hop (just %page) name data (many [0 2] lane)) + :(both hop (just %poke) name name data) + == + -- + :: + ++ test + |_ [eny=@uvJ len=@ud] + ++ name + =| i=@ud + |- ^- ? + ?: =(i len) & + =/ old eny + =^ nam eny (name:generator eny) + ?^ ron=(roundtrip nam en:^name $:de:^name) + ~&([i=i eny=old nam u.ron] |) + $(i +(i)) + :: + ++ data + =| i=@ud + |- ^- ? + ?: =(i len) & + =/ old eny + =^ dat eny (data:generator eny) + ?^ ron=(roundtrip dat en:^data $:de:^data) + ~&([i=i eny=old dat u.ron] |) + $(i +(i)) + :: + ++ all + =| i=@ud + |- ^- ? + ?: =(i len) & + =/ old eny + =^ pac eny (pactt:generator eny) + ?^ ron=(roundtrip pac en:pact $:de:pact) + ~&([i=i eny=old pac u.ron] |) + $(i +(i)) + -- + :: + -- ::ames +:: :::: +:::: ++behn :: (1b) timekeeping + :: :::: +++ behn ^? + |% + +$ gift :: out result <-$ + $% [%doze p=(unit @da)] :: next alarm + [%wake error=(unit tang)] :: wakeup or failed + [%meta p=vase] + [%heck syn=sign-arvo] :: response to %huck + == + +$ task :: in request ->$ + $~ [%vega ~] :: + $% $>(%born vane-task) :: new unix process + [%rest p=@da] :: cancel alarm + [%drip p=vase] :: give in next event + [%huck syn=sign-arvo] :: give back + $>(%trim vane-task) :: trim state + $>(%vega vane-task) :: report upgrade + [%wait p=@da] :: set alarm + [%wake ~] :: timer activate + == + -- ::behn +:: :::: +:::: ++clay :: (1c) versioning + :: :::: +++ clay ^? + |% + +$ gift :: out result <-$ + $% [%boon payload=*] :: ames response + [%croz rus=(map desk [r=regs w=regs])] :: rules for group + [%cruz cez=(map @ta crew)] :: permission groups + [%dirk p=@tas] :: mark mount dirty + [%ergo p=@tas q=mode] :: version update + [%hill p=(list @tas)] :: mount points + [%done error=(unit error:ames)] :: ames message (n)ack + [%mere p=(each (set path) (pair term tang))] :: merge result + [%ogre p=@tas] :: delete mount point + [%rule red=dict wit=dict] :: node r+w permissions + [%tire p=(each rock:tire wave:tire)] :: app state + [%writ p=riot] :: response + [%wris p=[%da p=@da] q=(set (pair care path))] :: many changes + == :: + +$ task :: in request ->$ + $~ [%vega ~] :: + $% [%boat ~] :: pier rebooted + [%cred nom=@ta cew=crew] :: set permission group + [%crew ~] :: permission groups + [%crow nom=@ta] :: group usage + [%drop des=desk] :: cancel pending merge + [%esse des=desk ese=?] :: edit essential desk + [%info des=desk dit=nori] :: internal edit + $>(%init vane-task) :: report install + [%into des=desk all=? fis=mode] :: external edit + $: %merg :: merge desks + des=desk :: target + her=@p dem=desk cas=case :: source + how=germ :: method + == :: + $: %fuse :: merge many + des=desk :: target desk + bas=beak :: base desk + con=(list [beak germ]) :: merges + == :: + [%mont pot=term bem=beam] :: mount to unix + [%dirk pot=term] :: mark mount dirty + [%ogre pot=$@(term beam)] :: delete mount point + [%park des=desk yok=yoki ran=rang] :: synchronous commit + [%perm des=desk pax=path rit=rite] :: change permissions + [%pork ~] :: resume commit + [%prep lat=(map lobe page)] :: prime clay store + [%rein des=desk ren=rein] :: extra apps + [%stir arg=*] :: debug + [%tire p=(unit ~)] :: app state subscribe + [%tomb =clue] :: tombstone specific + $>(%trim vane-task) :: trim state + $>(%vega vane-task) :: report upgrade + [%warp wer=ship rif=riff] :: internal file req + [%werp who=ship wer=ship rif=riff-any] :: external file req + [%wick ~] :: try upgrade + [%zeal lit=(list [=desk =zest])] :: batch zest + [%zest des=desk liv=zest] :: live + $>(%plea vane-task) :: ames request + == :: + :: :: + :::: :: (1c2) + :: :: + +$ aeon @ud :: version number + +$ beam [[p=ship q=desk r=case] s=path] :: global name + +$ beak [p=ship q=desk r=case] :: path prefix + +$ cable :: lib/sur/mark ref + $: face=(unit term) :: + file-path=term :: + == :: + +$ care :: clay submode + $? %a %b %c %d %e %f :: + %p %q %r %s %t %u :: + %v %w %x %y %z :: + == :: + +$ cash :: case or tako + $% [%tako p=tako] :: + case :: + == :: + +$ cass [ud=@ud da=@da] :: cases for revision + +$ clue :: murder weapon + $% [%lobe =lobe] :: specific lobe + [%all ~] :: all safe targets + [%pick ~] :: collect garbage + [%norm =ship =desk =norm] :: set default norm + [%worn =ship =desk =tako =norm] :: set commit norm + [%seek =ship =desk =cash] :: fetch source blobs + == :: + +$ cone (map [ship desk] dome) :: domes + :: + :: Desk state. + :: + :: Includes a checked-out ankh with current content, most recent version, map + :: of all version numbers to commit hashes (commits are in hut.rang), and map + :: of labels to version numbers. + :: + :: `mim` is a cache of the content in the directories that are mounted + :: to unix. Often, we convert to/from mime without anything really + :: having changed; this lets us short-circuit that in some cases. + :: Whenever you give an `%ergo`, you must update this. + :: + +$ dome + $: let=aeon :: top id + hit=(map aeon tako) :: versions by id + lab=(map @tas aeon) :: labels + tom=(map tako norm) :: tomb policies + nor=norm :: default policy + mim=(map path mime) :: mime cache + fod=flue :: ford cache + wic=(map weft yoki) :: commit-in-waiting + liv=zest :: running agents + ren=rein :: force agents on/off + == :: + +$ crew (set ship) :: permissions group + +$ dict [src=path rul=real] :: effective permission + +$ domo :: project state + $: let=@ud :: top id + hit=(map @ud tako) :: changes by id + lab=(map @tas @ud) :: labels + == :: + +$ germ :: merge style + $? %init :: new desk + %fine :: fast forward + %meet :: orthogonal files + %mate :: orthogonal changes + %meld :: force merge + %only-this :: ours with parents + %only-that :: hers with parents + %take-this :: ours unless absent + %take-that :: hers unless absent + %meet-this :: ours if conflict + %meet-that :: hers if conflict + == :: + +$ lobe @uvI :: blob ref + +$ miso :: file delta + $% [%del ~] :: delete + [%ins p=cage] :: insert + [%dif p=cage] :: mutate from diff + [%mut p=cage] :: mutate from raw + == :: + +$ misu :: computed delta + $% [%del ~] :: delete + [%ins p=cage] :: insert + [%dif p=lobe q=cage] :: mutate from diff + == :: + +$ mizu [p=@u q=(map @ud tako) r=rang] :: new state + +$ moar [p=@ud q=@ud] :: normal change range + +$ moat [from=case to=case =path] :: change range + +$ mode (list [path (unit mime)]) :: external files + +$ mood [=care =case =path] :: request in desk + +$ mool [=case paths=(set (pair care path))] :: requests in desk + +$ nori :: repository action + $% [%& p=soba] :: delta + [%| p=@tas q=(unit aeon)] :: label + == :: + +$ nuri :: repository action + $% [%& p=suba] :: delta + [%| p=@tas] :: label + == :: + +$ norm (axal ?) :: tombstone policy + +$ open $-(path vase) :: get prelude + +$ page ^page :: export for compat + +$ pour :: ford build w/content + $% [%file =path] + [%nave =mark] + [%dais =mark] + [%cast =mars] + [%tube =mars] + :: leafs + :: + [%vale =path =lobe] + [%arch =path =(map path lobe)] + == + +$ rang :: repository + $+ rang + $: hut=(map tako yaki) :: changes + lat=(map lobe page) :: data + == :: + +$ rant :: response to request + $: p=[p=care q=case r=desk] :: clade release book + q=path :: spur + r=cage :: data + == :: + +$ rave :: general request + $% [%sing =mood] :: single request + [%next =mood] :: await next version + [%mult =mool] :: next version of any + [%many track=? =moat] :: track range + == :: + +$ real :: resolved permissions + $: mod=?(%black %white) :: + who=(pair (set ship) (map @ta crew)) :: + == :: + +$ regs (map path rule) :: rules for paths + +$ rein (map dude:gall ?) :: extra apps + +$ riff [p=desk q=(unit rave)] :: request+desist + +$ riff-any :: + $% [%1 =riff] :: + == :: + +$ rite :: new permissions + $% [%r red=(unit rule)] :: for read + [%w wit=(unit rule)] :: for write + [%rw red=(unit rule) wit=(unit rule)] :: for read and write + == :: + +$ riot (unit rant) :: response+complete + +$ rule [mod=?(%black %white) who=(set whom)] :: node permission + +$ rump [p=care q=case r=@tas s=path] :: relative path + +$ saba [p=ship q=@tas r=moar s=dome] :: patch+merge + +$ soak :: ford result + $% [%cage =cage] + [%vase =vase] + [%arch dir=(map @ta vase)] + [%dais =dais] + [%tube =tube] + == + +$ soba (list [p=path q=miso]) :: delta + +$ suba (list [p=path q=misu]) :: delta + +$ tako @uvI :: yaki ref + +$ toro [p=@ta q=nori] :: general change + ++ unce :: change part + |* a=mold :: + $% [%& p=@ud] :: skip[copy] + [%| p=(list a) q=(list a)] :: p -> q[chunk] + == :: + ++ urge |*(a=mold (list (unce a))) :: list change + +$ waft :: kelvin range + $^ [[%1 ~] p=(set weft)] :: + weft :: + +$ whom (each ship @ta) :: ship or named crew + +$ yoki (each yuki yaki) :: commit + +$ yuki :: proto-commit + $: p=(list tako) :: parents + q=(map path (each page lobe)) :: namespace + == :: + +$ yaki :: commit + $: p=(list tako) :: parents + q=(map path lobe) :: namespace + r=tako :: self-reference + t=@da :: date + == :: + +$ zest $~(%dead ?(%dead %live %held)) :: how live + :: :: + ++ tire :: app state + |% :: + +$ rock (map desk [=zest wic=(set weft)]) :: + +$ wave :: + $% [%wait =desk =weft] :: blocked + [%warp =desk =weft] :: unblocked + [%zest =desk =zest] :: running + == :: + :: + ++ wash :: patch + |= [=rock =wave] + ^+ rock + ?- -.wave + %wait + =/ got=[=zest wic=(set weft)] + (~(gut by rock) desk.wave *zest ~) + (~(put by rock) desk.wave got(wic (~(put in wic.got) weft.wave))) + :: + %warp + %- ~(run by rock) + |= [=zest wic=(set weft)] + [zest (~(del in wic) weft.wave)] + :: + %zest + ?: ?=(%dead zest.wave) + (~(del by rock) desk.wave) + =/ got=[=zest wic=(set weft)] + (~(gut by rock) desk.wave *zest ~) + (~(put by rock) desk.wave got(zest zest.wave)) + == + :: + ++ walk :: diff + |= [a=rock b=rock] + ^- (list wave) + =/ adds (~(dif by b) a) + =/ dels (~(dif by a) b) + =/ bots (~(int by a) b) + ;: welp + ^- (list wave) + %- zing + %+ turn ~(tap by adds) + |= [=desk =zest wic=(set weft)] + ^- (list wave) + :- [%zest desk zest] + %+ turn ~(tap in wic) + |= =weft + [%wait desk weft] + :: + ^- (list wave) + %+ turn ~(tap by dels) + |= [=desk =zest wic=(set weft)] + ^- wave + [%zest desk %dead] + :: + ^- (list wave) + %- zing + %+ turn ~(tap by bots) + |= [=desk * *] + ^- (list wave) + =/ aa (~(got by a) desk) + =/ bb (~(got by b) desk) + =/ wadds (~(dif in wic.bb) wic.aa) + =/ wdels (~(dif in wic.aa) wic.bb) + ;: welp + ?: =(zest.aa zest.bb) + ~ + [%zest desk zest.bb]~ + :: + %+ turn ~(tap by wadds) + |= =weft + ^- wave + [%wait desk weft] + :: + %+ turn ~(tap by wdels) + |= =weft + ^- wave + [%warp desk weft] + == + == + -- + :: + :: +page-to-lobe: hash a page to get a lobe. + :: + ++ page-to-lobe |=(page (shax (jam +<))) + :: + ++ cord-to-waft + |= =cord + ^- waft + =/ wefts=(list weft) + %+ turn (rash cord (star (ifix [gay gay] tall:vast))) + |= =hoon + !<(weft (slap !>(~) hoon)) + ?: ?=([* ~] wefts) + i.wefts + [[%1 ~] (sy wefts)] + :: + ++ waft-to-wefts + |= kal=waft + ^- (set weft) + ?^ -.kal + p.kal + [kal ~ ~] + :: + :: +make-yaki: make commit out of a list of parents, content, and date. + :: + ++ make-yaki + |= [p=(list tako) q=(map path lobe) t=@da] + ^- yaki + =+ ^= has + %^ cat 7 (sham [%yaki (roll p add) q t]) + (sham [%tako (roll p add) q t]) + [p q has t] + :: + :: $leak: ford cache key + :: + :: This includes all build inputs, including transitive dependencies, + :: recursively. + :: + +$ leak + $~ [*pour ~] + $: =pour + deps=(set leak) + == + :: + :: $flow: global ford cache + :: + :: Refcount includes references from other items in the cache, and + :: from spills in each desk + :: + :: This is optimized for minimizing the number of rebuilds, and given + :: that, minimizing the amount of memory used. It is relatively slow + :: to lookup, because generating a cache key can be fairly slow (for + :: files, it requires parsing; for tubes, it even requires building + :: the marks). + :: + +$ flow (map leak [refs=@ud =soak]) + :: + :: Per-desk ford cache + :: + :: Spill is the set of "roots" we have into the global ford cache. + :: We add a root for everything referenced directly or indirectly on + :: a desk, then invalidate them on commit only if their dependencies + :: change. + :: + :: Sprig is a fast-lookup index over the global ford cache. The only + :: goal is to make cache hits fast. + :: + +$ flue [spill=(set leak) sprig=(map mist [=leak =soak])] + :: + :: Ford build without content. + :: + +$ mist + $% [%file =path] + [%nave =mark] + [%dais =mark] + [%cast =mars] + [%tube =mars] + [%vale =path] + [%arch =path] + == + :: + :: $pile: preprocessed hoon source file + :: + :: /- sur-file :: surface imports from /sur + :: /+ lib-file :: library imports from /lib + :: /= face /path :: imports built hoon file at path + :: /~ face type /path :: imports built hoon files from directory + :: /% face %mark :: imports mark definition from /mar + :: /$ face %from %to :: imports mark converter from /mar + :: /* face %mark /path :: unbuilt file imports, as mark + :: + +$ pile + $: sur=(list taut) + lib=(list taut) + raw=(list [face=term =path]) + raz=(list [face=term =spec =path]) + maz=(list [face=term =mark]) + caz=(list [face=term =mars]) + bar=(list [face=term =mark =path]) + =hoon + == + :: $taut: file import from /lib or /sur + :: + +$ taut [face=(unit term) pax=term] + :: $mars: mark conversion request + :: $tube: mark conversion gate + :: $nave: typed mark core + :: + +$ mars [a=mark b=mark] + +$ tube $-(vase vase) + ++ nave + |$ [typ dif] + $_ + ^? + |% + ++ diff |~([old=typ new=typ] *dif) + ++ form *mark + ++ join |~([a=dif b=dif] *(unit (unit dif))) + ++ mash + |~ [a=[ship desk dif] b=[ship desk dif]] + *(unit dif) + ++ pact |~([typ dif] *typ) + ++ vale |~(noun *typ) + -- + :: $dais: processed mark core + :: + +$ dais + $_ ^| + |_ sam=vase + ++ diff |~(new=_sam *vase) + ++ form *mark + ++ join |~([a=vase b=vase] *(unit (unit vase))) + ++ mash + |~ [a=[ship desk diff=vase] b=[ship desk diff=vase]] + *(unit vase) + ++ pact |~(diff=vase sam) + ++ vale |~(noun sam) + -- + :: + ++ get-fit + |= [bek=beak pre=@tas pax=@tas] + ^- (unit path) + =/ paz (segments pax) + |- ^- (unit path) + ?~ paz + ~ + =/ puz=path (snoc `path`[pre i.paz] %hoon) + =+ .^(=arch cy+[(scot %p p.bek) q.bek (scot r.bek) puz]) + ?^ fil.arch + `puz + $(paz t.paz) + :: +segments: compute all paths from :path-part, replacing some `/`s with `-`s + :: + :: For example, when passed a :path-part of 'foo-bar-baz', + :: the product will contain: + :: ``` + :: dojo> (segments 'foo-bar-baz') + :: ~[/foo-bar-baz /foo-bar/baz /foo/bar-baz /foo/bar/baz] + :: ``` + :: + ++ segments + |= suffix=@tas + ^- (list path) + =/ parser + (most hep (cook crip ;~(plug ;~(pose low nud) (star ;~(pose low nud))))) + =/ torn=(list @tas) (fall (rush suffix parser) ~[suffix]) + %- flop + |- ^- (list (list @tas)) + ?< ?=(~ torn) + ?: ?=([@ ~] torn) + ~[torn] + %- zing + %+ turn $(torn t.torn) + |= s=(list @tas) + ^- (list (list @tas)) + ?> ?=(^ s) + ~[[i.torn s] [(crip "{(trip i.torn)}-{(trip i.s)}") t.s]] + -- ::clay +:: :::: +:::: ++dill :: (1d) console + :: :::: +++ dill ^? + |% + +$ gift :: out result <-$ + $% [%blit p=(list blit)] :: terminal output + [%logo ~] :: logout + [%meld ~] :: unify memory + [%pack ~] :: compact memory + [%trim p=@ud] :: trim kernel state + [%logs =told] :: system output + [%meme p=(list quac)] :: memory report + [%quac ~] :: memory runtime + == :: + +$ task :: in request ->$ + $~ [%vega ~] :: + $% $>(%born vane-task) :: new unix process + [%boot lit=? p=*] :: weird %dill boot + [%crop p=@ud] :: trim kernel state + [%flog p=flog] :: wrapped error + [%heft ~] :: memory report + $>(%init vane-task) :: after gall ready + [%logs p=(unit ~)] :: watch system output + [%mass ~] :: run memory report + [%quac p=(list quac)] :: memory runtime + [%meld ~] :: unify memory + [%pack ~] :: compact memory + [%seat =desk] :: install desk + [%shot ses=@tas task=session-task] :: task for session + $>(%trim vane-task) :: trim state + $>(%vega vane-task) :: report upgrade + [%verb ~] :: verbose mode + [%knob tag=term level=?(%hush %soft %loud)] :: deprecated removeme + session-task :: for default session + told :: system output + == :: + :: :: + +$ session-task :: session request + $% [%belt p=belt] :: terminal input + [%blew p=blew] :: terminal config + [%flee ~] :: unwatch session + [%hail ~] :: terminal refresh + [%open p=dude:gall q=(list gill:gall)] :: setup session + [%shut ~] :: close session + [%view ~] :: watch session blits + == :: + :: :: + +$ told :: system output + $% [%crud p=@tas q=tang] :: error + [%talk p=(list tank)] :: tanks (in order) + [%text p=tape] :: tape + == :: + :: + :::: :: (1d2) + :: + +$ blew [p=@ud q=@ud] :: columns rows + +$ belt :: client input + $? bolt :: simple input + [%mod mod=?(%ctl %met %hyp) key=bolt] :: w/ modifier + [%txt p=(list @c)] :: utf32 text + ::TODO consider moving %hey, %rez, %yow here :: + == :: + +$ bolt :: simple input + $@ @c :: simple keystroke + $% [%aro p=?(%d %l %r %u)] :: arrow key + [%bac ~] :: true backspace + [%del ~] :: true delete + [%hit x=@ud y=@ud] :: mouse click + [%ret ~] :: return + == :: + +$ blit :: client output + $% [%bel ~] :: make a noise + [%clr ~] :: clear the screen + [%hop p=$@(@ud [x=@ud y=@ud])] :: set cursor col/pos + [%klr p=stub] :: put styled + [%mor p=(list blit)] :: multiple blits + [%nel ~] :: newline + [%put p=(list @c)] :: put text at cursor + [%sag p=path q=*] :: save to jamfile + [%sav p=path q=@] :: save to file + [%url p=@t] :: activate url + [%wyp ~] :: wipe cursor line + == :: + +$ dill-belt :: arvo input + $% belt :: client input + [%cru p=@tas q=(list tank)] :: errmsg (deprecated) + [%hey ~] :: refresh + [%rez p=@ud q=@ud] :: resize, cols, rows + [%yow p=gill:gall] :: connect to app + == :: + +$ dill-blit :: arvo output + $% blit :: client output + [%qit ~] :: close console + == :: + +$ flog :: sent to %dill + $% [%crop p=@ud] :: trim kernel state + $>(%crud told) :: + [%heft ~] :: + [%meld ~] :: unify memory + [%pack ~] :: compact memory + $>(%text told) :: + [%verb ~] :: verbose mode + == :: + :: :: + +$ poke :: dill to userspace + $: ses=@tas :: target session + dill-belt :: input + == :: + +$ quac :: memory report + $~ ['' 0 ~] + [name=@t size=@ud quacs=(list quac)] + -- ::dill +:: :::: +:::: ++eyre :: (1e) http-server + :: :::: +++ eyre ^? + |% + +$ cache-entry + $: auth=? + $= body + $% [%payload =simple-payload:http] + == == + +$ gift + $% :: ames responses + :: + $>(?(%boon %done) gift:ames) + :: set-config: configures the external http server + :: + :: TODO: We need to actually return a (map (unit @t) http-config) + :: so we can apply configurations on a per-site basis + :: + [%set-config =http-config] + :: sessions: valid authentication cookie strings + :: + [%sessions ses=(set @t)] + :: response: response to an event from earth + :: + [%response =http-event:http] + :: response to a %connect or %serve + :: + :: :accepted is whether :binding was valid. Duplicate bindings are + :: not allowed. + :: + [%bound accepted=? =binding] + :: notification that a cache entry has changed + :: + [%grow =path] + :: UIP-125 + :: + [%websocket-response wid=@ event=websocket-event] + == + :: + +$ task + $~ [%vega ~] + $% :: initializes ourselves with an identity + :: + $>(%init vane-task) + :: new unix process + :: + $>(%born vane-task) + :: network request + :: + $>(%plea vane-task) + :: trim state (in response to memory pressure) + :: + $>(%trim vane-task) + :: report upgrade + :: + $>(%vega vane-task) + :: notifies us of the ports of our live http servers + :: + [%live insecure=@ud secure=(unit @ud)] + :: update http configuration + :: + [%rule =http-rule] + :: set a base url for eauth, like `'https://sampel.com' + :: + :: eyre will append /~/eauth to it internally to redirect into eauth + :: + [%eauth-host host=(unit @t)] + :: starts handling an inbound http request + :: + [%request secure=? =address =request:http] + :: starts handling an backdoor http request + :: + [%request-local secure=? =address =request:http] + :: cancels a previous request + :: + [%cancel-request ~] + :: connects a binding to an app + :: + [%connect =binding app=term] + :: connects a binding to a generator + :: + [%serve =binding =generator] + :: disconnects a binding + :: + :: This must be called with the same duct that made the binding in + :: the first place. + :: + [%disconnect =binding] + :: notifies us that web login code changed + :: + [%code-changed ~] + :: start responding positively to cors requests from origin + :: + [%approve-origin =origin] + :: start responding negatively to cors requests from origin + :: + [%reject-origin =origin] + :: %spew: set verbosity toggle + :: + [%spew veb=@] + :: remember (or update) a cache mapping + :: + [%set-response url=@t entry=(unit cache-entry)] + :: UIP-125 + :: + [%websocket-event ws-id=@ event=websocket-event] + [%websocket-handshake ws-id=@ secure=? =address =request:http] + == + :: UIP-125 + +$ websocket-connection + $: app=term + =inbound-request + == + +$ websocket-message + $: opcode=@ud + message=(unit data=octs) + == + +$ websocket-event + $% [%accept ~] + [%reject ~] + [%disconnect ~] + [%message message=websocket-message] + == + :: +origin: request origin as specified in an Origin header + :: + +$ origin @torigin + :: +cors-registry: origins categorized by approval status + :: + +$ cors-registry + $: requests=(set origin) + approved=(set origin) + rejected=(set origin) + == + :: +outstanding-connection: open http connections not fully complete: + :: + :: This refers to outstanding connections where the connection to + :: outside is opened and we are currently waiting on an app to + :: produce the results. + :: + +$ outstanding-connection + $: :: action: the action that had matched + :: + =action + :: inbound-request: the original request which caused this connection + :: + =inbound-request + :: session-id: the session associated with this connection + :: identity: the identity associated with this connection + :: + ::NOTE technically the identity is associated with the session (id), + :: but we may still need to know the identity that was used + :: after the session proper expires. + :: + [session-id=@uv =identity] + :: response-header: set when we get our first %start + :: + response-header=(unit response-header:http) + :: bytes-sent: the total bytes sent in response + :: + bytes-sent=@ud + == + :: +authentication-state: state used in the login system + :: + +$ authentication-state + $: :: sessions: a mapping of session cookies to session information + :: + sessions=(map @uv session) + :: visitors: in-progress incoming eauth flows + :: + visitors=(map @uv visitor) + :: visiting: outgoing eauth state per ship + :: + visiting=(map ship logbook) + :: endpoint: hardcoded local eauth endpoint for %syn and %ack + :: + :: user-configured or auth-o-detected, with last-updated timestamp. + :: both shaped like 'prot://host' + :: + endpoint=[user=(unit @t) auth=(unit @t) =time] + == + :: +session: server side data about a session + :: + +$ session + $: :: identity: authentication level & id of this session + :: + =identity + :: expiry-time: when this session expires + :: + :: We check this server side, too, so we aren't relying on the browser + :: to properly handle cookie expiration as a security mechanism. + :: + expiry-time=@da + :: channels: channels opened by this session + :: + channels=(set @t) + :: + :: TODO: We should add a system for individual capabilities; we should + :: mint some sort of long lived cookie for mobile apps which only has + :: access to a single application path. + == + :: +visitor: completed or in-progress incoming eauth flow + :: + :: duct: boon duct + :: and + :: sesh: login completed, session exists + :: or + :: pend: awaiting %tune for %keen sent at time, for initial eauth http req + :: ship: the @p attempting to log in + :: base: local protocol+hostname the attempt started on, if any + :: last: the url to redirect to after log-in + :: toke: authentication secret received over ames or offered by visitor + :: + +$ visitor + $: duct=(unit duct) + $@ sesh=@uv + $: pend=(unit [http=duct keen=time]) + ship=ship + base=(unit @t) + last=@t + toke=(unit @uv) + == == + :: +logbook: record of outgoing eauth comms & state + :: + :: qeu: a queue of nonces for to-be-n/acked pleas + :: map: per nonce, completed or pending eauth session + :: + +$ logbook [=(qeu @uv) =(map @uv portkey)] + :: +portkey: completed or in-progress outgoing eauth flow + :: + :: made: live since + :: or + :: duct: confirm request awaiting redirect + :: toke: secret to include in redirect, unless aborting + :: + +$ portkey + $@ made=@da :: live since + $: pend=(unit duct) :: or await redir + toke=(unit @uv) :: with secret + == + :: +eauth-plea: client talking to host + :: + +$ eauth-plea + $: %0 + $% :: %open: client decided on an attempt, wants to return to url + :: %shut: client wants the attempt or session closed + :: + [%open nonce=@uv token=(unit @uv)] + [%shut nonce=@uv] + == == + :: +eauth-boon: host responding to client + :: + +$ eauth-boon + $: %0 + $% :: %okay: attempt heard, client to finish auth through url + :: %shut: host has expired the session + :: + [%okay nonce=@uv url=@t] + [%shut nonce=@uv] + == == + :: $identity: authentication method & @p + :: + +$ identity + $~ [%ours ~] + $% [%ours ~] :: local, root + [%fake who=@p] :: guest id + [%real who=@p] :: authed cross-ship + == + :: channel-state: state used in the channel system + :: + +$ channel-state + $: :: session: mapping between an arbitrary key to a channel + :: + session=(map @t channel) + :: by-duct: mapping from ducts to session key + :: + duct-to-key=(map duct @t) + == + :: +timer: a reference to a timer so we can cancel or update it. + :: + +$ timer + $: :: date: time when the timer will fire + :: + date=@da + :: duct: duct that set the timer so we can cancel + :: + =duct + == + :: channel-event: unacknowledged channel event, vaseless sign + :: + +$ channel-event + $% $>(%poke-ack sign:agent:gall) + $>(%watch-ack sign:agent:gall) + $>(%kick sign:agent:gall) + [%fact =desk =mark =noun] + == + :: channel: connection to the browser + :: + :: Channels are the main method where a webpage communicates with Gall + :: apps. Subscriptions and pokes are issues with PUT requests on a path, + :: while GET requests on that same path open a persistent EventSource + :: channel. + :: + :: The EventSource API is a sequence number based API that browser provide + :: which allow the server to push individual events to the browser over a + :: connection held open. In case of reconnection, the browser will send a + :: 'Last-Event-Id: ' header to the server; the server then resends all + :: events since then. + :: + +$ channel + $: mode=?(%json %jam) + =identity + :: channel-state: expiration time or the duct currently listening + :: + :: For each channel, there is at most one open EventSource + :: connection. A 400 is issues on duplicate attempts to connect to the + :: same channel. When an EventSource isn't connected, we set a timer + :: to reap the subscriptions. This timer shouldn't be too short + :: because the + :: + state=(each timer duct) + :: next-id: next sequence number to use + :: + next-id=@ud + :: last-ack: time of last client ack + :: + :: used for clog calculations, in combination with :unacked + :: + last-ack=@da + :: events: unacknowledged events + :: + :: We keep track of all events where we haven't received a + :: 'Last-Event-Id: ' response from the client or a per-poke {'ack': + :: ...} call. When there's an active EventSource connection on this + :: channel, we send the event but we still add it to events because we + :: can't assume it got received until we get an acknowledgment. + :: + events=(qeu [id=@ud request-id=@ud =channel-event]) + :: unacked: unacknowledged event counts by request-id + :: + :: used for clog calculations, in combination with :last-ack + :: + unacked=(map @ud @ud) + :: subscriptions: gall subscriptions by request-id + :: + :: We maintain a list of subscriptions so if a channel times out, we + :: can cancel all the subscriptions we've made. + :: + subscriptions=(map @ud [ship=@p app=term =path duc=duct]) + :: heartbeat: sse heartbeat timer + :: + heartbeat=(unit timer) + == + :: +binding: A rule to match a path. + :: + :: A +binding is a system unique mapping for a path to match. A +binding + :: must be system unique because we don't want two handlers for a path; + :: what happens if there are two different actions for [~ /]? + :: + +$ binding + $: :: site: the site to match. + :: + :: A ~ will match the Urbit's identity site (your.urbit.org). Any + :: other value will match a domain literal. + :: + site=(unit @t) + :: path: matches this prefix path + :: + :: /~myapp will match /~myapp or /~myapp/longer/path + :: + path=(list @t) + == + :: +action: the action to take when a binding matches an incoming request + :: + +$ action + $% :: dispatch to a generator + :: + [%gen =generator] + :: dispatch to an application + :: + [%app app=term] + :: internal authentication page + :: + [%authentication ~] + :: cross-ship authentication handling + :: + [%eauth ~] + :: internal logout page + :: + [%logout ~] + :: gall channel system + :: + [%channel ~] + :: gall scry endpoint + :: + [%scry ~] + :: respond with the @p the requester is authenticated as + :: + [%name ~] + :: respond with the @p of the ship serving the response + :: + [%host ~] + :: respond with the ip address of the requester + :: + [%ip ~] + :: returns data used to verify sync status between ship and network + :: in double boot protection + :: + [%boot ~] + :: responds with the @p of the galaxy of the provided ship + :: + [%sponsor ~] + :: respond with the default file not found page + :: + [%four-oh-four ~] + == + :: +generator: a generator on the local ship that handles requests + :: + :: This refers to a generator on the local ship, run with a set of + :: arguments. Since http requests are time sensitive, we require that the + :: generator be on the current ship. + :: + +$ generator + $: :: desk: desk on current ship that contains the generator + :: + =desk + :: path: path on :desk to the generator's hoon file + :: + path=(list @t) + :: args: arguments passed to the gate + :: + args=* + == + :: +http-config: full http-server configuration + :: + +$ http-config + $: :: secure: PEM-encoded RSA private key and cert or cert chain + :: + secure=(unit [key=wain cert=wain]) + :: proxy: reverse TCP proxy HTTP(s) + :: + proxy=_| + :: log: keep HTTP(s) access logs + :: + log=? + :: redirect: send 301 redirects to upgrade HTTP to HTTPS + :: + :: Note: requires certificate. + :: + redirect=? + == + :: +http-rule: update configuration + :: + +$ http-rule + $% :: %cert: set or clear certificate and keypair + :: + [%cert cert=(unit [key=wain cert=wain])] + :: %turf: add remove or reset established dns binding + :: + $: %turf + $= action + $% [%put =turf] + [%del =turf] + [%new turfs=(set turf)] + == == == + :: +address: client IP address + :: + +$ address + $% [%ipv4 @if] + [%ipv6 @is] + :: [%ames @p] + == + :: +inbound-request: +http-request and metadata + :: + +$ inbound-request + $: :: authenticated: has a valid session cookie + :: + authenticated=? + :: secure: whether this request was encrypted (https) + :: + secure=? + :: address: the source address of this request + :: + =address + :: request: the http-request itself + :: + =request:http + == + :: + +$ cred :: credential + $: hut=hart :: client host + aut=(jug @tas @t) :: client identities + orx=oryx :: CSRF secret + acl=(unit @t) :: accept-language + cip=(each @if @is) :: client IP + cum=(map @tas *) :: custom dirt + == :: + +$ epic :: FCGI parameters + $: qix=(map @t @t) :: query + ced=cred :: client credentials + bem=beam :: original path + == :: + :: + +$ hart [p=? q=(unit @ud) r=host] :: http sec+port+host + +$ hate [p=purl q=@p r=moth] :: semi-cooked request + +$ hiss [p=purl q=moth] :: outbound request + +$ host (each turf @if) :: http host + +$ hoke %+ each [%localhost ~] :: local host + ?(%.0.0.0.0 %.127.0.0.1) :: + +$ httq :: raw http request + $: p=meth :: method + q=@t :: unparsed url + r=(list [p=@t q=@t]) :: headers + s=(unit octs) :: body + == :: + +$ httr [p=@ud q=mess r=(unit octs)] :: raw http response + +$ math (map @t (list @t)) :: semiparsed headers + +$ mess (list [p=@t q=@t]) :: raw http headers + +$ meth :: http methods + $? %conn :: CONNECT + %delt :: DELETE + %get :: GET + %head :: HEAD + %opts :: OPTIONS + %post :: POST + %put :: PUT + %trac :: TRACE + == :: + +$ moth [p=meth q=math r=(unit octs)] :: http operation + +$ oryx @t :: CSRF secret + +$ pork [p=(unit @ta) q=(list @t)] :: fully parsed url + :: +prox: proxy notification + :: + :: Used on both the proxy (ward) and upstream sides for + :: sending/receiving proxied-request notifications. + :: + +$ prox + $: :: por: tcp port + :: + por=@ud + :: sek: secure? + :: + sek=? + :: non: authentication nonce + :: + non=@uvJ + == + +$ purf (pair purl (unit @t)) :: url with fragment + +$ purl [p=hart q=pork r=quay] :: parsed url + +$ quay (list [p=@t q=@t]) :: parsed url query + ++ quer |-($@(~ [p=@t q=@t t=$])) :: query tree + +$ quri :: request-uri + $% [%& p=purl] :: absolute + [%| p=pork q=quay] :: relative + == :: + :: +reserved: check if an ipv4 address is in a reserved range + :: + ++ reserved + |= a=@if + ^- ? + =/ b (flop (rip 3 a)) + :: 0.0.0.0/8 (software) + :: + ?. ?=([@ @ @ @ ~] b) & + ?| :: 10.0.0.0/8 (private) + :: + =(10 i.b) + :: 100.64.0.0/10 (carrier-grade NAT) + :: + &(=(100 i.b) (gte i.t.b 64) (lte i.t.b 127)) + :: 127.0.0.0/8 (localhost) + :: + =(127 i.b) + :: 169.254.0.0/16 (link-local) + :: + &(=(169 i.b) =(254 i.t.b)) + :: 172.16.0.0/12 (private) + :: + &(=(172 i.b) (gte i.t.b 16) (lte i.t.b 31)) + :: 192.0.0.0/24 (protocol assignment) + :: + &(=(192 i.b) =(0 i.t.b) =(0 i.t.t.b)) + :: 192.0.2.0/24 (documentation) + :: + &(=(192 i.b) =(0 i.t.b) =(2 i.t.t.b)) + :: 192.18.0.0/15 (reserved, benchmark) + :: + &(=(192 i.b) |(=(18 i.t.b) =(19 i.t.b))) + :: 192.51.100.0/24 (documentation) + :: + &(=(192 i.b) =(51 i.t.b) =(100 i.t.t.b)) + :: 192.88.99.0/24 (reserved, ex-anycast) + :: + &(=(192 i.b) =(88 i.t.b) =(99 i.t.t.b)) + :: 192.168.0.0/16 (private) + :: + &(=(192 i.b) =(168 i.t.b)) + :: 203.0.113/24 (documentation) + :: + &(=(203 i.b) =(0 i.t.b) =(113 i.t.t.b)) + :: 224.0.0.0/8 (multicast) + :: 240.0.0.0/4 (reserved, future) + :: 255.255.255.255/32 (broadcast) + :: + (gte i.b 224) + == + :: +ipa: parse ip address + :: + ++ ipa + ;~(pose (stag %ipv4 ip4) (stag %ipv6 ip6)) + :: +ip4: parse ipv4 address + :: + ++ ip4 + =+ byt=(ape:ag ted:ab) + (bass 256 ;~(plug byt (stun [3 3] ;~(pfix dot byt)))) + :: +ip6: parse ipv6 address + :: + ++ ip6 + %+ bass 0x1.0000 + %+ sear + |= hexts=(list $@(@ [~ %zeros])) + ^- (unit (list @)) + :: not every list of hextets is an ipv6 address + :: + =/ legit=? + =+ l=(lent hexts) + =+ c=|=(a=* ?=([~ %zeros] a)) + ?| &((lth l 8) ?=([* ~] (skim hexts c))) + &(=(8 l) !(lien hexts c)) + == + ?. legit ~ + %- some + :: expand zeros + :: + %- zing + %+ turn hexts + |= hext=$@(@ [~ %zeros]) + ?@ hext [hext]~ + (reap (sub 9 (lent hexts)) 0) + :: parse hextets, producing cell for shorthand zeroes + :: + |^ %+ cook + |= [a=(list @) b=(list [~ %zeros]) c=(list @)] + :(welp a b c) + ;~ plug + (more col het) + (stun [0 1] cel) + (more col het) + == + ++ cel (cold `%zeros ;~(plug col col)) + ++ het (bass 16 (stun [1 4] six:ab)) + -- + :: + +$ rout [p=(list host) q=path r=oryx s=path] :: http route (new) + +$ user knot :: username + :: + :: Boot response + :: + +$ boot + $: %1 + sponsor=ship + =rift + =life + bone=(unit @udbone) + last-acked=(unit @udmessagenum) + == + -- ::eyre +:: :::: +:::: ++gall :: (1g) extensions + :: :::: +++ gall ^? + |% + +$ gift :: outgoing result + $% [%boon payload=*] :: ames response + [%noon id=* payload=*] + [%done error=(unit error:ames)] :: ames message (n)ack + [%flub ~] :: not ready to handle plea + [%unto p=unto] :: + == :: + +$ task :: incoming request + $~ [%vega ~] :: + $% [%clog id=*] :: clog notification + [%deal p=sack q=term r=deal] :: full transmission + [%sear =ship] :: clear pending queues + [%jolt =desk =dude] :: (re)start agent + [%idle =dude] :: suspend agent + [%load =load] :: load agent + [%nuke =dude] :: delete agent + [%doff dude=(unit dude) ship=(unit ship)] :: kill subscriptions + [%rake dude=(unit dude) all=?] :: reclaim old subs + [%lave subs=(list [?(%g %a) ship dude duct])] :: delete stale bitt(s) + $>(%init vane-task) :: set owner + $>(%trim vane-task) :: trim state + $>(%vega vane-task) :: report upgrade + $>(%plea vane-task) :: network request + [%spew veb=(list verb)] :: set verbosity + [%sift dudes=(list dude)] :: per agent + == :: + +$ bitt (map duct (pair ship path)) :: incoming subs + +$ boat (map [=wire =ship =term] [acked=? =path]) :: outgoing subs + +$ boar (map [=wire =ship =term] nonce=@) :: and their nonces + :: + +$ fans ((mop @ud (pair @da (each page @uvI))) lte) + +$ plot + $: bob=(unit @ud) + fan=fans + == + +$ stats :: statistics + $: change=@ud :: processed move count + eny=@uvJ :: entropy + time=@da :: current event time + == + +$ hutch [rev=@ud idx=@ud key=@] + :: + +$ farm + $+ farm + $~ [%plot ~ ~] + $% [%coop p=hutch q=(map path plot)] + [%plot p=(unit plot) q=(map @ta farm)] + == + :: + +$ egg :: migratory agent state + $% [%nuke sky=(map spur @ud) cop=(map coop hutch)] :: see /sys/gall $yoke + $: %live + control-duct=duct + run-nonce=@t + sub-nonce=@ + =stats + =bitt + =boat + =boar + code=~ + old-state=[%| vase] + =beak + marks=(map duct mark) + sky=farm + ken=(jug spar:ames wire) + pen=(jug spar:ames wire) + gem=(jug coop [path page]) + == == + +$ egg-any $%([%15 egg-15] [%16 egg]) + +$ egg-15 + $% [%nuke sky=(map spur @ud)] + $: %live + control-duct=duct + run-nonce=@t + sub-nonce=@ + =stats + =bitt + =boat + =boar + code=~ + old-state=[%| vase] + =beak + marks=(map duct mark) + sky=(map spur plot) + ken=(jug spar:ames wire) + == == + :: + +$ bowl :: standard app state + $+ gall-agent-bowl :: + $: $: our=ship :: host + src=ship :: guest + dap=term :: agent + sap=path :: provenance + == :: + $: wex=boat :: outgoing subs + sup=bitt :: incoming subs + sky=(map path fans) :: scry bindings + == :: + $: act=@ud :: change number + eny=@uvJ :: entropy + now=@da :: current time + byk=beak :: load source + == == :: :: + +$ dude term :: server identity + +$ gill (pair ship term) :: general contact + +$ load (list [=dude =beak =agent]) :: loadout + +$ scar :: opaque duct + $: p=@ud :: bone sequence + q=(map duct bone) :: by duct + r=(map bone duct) :: by bone + == :: + +$ suss (trel dude @tas @da) :: config report + +$ well (pair desk term) :: + +$ deal + $% [%raw-poke =mark =noun] + task:agent + == + +$ unto + $% [%raw-fact =mark =noun] + sign:agent + == + :: TODO: add more flags? + :: + +$ verb ?(%odd) + +$ coop spur + :: + :: +agent: app core + :: + ++ agent + =< form + |% + +$ step (quip card form) + +$ card + $+ gall-agent-card + (wind note gift) + +$ note + $+ gall-agent-note + $% [%agent [=ship name=term] =task] + [%arvo note-arvo] + [%pyre =tang] + :: + [%grow =spur =page] + [%tomb =case =spur] + [%cull =case =spur] + :: + [%tend =coop =path =page] + [%germ =coop] + [%snip =coop] + :: + [%keen secret=? spar:ames] + == + +$ task + $+ gall-agent-task + $% [%watch =path] + [%watch-as =mark =path] + [%leave ~] + [%poke =cage] + [%poke-as =mark =cage] + == + +$ gift + $+ gall-agent-gift + $% [%fact paths=(list path) =cage] + [%kick paths=(list path) ship=(unit ship)] + [%watch-ack p=(unit tang)] + [%poke-ack p=(unit tang)] + == + +$ sign + $+ gall-agent-sign + $% [%poke-ack p=(unit tang)] + [%watch-ack p=(unit tang)] + [%fact =cage] + [%kick ~] + == + ++ form + $_ ^| + |_ bowl + ++ on-init + *(quip card _^|(..on-init)) + :: + ++ on-save + *vase + :: + ++ on-load + |~ old-state=vase + *(quip card _^|(..on-init)) + :: + ++ on-poke + |~ [mark vase] + *(quip card _^|(..on-init)) + :: + ++ on-watch + |~ path + *(quip card _^|(..on-init)) + :: + ++ on-leave + |~ path + *(quip card _^|(..on-init)) + :: + ++ on-peek + |~ path + *(unit (unit cage)) + :: + ++ on-agent + |~ [wire sign] + *(quip card _^|(..on-init)) + :: + ++ on-arvo + |~ [wire sign-arvo] + *(quip card _^|(..on-init)) + :: + ++ on-fail + |~ [term tang] + *(quip card _^|(..on-init)) + -- + -- + -- ::gall +:: %iris http-client interface +:: +++ iris ^? + |% + :: +gift: effects the client can emit + :: + +$ gift + $% :: %request: outbound http-request to earth + :: + :: TODO: id is sort of wrong for this interface; the duct should + :: be enough to identify which request we're talking about? + :: + [%request id=@ud request=request:http] + :: %cancel-request: tell earth to cancel a previous %request + :: + [%cancel-request id=@ud] + :: %response: response to the caller + :: + [%http-response =client-response] + :: UIP-125 + :: + [%websocket-handshake id=@ud url=@t] + [%websocket-response id=@ud websocket-event:eyre] + == + :: + +$ task + $~ [%vega ~] + $% :: system started up; reset open connections + :: + $>(%born vane-task) + :: trim state (in response to memory pressure) + :: + $>(%trim vane-task) + :: report upgrade + :: + $>(%vega vane-task) + :: fetches a remote resource + :: + [%request =request:http =outbound-config] + :: cancels a previous fetch + :: + [%cancel-request ~] + :: receives http data from outside + :: + [%receive id=@ud =http-event:http] + :: UIP-125 + :: + [%websocket-connect app=term url=@t] + :: receives websocket event from earth + :: + [%websocket-event id=@ud event=websocket-event:eyre] + == + :: UIP-125 + :: + +$ websocket-connection + $: app=term + =duct + id=@ud + url=@t + status=?(%pending %accepted) + == + :: +client-response: one or more client responses given to the caller + :: + +$ client-response + $% :: periodically sent as an update on the duct that sent %fetch + :: + $: %progress + :: http-response-header: full transaction header + :: + :: In case of a redirect chain, this is the target of the + :: final redirect. + :: + =response-header:http + :: bytes-read: bytes fetched so far + :: + bytes-read=@ud + :: expected-size: the total size if response had a content-length + :: + expected-size=(unit @ud) + :: incremental: data received since the last %http-progress + :: + incremental=(unit octs) + == + :: final response of a download, parsed as mime-data if successful + :: + [%finished =response-header:http full-file=(unit mime-data)] + :: canceled by the runtime system + :: + [%cancel ~] + == + :: mime-data: externally received but unvalidated mimed data + :: + +$ mime-data + [type=@t data=octs] + :: +outbound-config: configuration for outbound http requests + :: + +$ outbound-config + $: :: number of times to follow a 300 redirect before erroring + :: + :: Common values for this will be 3 (the limit most browsers use), 5 + :: (the limit recommended by the http standard), or 0 (let the + :: requester deal with 300 redirects). + :: + redirects=_5 + :: number of times to retry before failing + :: + :: When we retry, we'll automatically try to use the 'Range' header + :: to resume the download where we left off if we have the + :: 'Accept-Range: bytes' in the original response. + :: + retries=_3 + == + :: +to-httr: adapts to old eyre interface + :: + ++ to-httr + |= [header=response-header:http full-file=(unit mime-data)] + ^- httr:eyre + :: + =/ data=(unit octs) + ?~(full-file ~ `data.u.full-file) + :: + [status-code.header headers.header data] + -- +:: :::: +:::: ++jael :: (1h) security + :: :::: +++ jael ^? + |% + +$ public-keys-result + $% [%full points=(map ship point)] + [%diff who=ship =diff:point] + [%breach who=ship] + == + :: :: + +$ gift :: out result <-$ + $% [%done error=(unit error:ames)] :: ames message (n)ack + [%boon payload=*] :: ames response + [%private-keys =life vein=(map life ring)] :: private keys + [%public-keys =public-keys-result] :: ethereum changes + [%turf turf=(list turf)] :: domains + == :: + :: +feed: potential boot parameters + :: + +$ feed + $^ $% [[%1 ~] who=ship kyz=(list [lyf=life key=ring])] + [[%2 ~] who=ship ryf=rift kyz=(list [lyf=life key=ring])] + == + seed + +$ seed [who=ship lyf=life key=ring sig=(unit oath:pki)] + :: + +$ task :: in request ->$ + $~ [%vega ~] :: + $% [%dawn dawn-event] :: boot from keys + [%fake =ship] :: fake boot + [%listen whos=(set ship) =source] :: set ethereum source + ::TODO %next for generating/putting new private key + [%meet =ship =life =pass] :: met after breach + [%moon =ship =udiff:point] :: register moon keys + [%nuke whos=(set ship)] :: cancel tracker from + [%private-keys ~] :: sub to privates + [%public-keys ships=(set ship)] :: sub to publics + [%rekey =life =ring] :: update private keys + [%resend ~] :: resend private key + [%ruin ships=(set ship)] :: pretend breach + $>(%trim vane-task) :: trim state + [%turf ~] :: view domains + $>(%vega vane-task) :: report upgrade + $>(%plea vane-task) :: ames request + [%step ~] :: reset web login code + == :: + :: + +$ dawn-event + $: =feed + spon=(list [=ship point:azimuth-types]) + czar=(map ship [=rift =life =pass]) + turf=(list turf) + bloq=@ud + node=(unit purl:eyre) + == + :: + ++ block + =< block + |% + +$ hash @uxblockhash + +$ number @udblocknumber + +$ id [=hash =number] + +$ block [=id =parent=hash] + -- + :: + :: Azimuth points form a groupoid, where the objects are all the + :: possible values of +point and the arrows are the possible values + :: of (list point-diff). Composition of arrows is concatenation, + :: and you can apply the diffs to a +point with +apply. + :: + :: It's simplest to consider +point as the coproduct of three + :: groupoids, Rift, Keys, and Sponsor. Recall that the coproduct + :: of monoids is the free monoid (Kleene star) of the coproduct of + :: the underlying sets of the monoids. The construction for + :: groupoids is similar. Thus, the objects of the coproduct are + :: the product of the objects of the underlying groupoids. The + :: arrows are a list of a sum of the diff types of the underlying + :: groupoids. Given an arrow=(list diff), you can project to the + :: underlying arrows with +skim filtering on the head of each diff. + :: + :: The identity element is ~. Clearly, composing this with any + :: +diff gives the original +diff. Since this is a category, + :: +compose must be associative (true, because concatenation is + :: associative). This is a groupoid, so we must further have that + :: every +point-diff has an inverse. These are given by the + :: +inverse operation. + :: + ++ point + =< point + |% + +$ point + $: =rift + =life + keys=(map life [crypto-suite=@ud =pass]) + sponsor=(unit @p) + == + :: + +$ key-update [=life crypto-suite=@ud =pass] + :: + :: Invertible diffs + :: + +$ diffs (list diff) + +$ diff + $% [%rift from=rift to=rift] + [%keys from=key-update to=key-update] + [%spon from=(unit @p) to=(unit @p)] + == + :: + :: Non-invertible diffs + :: + +$ udiffs (list [=ship =udiff]) + +$ udiff + $: =id:block + $% [%rift =rift boot=?] + [%keys key-update boot=?] + [%spon sponsor=(unit @p)] + [%disavow ~] + == == + :: + ++ udiff-to-diff + |= [=a=udiff =a=point] + ^- (unit diff) + ?- +<.a-udiff + %disavow ~|(%udiff-to-diff-disavow !!) + %spon `[%spon sponsor.a-point sponsor.a-udiff] + %rift + ?. (gth rift.a-udiff rift.a-point) + ~ + ~? &(!=(rift.a-udiff +(rift.a-point)) !boot.a-udiff) + [%udiff-to-diff-skipped-rift a-udiff a-point] + `[%rift rift.a-point rift.a-udiff] + :: + %keys + ?. (gth life.a-udiff life.a-point) + ~ + ~? &(!=(life.a-udiff +(life.a-point)) !boot.a-udiff) + [%udiff-to-diff-skipped-life a-udiff a-point] + :^ ~ %keys + [life.a-point (~(gut by keys.a-point) life.a-point *[@ud pass])] + [life crypto-suite pass]:a-udiff + == + :: + ++ inverse + |= diffs=(list diff) + ^- (list diff) + %- flop + %+ turn diffs + |= =diff + ^- ^diff + ?- -.diff + %rift [%rift to from]:diff + %keys [%keys to from]:diff + %spon [%spon to from]:diff + == + :: + ++ compose + (bake weld ,[(list diff) (list diff)]) + :: + ++ apply + |= [diffs=(list diff) =a=point] + (roll diffs (apply-diff a-point)) + :: + ++ apply-diff + |= a=point + |: [*=diff a-point=a] + ^- point + ?- -.diff + %rift + ?> =(rift.a-point from.diff) + a-point(rift to.diff) + :: + %keys + ?> =(life.a-point life.from.diff) + ?> =((~(get by keys.a-point) life.a-point) `+.from.diff) + %_ a-point + life life.to.diff + keys (~(put by keys.a-point) life.to.diff +.to.diff) + == + :: + %spon + ?> =(sponsor.a-point from.diff) + a-point(sponsor to.diff) + == + -- + :: :: + :::: :: + :: :: + +$ source (each ship term) + +$ source-id @udsourceid + :: + :: +state-eth-node: state of a connection to an ethereum node + :: + +$ state-eth-node :: node config + meta + $: top-source-id=source-id + sources=(map source-id source) + sources-reverse=(map source source-id) + default-source=source-id + ship-sources=(map ship source-id) + ship-sources-reverse=(jug source-id ship) + == :: + :: :: + :::: ++pki:jael :: (1h2) certificates + :: :::: + ++ pki ^? + |% + ::TODO update to fit azimuth-style keys + :: the urbit meta-certificate (++will) is a sequence + :: of certificates (++cert). each cert in a will + :: revokes and replaces the previous cert. the + :: version number of a ship is a ++life. + :: + :: the deed contains an ++arms, a definition + :: of cosmetic identity; a semi-trusted parent, + :: which signs the initial certificate and provides + :: routing services; and a dirty bit. if the dirty + :: bit is set, the new life of this ship may have + :: lost information that the old life had. + :: + +$ hand @uvH :: 128-bit hash + +$ mind [who=ship lyf=life] :: key identifier + +$ name (pair @ta @t) :: ascii / unicode + +$ oath @ :: signature + ++ tale :: urbit-signed * + |$ [typ] :: payload mold + $: dat=typ :: data + syg=(map ship (pair life oath)) :: signatures + == :: + -- :: pki + -- :: jael +:: :::: +:::: ++khan :: (1i) threads + :: :::: +++ khan ^? + |% + +$ gift :: out result <-$ + $% [%arow p=(avow cage)] :: in-arvo result + [%avow p=(avow page)] :: external result + == :: + +$ task :: in request ->$ + $~ [%vega ~] :: + $% $>(%born vane-task) :: new unix process + [%done ~] :: socket closed + :: TODO mark ignored :: + :: :: + [%fard p=(fyrd cage)] :: in-arvo thread + [%fyrd p=(fyrd cast)] :: external thread + [%lard =bear =shed] :: inline thread + $>(%trim vane-task) :: trim state + $>(%vega vane-task) :: report upgrade + == :: + :: :: + ++ avow |$ [a] (each a goof) :: $fyrd result + +$ bear $@(desk beak) :: partial $beak + +$ cast (pair mark page) :: output mark + input + ++ fyrd |$ [a] [=bear name=term args=a] :: thread run request + :: :: + +$ shed _*form:(strand:rand ,vase) :: compute vase + -- ::khan +:: :::: +:::: ++lick :: (1j) IPC + :: :::: +++ lick ^? + |% + +$ gift :: out result <-$ + $% [%spin =name] :: open an IPC port + [%shut =name] :: close an IPC port + [%spit =name =mark =noun] :: spit a noun to the IPC port + [%soak =name =mark =noun] :: soak a noun from the IPC port + == + +$ task :: in request ->$ + $~ [%vega ~] :: + $% $>(%born vane-task) :: new unix process + $>(%trim vane-task) :: trim state + $>(%vega vane-task) :: report upgrade + [%spin =name] :: open an IPC port + [%shut =name] :: close an IPC port + [%spit =name =mark =noun] :: spit a noun to the IPC port + [%soak =name =mark =noun] :: soak a noun from the IPC port + == + :: + +$ name path + -- ::lick +:: +++ rand :: computation + |% + +$ card card:agent:gall + +$ input + $+ input + $% [%poke =cage] + [%sign =wire =sign-arvo] + [%agent =wire =sign:agent:gall] + [%watch =path] + == + +$ error (pair term $+(tang tang)) + +$ strand-input + $+ strand-input + [=bowl in=(unit input)] + +$ tid @tatid + +$ bowl + $+ strand-bowl + $: our=ship + src=ship + tid=tid + mom=(unit tid) + wex=boat:gall + sup=bitt:gall + eny=@uvJ + now=@da + byk=beak + == + :: + :: cards: cards to send immediately. These will go out even if a + :: later stage of the computation fails, so they shouldn't have + :: any semantic effect on the rest of the system. + :: Alternately, they may record an entry in contracts with + :: enough information to undo the effect if the computation + :: fails. + :: wait: don't move on, stay here. The next sign should come back + :: to this same callback. + :: skip: didn't expect this input; drop it down to be handled + :: elsewhere + :: cont: continue computation with new callback. + :: fail: abort computation; don't send effects + :: done: finish computation; send effects + :: + ++ strand-output-raw + |* a=mold + $+ strand-output-raw + $~ [~ %done *a] + $: cards=(list card) + $= next + $% [%wait ~] + [%skip ~] + [%cont self=(strand-form-raw a)] + [%fail err=error] + [%done value=a] + == + == + :: + ++ strand-form-raw + |* a=mold + $+ strand-form-raw + $-(strand-input (strand-output-raw a)) + :: + :: Abort strand computation with error message + :: + ++ strand-fail + |= =error + |= strand-input + [~ %fail error] + :: + :: Asynchronous transcaction monad. + :: + :: Combo of four monads: + :: - Reader on input + :: - Writer on card + :: - Continuation + :: - Exception + :: + ++ strand + |* a=mold + |% + ++ output $+(output (strand-output-raw a)) + :: + :: Type of an strand computation. + :: + ++ form $+(form (strand-form-raw a)) + :: + :: Monadic pure. Identity computation for bind. + :: + ++ pure + |= arg=a + ^- form + |= strand-input + [~ %done arg] + :: + :: Monadic bind. Combines two computations, associatively. + :: + ++ bind + |* b=mold + |= [m-b=(strand-form-raw b) fun=$-(b form)] + ^- form + |= input=strand-input + =/ b-res=(strand-output-raw b) + (m-b input) + ^- output + :- cards.b-res + ?- -.next.b-res + %wait [%wait ~] + %skip [%skip ~] + %cont [%cont ..$(m-b self.next.b-res)] + %fail [%fail err.next.b-res] + %done [%cont (fun value.next.b-res)] + == + :: + :: The strand monad must be evaluted in a particular way to maintain + :: its monadic character. +take:eval implements this. + :: + ++ eval + |% + :: Indelible state of a strand + :: + +$ eval-form + $: =form + == + :: + :: Convert initial form to eval-form + :: + ++ from-form + |= =form + ^- eval-form + form + :: + :: The cases of results of +take + :: + +$ eval-result + $% [%next ~] + [%fail err=(pair term tang)] + [%done value=a] + == + :: + ++ validate-mark + |= [in=* =mark =bowl] + ^- cage + =+ .^ =dais:clay %cb + /(scot %p our.bowl)/[q.byk.bowl]/(scot %da now.bowl)/[mark] + == + =/ res (mule |.((vale.dais in))) + ?: ?=(%| -.res) + ~| %spider-mark-fail + (mean leaf+"spider: ames vale fail {<mark>}" p.res) + [mark p.res] + :: + :: Take a new sign and run the strand against it + :: + ++ take + :: cards: accumulate throughout recursion the cards to be + :: produced now + =| cards=(list card) + |= [=eval-form =strand-input] + ^- [[(list card) =eval-result] _eval-form] + =* take-loop $ + =. in.strand-input + ?~ in.strand-input ~ + =/ in u.in.strand-input + ?. ?=(%agent -.in) `in + ?. ?=(%fact -.sign.in) `in + ?: ?=(%thread-done p.cage.sign.in) `in + :: + :- ~ + :^ %agent wire.in %fact + (validate-mark q.q.cage.sign.in p.cage.sign.in bowl.strand-input) + :: run the strand callback + :: + =/ =output (form.eval-form strand-input) + :: add cards to cards + :: + =. cards + %+ welp + cards + :: XX add tag to wires? + cards.output + :: case-wise handle next steps + :: + ?- -.next.output + %wait [[cards %next ~] eval-form] + %skip [[cards %next ~] eval-form] + %fail [[cards %fail err.next.output] eval-form] + %done [[cards %done value.next.output] eval-form] + %cont + :: recurse to run continuation with initialization input + :: + %_ take-loop + form.eval-form self.next.output + strand-input [bowl.strand-input ~] + == + == + -- + -- + -- ::strand +:: ++$ gift-arvo :: out result <-$ + $~ [%doze ~] + $% gift:ames + gift:behn + gift:clay + gift:dill + gift:eyre + gift:gall + gift:iris + gift:jael + gift:khan + gift:lick + == ++$ task-arvo :: in request ->$ + $% task:ames + task:clay + task:behn + task:dill + task:eyre + task:gall + task:iris + task:jael + task:khan + task:lick + == ++$ note-arvo :: out request $-> + $~ [%b %wake ~] + $% [%a task:ames] + [%b task:behn] + [%c task:clay] + [%d task:dill] + [%e task:eyre] + [%g task:gall] + [%i task:iris] + [%j task:jael] + [%k task:khan] + [%l task:lick] + [%$ %whiz ~] + [@tas %meta vase] + == +:: full vane names are required in vanes +:: ++$ sign-arvo :: in result $<- + $% [%ames gift:ames] + $: %behn + $% gift:behn + $>(%wris gift:clay) + $>(%writ gift:clay) + $>(%mere gift:clay) + $>(%unto gift:gall) + == + == + [%clay gift:clay] + [%dill gift:dill] + [%eyre gift:eyre] + [%gall gift:gall] + [%iris gift:iris] + [%jael gift:jael] + [%khan gift:khan] + [%lick gift:lick] + == +:: $unix-task: input from unix +:: ++$ unix-task :: input from unix + $~ [%wake ~] + $% :: %dill: keyboard input + :: + $>(%belt task:dill) + :: %dill: configure terminal (resized) + :: + $>(%blew task:dill) + :: %clay: new process + :: + $>(%boat task:clay) + :: %behn/%eyre/%iris: new process + :: + $>(%born vane-task) + :: %eyre: cancel request + :: + [%cancel-request ~] + :: %dill: reset terminal configuration + :: + $>(%hail task:dill) + :: %ames: hear packet + :: + $>(?(%hear %heer) task:ames) + :: %clay: external edit + :: + $>(%into task:clay) + :: %clay: synchronous commit + :: + :: TODO: make $yuki an option for %into? + :: + $>(%park task:clay) + :: %clay: load blob store + :: + $>(%prep task:clay) + :: %clay: set essential desk + :: + $>(%esse task:clay) + :: %eyre: learn ports of live http servers + :: + $>(%live task:eyre) + :: %iris: hear (partial) http response + :: + $>(%receive task:iris) + :: %eyre: starts handling an inbound http request + :: + $>(%request task:eyre) + :: %eyre: starts handling an backdoor http request + :: + $>(%request-local task:eyre) + :: %dill: close session + :: + $>(%shut task:dill) + :: %behn: wakeup + :: + $>(%wake task:behn) + == +-- :: |
