-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathoverview.html
1420 lines (1384 loc) · 93.6 KB
/
overview.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!doctype html>
<html class="no-js" lang="en">
<head><meta charset="utf-8"/>
<meta name="viewport" content="width=device-width,initial-scale=1"/>
<meta name="color-scheme" content="light dark"><meta name="generator" content="Docutils 0.19: https://docutils.sourceforge.io/" />
<link rel="index" title="Index" href="genindex.html" /><link rel="search" title="Search" href="search.html" /><link rel="next" title="System Architecture" href="system-architecture.html" /><link rel="prev" title="Introduction" href="introduction.html" />
<!-- Generated with Sphinx 5.3.0 and Furo 2023.03.27 -->
<title>Technical Overview - snackabra 1.0 documentation</title>
<link rel="stylesheet" type="text/css" href="_static/pygments.css" />
<link rel="stylesheet" type="text/css" href="_static/styles/furo.css?digest=fad236701ea90a88636c2a8c73b44ae642ed2a53" />
<link rel="stylesheet" type="text/css" href="_static/graphviz.css" />
<link rel="stylesheet" type="text/css" href="_static/styles/furo-extensions.css?digest=30d1aed668e5c3a91c3e3bf6a60b675221979f0e" />
<link rel="stylesheet" type="text/css" href="_static/css/custom.css" />
<style>
body {
--color-code-background: #f8f8f8;
--color-code-foreground: black;
}
@media not print {
body[data-theme="dark"] {
--color-code-background: #202020;
--color-code-foreground: #d0d0d0;
}
@media (prefers-color-scheme: dark) {
body:not([data-theme="light"]) {
--color-code-background: #202020;
--color-code-foreground: #d0d0d0;
}
}
}
</style></head>
<body>
<script>
document.body.dataset.theme = localStorage.getItem("theme") || "auto";
</script>
<svg xmlns="http://www.w3.org/2000/svg" style="display: none;">
<symbol id="svg-toc" viewBox="0 0 24 24">
<title>Contents</title>
<svg stroke="currentColor" fill="currentColor" stroke-width="0" viewBox="0 0 1024 1024">
<path d="M408 442h480c4.4 0 8-3.6 8-8v-56c0-4.4-3.6-8-8-8H408c-4.4 0-8 3.6-8 8v56c0 4.4 3.6 8 8 8zm-8 204c0 4.4 3.6 8 8 8h480c4.4 0 8-3.6 8-8v-56c0-4.4-3.6-8-8-8H408c-4.4 0-8 3.6-8 8v56zm504-486H120c-4.4 0-8 3.6-8 8v56c0 4.4 3.6 8 8 8h784c4.4 0 8-3.6 8-8v-56c0-4.4-3.6-8-8-8zm0 632H120c-4.4 0-8 3.6-8 8v56c0 4.4 3.6 8 8 8h784c4.4 0 8-3.6 8-8v-56c0-4.4-3.6-8-8-8zM115.4 518.9L271.7 642c5.8 4.6 14.4.5 14.4-6.9V388.9c0-7.4-8.5-11.5-14.4-6.9L115.4 505.1a8.74 8.74 0 0 0 0 13.8z"/>
</svg>
</symbol>
<symbol id="svg-menu" viewBox="0 0 24 24">
<title>Menu</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather-menu">
<line x1="3" y1="12" x2="21" y2="12"></line>
<line x1="3" y1="6" x2="21" y2="6"></line>
<line x1="3" y1="18" x2="21" y2="18"></line>
</svg>
</symbol>
<symbol id="svg-arrow-right" viewBox="0 0 24 24">
<title>Expand</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="feather-chevron-right">
<polyline points="9 18 15 12 9 6"></polyline>
</svg>
</symbol>
<symbol id="svg-sun" viewBox="0 0 24 24">
<title>Light mode</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round" class="feather-sun">
<circle cx="12" cy="12" r="5"></circle>
<line x1="12" y1="1" x2="12" y2="3"></line>
<line x1="12" y1="21" x2="12" y2="23"></line>
<line x1="4.22" y1="4.22" x2="5.64" y2="5.64"></line>
<line x1="18.36" y1="18.36" x2="19.78" y2="19.78"></line>
<line x1="1" y1="12" x2="3" y2="12"></line>
<line x1="21" y1="12" x2="23" y2="12"></line>
<line x1="4.22" y1="19.78" x2="5.64" y2="18.36"></line>
<line x1="18.36" y1="5.64" x2="19.78" y2="4.22"></line>
</svg>
</symbol>
<symbol id="svg-moon" viewBox="0 0 24 24">
<title>Dark mode</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round" class="icon-tabler-moon">
<path stroke="none" d="M0 0h24v24H0z" fill="none" />
<path d="M12 3c.132 0 .263 0 .393 0a7.5 7.5 0 0 0 7.92 12.446a9 9 0 1 1 -8.313 -12.454z" />
</svg>
</symbol>
<symbol id="svg-sun-half" viewBox="0 0 24 24">
<title>Auto light/dark mode</title>
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24" fill="none" stroke="currentColor"
stroke-width="1.5" stroke-linecap="round" stroke-linejoin="round" class="icon-tabler-shadow">
<path stroke="none" d="M0 0h24v24H0z" fill="none"/>
<circle cx="12" cy="12" r="9" />
<path d="M13 12h5" />
<path d="M13 15h4" />
<path d="M13 18h1" />
<path d="M13 9h4" />
<path d="M13 6h1" />
</svg>
</symbol>
</svg>
<input type="checkbox" class="sidebar-toggle" name="__navigation" id="__navigation">
<input type="checkbox" class="sidebar-toggle" name="__toc" id="__toc">
<label class="overlay sidebar-overlay" for="__navigation">
<div class="visually-hidden">Hide navigation sidebar</div>
</label>
<label class="overlay toc-overlay" for="__toc">
<div class="visually-hidden">Hide table of contents sidebar</div>
</label>
<div class="page">
<header class="mobile-header">
<div class="header-left">
<label class="nav-overlay-icon" for="__navigation">
<div class="visually-hidden">Toggle site navigation sidebar</div>
<i class="icon"><svg><use href="#svg-menu"></use></svg></i>
</label>
</div>
<div class="header-center">
<a href="index.html"><div class="brand">snackabra 1.0 documentation</div></a>
</div>
<div class="header-right">
<div class="theme-toggle-container theme-toggle-header">
<button class="theme-toggle">
<div class="visually-hidden">Toggle Light / Dark / Auto color theme</div>
<svg class="theme-icon-when-auto"><use href="#svg-sun-half"></use></svg>
<svg class="theme-icon-when-dark"><use href="#svg-moon"></use></svg>
<svg class="theme-icon-when-light"><use href="#svg-sun"></use></svg>
</button>
</div>
<label class="toc-overlay-icon toc-header-icon" for="__toc">
<div class="visually-hidden">Toggle table of contents sidebar</div>
<i class="icon"><svg><use href="#svg-toc"></use></svg></i>
</label>
</div>
</header>
<aside class="sidebar-drawer">
<div class="sidebar-container">
<div class="sidebar-sticky"><a class="sidebar-brand" href="index.html">
<span class="sidebar-brand-text">snackabra 1.0 documentation</span>
</a><form class="sidebar-search-container" method="get" action="search.html" role="search">
<input class="sidebar-search" placeholder="Search" name="q" aria-label="Search">
<input type="hidden" name="check_keywords" value="yes">
<input type="hidden" name="area" value="default">
</form>
<div id="searchbox"></div><div class="sidebar-scroll"><div class="sidebar-tree">
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="introduction.html">Introduction</a></li>
<li class="toctree-l1 current current-page"><a class="current reference internal" href="#">Technical Overview</a></li>
<li class="toctree-l1"><a class="reference internal" href="system-architecture.html">System Architecture</a></li>
<li class="toctree-l1"><a class="reference internal" href="discussion.html">Background and Discussion</a></li>
<li class="toctree-l1"><a class="reference internal" href="formal.html">Formal Treatment</a></li>
<li class="toctree-l1"><a class="reference internal" href="install.html">Installation</a></li>
<li class="toctree-l1"><a class="reference internal" href="contact.html">Contact and Feedback</a></li>
<li class="toctree-l1"><a class="reference internal" href="glossary.html">Glossary</a></li>
<li class="toctree-l1"><a class="reference internal" href="future.html">Future Work</a></li>
<li class="toctree-l1"><a class="reference internal" href="references.html">References / Further Reading</a></li>
<li class="toctree-l1"><a class="reference internal" href="license.html">LICENSE</a></li>
<li class="toctree-l1"><a class="reference internal" href="jslib.html">JSLib User Manual</a></li>
<li class="toctree-l1"><a class="reference internal" href="modules.html">JSLib Reference Manual</a></li>
<li class="toctree-l1"><a class="reference internal" href="server.html">Snackabra Server</a></li>
<li class="toctree-l1"><a class="reference internal" href="pylib.html">Python Library</a></li>
<li class="toctree-l1"><a class="reference internal" href="appendix-a-crypto.html">Appendix A: Cryptography</a></li>
<li class="toctree-l1"><a class="reference internal" href="user-guide.html">Appendix B: Privacy.App Chat Room User Guide</a></li>
<li class="toctree-l1"><a class="reference internal" href="diag-sample.html">(ignore)</a></li>
<li class="toctree-l1"><a class="reference internal" href="motivation.html">Motivation</a></li>
</ul>
</div>
</div>
</div>
</div>
</aside>
<div class="main">
<div class="content">
<div class="article-container">
<a href="#" class="back-to-top muted-link">
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 24 24">
<path d="M13 20h-2V8l-5.5 5.5-1.42-1.42L12 4.16l7.92 7.92-1.42 1.42L13 8v12z"></path>
</svg>
<span>Back to top</span>
</a>
<div class="content-icon-container">
<div class="theme-toggle-container theme-toggle-content">
<button class="theme-toggle">
<div class="visually-hidden">Toggle Light / Dark / Auto color theme</div>
<svg class="theme-icon-when-auto"><use href="#svg-sun-half"></use></svg>
<svg class="theme-icon-when-dark"><use href="#svg-moon"></use></svg>
<svg class="theme-icon-when-light"><use href="#svg-sun"></use></svg>
</button>
</div>
<label class="toc-overlay-icon toc-content-icon" for="__toc">
<div class="visually-hidden">Toggle table of contents sidebar</div>
<i class="icon"><svg><use href="#svg-toc"></use></svg></i>
</label>
</div>
<article role="main">
<section id="technical-overview">
<span id="overview"></span><h1>Technical Overview<a class="headerlink" href="#technical-overview" title="Permalink to this heading">#</a></h1>
<p>This part will give a fairly technical overview of the system(s).
Make sure you’ve read through the <a class="reference internal" href="introduction.html#introduction"><span class="std std-ref">Introduction</span></a>.</p>
<section id="rooms">
<h2>Rooms<a class="headerlink" href="#rooms" title="Permalink to this heading">#</a></h2>
<p>A client connecting to a room will first check for their own
participant keys in their localstorage.</p>
<p>If the local_storage has no participant keys for the room, that means
the person joining is new to the room. A dialog box is shown to
confirm that the user understands this, if not, they’re prompted to
load keys from backup.</p>
<p>Otherwise, a new participant key is generated and stored in
localstorage.</p>
<p>Otherwise, the client connects to websocket and sends their public
participant key to the DO. The DO replies with a ‘ready’ message
containing the following data from <a class="reference internal" href="glossary.html#term-KV_local"><span class="xref std std-term">KV_local</span></a>:</p>
<ul class="simple">
<li><p>keys:</p>
<ul>
<li><p><room>_ownerKey</p></li>
<li><p><room>_encryptionKey</p></li>
<li><p><room>_signKey</p></li>
<li><p><room>_guestKey (‘Null’ if not present in <a class="reference internal" href="glossary.html#term-KV_global"><span class="xref std std-term">KV_global</span></a>)</p></li>
<li><p><room>_authorizationKey</p></li>
</ul>
</li>
<li><p>MotD</p></li>
</ul>
<p>(See the <a class="reference internal" href="#roomuserkeydetails"><span class="std std-ref">Room and User Keys</span></a> section for
details, the above is just a summary.)</p>
<p>If the _ownerKey is Null, the room is “non-existent” meaning it has
not been created. If a participant tries to message a non-existent
room, a system message is displayed to the client which says that the
room has not been initiated yet. The keys (and the existence of the
room) is generated by administrative tools (CLI).</p>
<p>If the _guestKey is Null, it is set to the public half of the first
participant (who does <em>not</em> have the cookie, e.g., who is not Owner).</p>
</section>
<section id="owner-and-admin">
<h2>Owner and Admin<a class="headerlink" href="#owner-and-admin" title="Permalink to this heading">#</a></h2>
<p>If the participant key is the same as the roomKey, the client UI will
assume that it’s the Owner. This only affects the whisper UI.</p>
<p>If there is a correctly formed cookie (from the SSO backend), the Chat
UI will show the Admin tab, and the Chat backend will consider this
participant <em>authenticated</em> Owner and allow Admin API calls (cookie is
included in each call). Current Admin operations are: restrict a room,
accept a request to join a restricted room, and set the MotD (see
below).</p>
<p>The time to live for the cookie is one day:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">token_</span><span class="o"><</span><span class="n">room</span><span class="o">></span><span class="p">:</span> <span class="n">epoch</span><span class="o">+</span><span class="s1">'.’+sign(<epoch>+'</span><span class="n">_</span><span class="o"><</span><span class="n">roomId</span><span class="o">></span><span class="p">);</span><span class="n">domain</span><span class="o">=.</span><span class="n">privacy</span><span class="o">.</span><span class="n">app</span><span class="p">;</span><span class="n">secure</span><span class="p">;</span><span class="n">samesite</span><span class="o">=</span><span class="n">strict</span><span class="p">;</span><span class="nb">max</span><span class="o">-</span><span class="n">age</span><span class="o">=</span><span class="mi">86400</span><span class="s1">';</span>
</pre></div>
</div>
<p>Where signing is done with ECDSA using <roomId>_authorizationKey (with SHA-256)</p>
<p>Some notes:</p>
<ol class="arabic">
<li><p>For an Owner to join “as Owner”, both the above cookie and the
participant key (private half of the ownerKey) will need to be set,
such as by using sendMessage/iFrame.</p></li>
<li><p>In default setup, the Owner’s (private) room owner keys need to be
kept separate; in our SSO database.</p>
<p>If the Owner so wishes, they can generate a new pair in the SSO,
overwrite the corresponding room ownerKey in <a class="reference internal" href="glossary.html#term-KV_global"><span class="xref std std-term">KV_global</span></a>, and store the
private key <em>only</em> in their localstorage.</p>
<p>[internal: In a future upgrade to the membership kits, we can
include the private keys for all the rooms (current and future!) on
the USB. This would allow us to offer <em>physical</em> key recovery service
for members, without <em>ever</em> having the private keys on a networked
computer.]</p>
</li>
<li><p>Restriction is a “fire once” operation - when the Owner restricts a
room, all participant clients can verify the public key of the
Owner <em>as well as</em> the assurance from the worker back end <em>and</em> the
separate SSO that this is indeed the Owner.</p></li>
</ol>
</section>
<section id="message-of-the-day">
<h2>Message of the Day<a class="headerlink" href="#message-of-the-day" title="Permalink to this heading">#</a></h2>
<p>Owner can change the MotD. It’s shown every time the websocket
connects (e.g. whenever you reload the room, enter it for the first
time, etc).</p>
</section>
<section id="whisper">
<h2>Whisper:<a class="headerlink" href="#whisper" title="Permalink to this heading">#</a></h2>
<p>Whispers are for communication between just the Owner and one
participant, and can be initiated from either party. For everyone
else, the message appears in a yellow background as <em>(whispered)</em>
unless the room is restricted. <a class="footnote-reference brackets" href="#f025" id="id1" role="doc-noteref"><span class="fn-bracket">[</span>1<span class="fn-bracket">]</span></a></p>
<p>Guests can whisper to Owner by tapping the profile icon at the top
right; Owners can whisper by long pressing the “Name” circle of any
message.</p>
<p>This encrypts the message (text and image separately) using the
encryption key derived from the private key of the guest and the
<room>_ownerKey. <a class="footnote-reference brackets" href="#f026" id="id2" role="doc-noteref"><span class="fn-bracket">[</span>2<span class="fn-bracket">]</span></a></p>
<p>The owner can whisper to anyone who has sent a message to the room
once by long pressing/long clicking on the user icon for the guest
they want to whisper to. This encrypts the message (text and image
separately) using the encryption key derived from the private key of
the owner (generated when first joined) and the public key of the
guest.</p>
<p>To decrypt a whispered message, key generation is done using the
user’s private key and the sender’s public key which is included in
the message.</p>
<p>(A whisper precludes the need for signing the message.)</p>
</section>
<section id="signing">
<h2>Signing:<a class="headerlink" href="#signing" title="Permalink to this heading">#</a></h2>
<p>If a message is not whispered, it will be signed by the sender. If a
message is not whispered and fails verification (sign not
present/corrupted), it is displayed with a red outline.</p>
<p>Each part to be signed (text, image, image metadata) is signed using
the sign key derived from the private key of the sender and the public
half from <room>_signKey.</p>
<p>Each part is signed using an ‘HMAC’ key derived using the private half
of the sender’s participant key or <room> key and the public half of
the <room>_signKey. All 3 signs are verified using the key derived
from the public key of the sender and the private half of the
<room>_signKey.</p>
</section>
<section id="restricting-a-room">
<h2>Restricting a room<a class="headerlink" href="#restricting-a-room" title="Permalink to this heading">#</a></h2>
<p>When the owner restricts the room, a new encryption key is generated
and stored in the local_storage. All guests who have visited the room
once will be added to “Visitor Requests”. The owner will also be added
to this list and automatically approved.</p>
<p>A restricted room will result in a conversation that nobody outside
the group of participants can read (any participant can read all
messages).</p>
<p>Any new visitor will automatically generate a new request to the
Owner.</p>
<p>A restricted room has a green “locked” icon next to its name.</p>
<p>The Durable Object backend maintains a list of the public key of
‘accepted visitors’ in <a class="reference internal" href="glossary.html#term-KV_local"><span class="xref std std-term">KV_local</span></a>. The Durable Object backend also
maintains a JavaScript Object of all ‘locked_keys’ wherein the
‘encrypted’ locked_key for each ‘accepted guest’ (look at the section
‘Accepting a guest’) is stored corresponding to the public half of the
visitor’s participant or <room> key.</p>
</section>
<section id="accepting-a-guest">
<h2>Accepting a guest<a class="headerlink" href="#accepting-a-guest" title="Permalink to this heading">#</a></h2>
<p>When the room owner accepts a guest to a restricted room, the key
stored in the local_storage of the owner as <room>_lockedKey will be
encrypted using the encryption key derived from the private key of the
owner and public key of the guest. This encrypted key will be sent to
the Durable Objects backend and stored there (<a class="reference internal" href="glossary.html#term-KV_local"><span class="xref std std-term">KV_local</span></a>).</p>
<p>Whenever a guest joins a restricted room, if they have been accepted,
this encrypted key will be sent to them as the first ‘message’ from
the Websocket. The key will be decrypted using the key derived from
their private key and the public key of the owner and then stored to
the localstorage.</p>
</section>
<section id="owner-key-management">
<h2>Owner Key Management<a class="headerlink" href="#owner-key-management" title="Permalink to this heading">#</a></h2>
<p>The Owner keys are initially managed by the SSO, a bit like if they
were using a password manager. However, this default setup exposes the
Owner to the Institute, for whatever reason, wanting to “impersonate”
them (since the membership page is the SSO service). It also exposes
participants to some extent to security issues in underlying
infrastructure (see the ‘Discussion’ section at the end of this
document).</p>
<p>To give Owners an option for stronger privacy <a class="footnote-reference brackets" href="#f027" id="id3" role="doc-noteref"><span class="fn-bracket">[</span>3<span class="fn-bracket">]</span></a> they can
regenerate their keys for any given room, from their membership
page. When that happens, a new public ECDH key pair is created in the
client. However the <em>private</em> key is <em>not</em> stored in the SSO
system. The public half of this key is then signed using the current
<room>_ownerKey (the key before rotation occurs). The sign and the
public half of the new key are then sent to <a class="reference internal" href="glossary.html#term-KV_global"><span class="xref std std-term">KV_global</span></a> in a fetch
request which stores the received value (key + sign) as
<room>_ownerKey<ts>, where ts is the timestamp when it performs the
store operation. (We also refer to this as ‘key rotation’ or ‘locking’
a room.)</p>
<p>The owner’s chat client will then make a fetch request to the Durable
Object to refresh it’s maintained copy of the ownerKey in
<a class="reference internal" href="glossary.html#term-KV_local"><span class="xref std std-term">KV_local</span></a>. The Durable Object pings <a class="reference internal" href="glossary.html#term-KV_global"><span class="xref std std-term">KV_global</span></a> and if the <a class="reference internal" href="glossary.html#term-KV_global"><span class="xref std std-term">KV_global</span></a>
returns an ownerKey different from the ownerKey in <a class="reference internal" href="glossary.html#term-KV_local"><span class="xref std std-term">KV_local</span></a>, the DO
broadcasts it to all active chat clients using websockets.</p>
<p>Note: In the current design, the sign which is stored along with the
key is not utilized. However, in future iterations, any user will
independently be able to fetch all <room>_ownerKeys (meaning all
rotated keys) and verify that all room_ownerKey rotations were signed
by the owner key before rotation and hence, verify that key rotations
were initiated by the owner.</p>
<p>Note: only restricted rooms can be locked-down.</p>
<p>Note: once the Owner has rotated their keys, all the other participant
clients will note that hereafter, only commands (such as additional
key rotations) signed by this set of Owner keys are respected.</p>
<p>When a room is both restricted and rotated (‘locked-down’) a different
lock icon is shown next to the room name.</p>
</section>
<section id="image-photo-sharing">
<span id="photosharing"></span><h2>Image (Photo) Sharing <a class="footnote-reference brackets" href="#f043" id="id4" role="doc-noteref"><span class="fn-bracket">[</span>17<span class="fn-bracket">]</span></a><a class="headerlink" href="#image-photo-sharing" title="Permalink to this heading">#</a></h2>
<p>A major design element is to accomplish scaleable, reliable,
economical, secure, and private mechanisms for sharing and accessing
images.</p>
<p>The attach icon allows you to send an image, or (on modern
device/browser combinations) take a picture from the camera straight
to the chat app.</p>
<p>The instant the image is “uploaded” (given to the web app), it is
processed <a class="footnote-reference brackets" href="#f030" id="id5" role="doc-noteref"><span class="fn-bracket">[</span>4<span class="fn-bracket">]</span></a>, in JPG format. This transform has as a side
effect that any metadata is removed.</p>
<p>If the image in the message is “sent”, then in fact three versions are
processed:</p>
<ul class="simple">
<li><p><a class="reference internal" href="glossary.html#term-thumbnail"><span class="xref std std-term">thumbnail</span></a> - this is a (max) 20KB version of the processed
image. This is transmitted using websockets and stored within the
message as a DataURL (<em>not</em> an image link) and should appear close
to instantly for all participants. The 20KB limit was originally chosen to allow
the full image to fit in the <a class="reference internal" href="glossary.html#term-KV_local"><span class="xref std std-term">KV_local</span></a>. The entire object is enclosed
in the (encrypted) image.</p></li>
<li><p><a class="reference internal" href="glossary.html#term-preview"><span class="xref std std-term">preview</span></a> - this is a (max) 4MB processed image. This is stored in
the <a class="reference internal" href="glossary.html#term-KV_global"><span class="xref std std-term">KV_global</span></a> (see next section) and can be retrieved by anyone who
has access to the message with which it was shared (and that has the
<a class="reference internal" href="glossary.html#term-thumbnail"><span class="xref std std-term">thumbnail</span></a>).</p></li>
<li><p><strong>full</strong> - this is the 16MB or less version of the image. If the
“original” is small enough, then it is stored unaltered. If it’s
bigger, then it is processed as above. <a class="footnote-reference brackets" href="#f031" id="id6" role="doc-noteref"><span class="fn-bracket">[</span>5<span class="fn-bracket">]</span></a> This is stored
in the <a class="reference internal" href="glossary.html#term-KV_global"><span class="xref std std-term">KV_global</span></a> in the same way as the preview. Only the owner has
access to the original.</p></li>
</ul>
<p>Thus, a small thumbnail is sent and shared synchronously and
immediately in the front end, and should appear almost instantly to
everybody connected to that room. The larger versions are sent with
eventual-consistency to <a class="reference internal" href="glossary.html#term-KV_global"><span class="xref std std-term">KV_global</span></a>-backed workers as detailed in the
next section.</p>
</section>
<section id="image-dedup-encryption-storage">
<h2>Image Dedup + Encryption + Storage:<a class="headerlink" href="#image-dedup-encryption-storage" title="Permalink to this heading">#</a></h2>
<p>The preview and full image are stored as secure ‘objects.’ An <a class="reference internal" href="glossary.html#term-object"><span class="xref std std-term">object</span></a> is
a specific construct that is particular to this design. The image
information itself is referred to below as ‘data’. Objects are
ultimately stored in <a class="reference internal" href="glossary.html#term-KV_global"><span class="xref std std-term">KV_global</span></a> with the following information:</p>
<ul class="simple" id="object">
<li><p>Their full name (also the <a class="reference internal" href="glossary.html#term-KV_global"><span class="xref std std-term">KV_global</span></a> key). The name is a 512-bit
string constructed in two halves, in two steps. The first half is
the first half of the SHA-512 hash of the original (unencrypted)
contents, the second half is a SHA-256 hash of (final) encrypted
contents.</p></li>
<li><p>Nonce and salt used for the encryption. These can be accessed using
just the first half of the full name (prefix search).</p></li>
<li><p>Contents, which is the encrypted version of the padded form of the
original contents.</p></li>
<li><p>A random 16-byte value, the <a class="reference internal" href="glossary.html#term-verification"><span class="xref std std-term">verification</span></a>.</p></li>
<li><p>A random 48-bit value, the ‘version_id’ which might be used in the
future for version control on files</p></li>
</ul>
<p>Starting from an image (or some other arbitrary data), the above is
accomplished as follows:</p>
<ul class="simple">
<li><p>The client generates the SHA-512 hash based on original data (eg
image). It sends a ‘request to store’ query with the first half of
this hash (the ‘partial name’); it will eventually receive a 12-byte
nonce and a 16-byte salt.</p></li>
<li><p>While waiting for this, the client constructs the ‘shared image’
message, with the <a class="reference internal" href="glossary.html#term-thumbnail"><span class="xref std std-term">thumbnail</span></a>, and forwards data and the first half of
the SHA-512 hash for the compressed data (preview) and original data
(full image), and sends the message.</p></li>
<li><p>Next, the client makes a fetch request (once each for the preview
and full <a class="reference internal" href="glossary.html#term-object"><span class="xref std std-term">object</span></a>) to <a class="reference internal" href="glossary.html#term-KV_global"><span class="xref std std-term">KV_global</span></a> with the first half of the full name
of the file. When the <a class="reference internal" href="glossary.html#term-KV_global"><span class="xref std std-term">KV_global</span></a> receives the ‘request to store’
query, if it’s a new object, then it generates random new nonce and
salt and stores those with the partial name (it doesn’t have the
full name yet); if it’s not a new object, it returns previously
generated values. <a class="footnote-reference brackets" href="#f032" id="id7" role="doc-noteref"><span class="fn-bracket">[</span>6<span class="fn-bracket">]</span></a></p></li>
<li><p>The client prepares the data by padding it to be almost exactly the
size of the nearest exponent of two (2) larger than its actual
(possibly new) size, no less than 128KB (this is the “target size”
mentioned above). Regardless of image, the resulting ‘preview’ thus
ends up appearing to be one of only six different sizes. <a class="footnote-reference brackets" href="#f033" id="id8" role="doc-noteref"><span class="fn-bracket">[</span>7<span class="fn-bracket">]</span></a>
The padding is done using ‘bit’-padding, specifically, the length is
padded only in increments of 128 bits <a class="footnote-reference brackets" href="#f042" id="id9" role="doc-noteref"><span class="fn-bracket">[</span>8<span class="fn-bracket">]</span></a> up to one block
<em>less</em> than the target size - if the target size is on a 128-bit
boundary, a full 128bits are left. ‘Bit’ padding is 0x80 followed by
zeroes. The last thus added block is then truncated by 4 bytes (32
bits), and the length of the original data is stored.</p></li>
<li><p>Next the client encrypts this padded block with a key derived from
the entropy of the second half of the above first hash (using
PBKDF2; 100,000 iterations; SHA-256), with the nonce and salt
returned by the previous ‘request to store’ query.</p></li>
<li><p>Next the client generates a SHA-256 hash based on the encrypted
block (which after encryption should be on a perfect exponent-of-2
boundary) and concatenates with the ‘partial name’ from earlier to
form the ‘full name.’</p></li>
<li><p>The client then makes API calls (once each for the preview and full
<a class="reference internal" href="glossary.html#term-object"><span class="xref std std-term">object</span></a>) to the <a class="reference internal" href="glossary.html#term-KV_local"><span class="xref std std-term">KV_local</span></a> with the final size value (in cleartext) of
the object (preview/full) (rounded up as per below); the room server
inspects and approves the size (or not).</p></li>
<li><p>If approved, the <a class="reference internal" href="glossary.html#term-KV_local"><span class="xref std std-term">KV_local</span></a> makes a fetch request to the
<a class="reference internal" href="glossary.html#term-Ledger-Backend"><span class="xref std std-term">Ledger Backend</span></a> to generate a token for the requested
size. The Ledger Backend returns a token_id. The <a class="reference internal" href="glossary.html#term-KV_local"><span class="xref std std-term">KV_local</span></a>
then encrypts this token_id with the public half of the
<a class="reference internal" href="glossary.html#term-LEDGER_KEY"><span class="xref std std-term">LEDGER_KEY</span></a>. Finally, the <a class="reference internal" href="glossary.html#term-KV_local"><span class="xref std std-term">KV_local</span></a> returns the hash of the
token_id, the encrypted token_id and the hashed roomId as the
storage token back to the client. <a class="footnote-reference brackets" href="#f034" id="id10" role="doc-noteref"><span class="fn-bracket">[</span>9<span class="fn-bracket">]</span></a></p></li>
<li><p>The client then requests this encrypted <a class="reference internal" href="glossary.html#term-object"><span class="xref std std-term">object</span></a> to be stored under
the full name provided, including token approving storage usage; in
reply it will receive the 16-byte <a class="reference internal" href="glossary.html#term-verification"><span class="xref std std-term">verification</span></a>. This encrypted
object is sent asynchronously to the (non-room) worker API.</p></li>
<li><p>The client then generates a control message that contains the full
name of previously shared (<a class="reference internal" href="glossary.html#term-thumbnail"><span class="xref std std-term">thumbnail</span></a> only) image together with the
<a class="reference internal" href="glossary.html#term-verification"><span class="xref std std-term">verification</span></a> as well as (again) the storage token. This control
message would be sent with the same encryption layers as the
original message containing the thumbnail.</p></li>
<li><p>When the backend receives the object, it independently generates the
same second hash based on the encrypted object to verify the
integrity. It then verifies the storage token is valid (i.e. has
been created by <a class="reference internal" href="glossary.html#term-KV_local"><span class="xref std std-term">KV_local</span></a>, hasn’t already been spent and the size of
the object sent to be stored does not exceed the size stored in the
ledger) by making a fetch request to the ledger backend. If valid,
the <a class="reference internal" href="glossary.html#term-KV_global"><span class="xref std std-term">KV_global</span></a> stores three separate entries in the
RECOVERY_NAMESPACE (“D3”) -</p>
<ul>
<li><p><hashed_room_id>_<encrypted_token_id></p></li>
<li><p><hashed_token_id>_<image_id></p></li>
<li><p><image_id>_<hashed_token_id></p></li>
</ul>
</li>
<li><p>If this full name of the object requested to be stored exists in its
storage, then it can discard the received data, and return the
stored <a class="reference internal" href="glossary.html#term-verification"><span class="xref std std-term">verification</span></a>. If it doesn’t, it creates an entry, with the
<em>full</em> name as the key, and saves the encrypted object <a class="footnote-reference brackets" href="#f035" id="id11" role="doc-noteref"><span class="fn-bracket">[</span>10<span class="fn-bracket">]</span></a>
together with nonce and salt, generates a random 48-bit version_id,
generates a random 16-byte verification, and returns that.</p></li>
</ul>
<p>When a client wants to open a preview, the following happens:</p>
<ul class="simple">
<li><p>The <a class="reference internal" href="glossary.html#term-thumbnail"><span class="xref std std-term">thumbnail</span></a> needs to have been matched with a control message with
the full name and the final <a class="reference internal" href="glossary.html#term-verification"><span class="xref std std-term">verification</span></a> returned by a previous
storage.</p></li>
<li><p>The client requests to read the object based on the full name with
the <a class="reference internal" href="glossary.html#term-verification"><span class="xref std std-term">verification</span></a> token.</p></li>
<li><p>When the client receives the (raw) contents, it will also receive
the nonce and salt, it applies the stored (secret) key, and decrypts
and displays the object.</p></li>
<li><p>The backend will only reply if the full name corresponds to an
entry, and the <a class="reference internal" href="glossary.html#term-verification"><span class="xref std std-term">verification</span></a> number matches the stored verification
number..</p></li>
<li><p>An honest client will also confirm that the partial name (and key)
matches a regenerated SHA-512 hash of the decrypted object, and
signal in the UI (such as a red border) and possibly ‘report’ to the
backend that the object is suspect.</p></li>
</ul>
<p>A few comments that follow from the above process.</p>
<ul class="simple">
<li><p>This design retains the ability to de-duplicate any stored binary
data, without having the ability to inspect contents.</p></li>
<li><p>The padding method obscures the precise length of any data,
complicating any brute force attacks against contents of a
compromised server: all stored objects in the same ‘bucket’ of size
would have to be attacked.</p></li>
<li><p>The chained hashing makes it impossible for a client to fake binary
contents: since the second half of the full name is a hash of the
encrypted contents, the backend can check for consistency - the
computational difficulty of generating a file to match a second half
(equivalent to a pre-image attack) is high. A client can obviously
store random data, but that’s immaterial: what’s important is for
the client not to be able to design a hash collision in the full
name.</p></li>
<li><p>A client can obviously avoid duplication by some manner of modifying
the image, even trivially. But this is no different from any other
encrypted storage.</p></li>
<li><p>The client can be dishonest about the first half of the hash, but
that also does not enable any control over hash collisions.</p></li>
<li><p>Dishonesty in a client in constructing the full name will stay with
the image sharing message, with a certain probability of being
detected down the road.</p></li>
</ul>
<p>Regardless of level of misuse, the “insider” privacy model (discussed
at the end of this document) will still be in force. Any participant
to any chat, who has access to decrypting a message with the full key
to the object, can report it, or save the information for future use,
as well as identify if the naming has been tampered with. If we
receive a report on an object with the missing pieces of the key, we
can decrypt the object in storage, and both verify whether it is
correctly reported content, as well as verify integrity, such as
confirming (post facto) that the client was breaking the protocol. At
that point, we can overwrite the object per policy, and re-encrypt
with the provided key information, such that any future access using
the dishonest or manipulated object name will not yield the original,
but just the take-down notice. In other words: the design deliberately
allows for the party operating the server to enforce their content
policy, but does not allow them to pre-emptively scan or review any content.</p>
<p>Another scenario is that a user shares with themselves, or in some
other manner uses the service as a strongly encrypted storage, and
acts maliciously. But this is no different than if they were to simply
encrypt locally and only upload encrypted data to any cloud storage.</p>
</section>
<section id="storage-ledger-server">
<span id="ledgerserver"></span><h2>Storage Ledger Server<a class="headerlink" href="#storage-ledger-server" title="Permalink to this heading">#</a></h2>
<p>A core challenge in providing long-term storage of files <a class="footnote-reference brackets" href="#f044" id="id12" role="doc-noteref"><span class="fn-bracket">[</span>18<span class="fn-bracket">]</span></a>, is how
to accomplish the following (a more formal treatment is <a class="reference internal" href="formal.html#ledger-formal"><span class="std std-ref">here</span></a>):</p>
<ul class="simple">
<li><p>The system should be highly secure and private: contents
at rest should be strongly encrypted, and not (easily)
attributed to whomever uploaded, shared it, and/or
downloaded it.</p></li>
<li><p>Operating expenses. In a multi-user (multi-owner) context,
the costs of respective total storage usage needs to be allocated
to the correct party.</p></li>
<li><p>The system should not allow tracing of who uploaded what (or even,
preferably, when).</p></li>
<li><p>The system should not allow tracing of who is sharing (“re-linking”)
any file.</p></li>
<li><p>It should not be possibly to inquire whether a file
exists on the system, e.g., it should not be possible to determine
if anybody has at any time stored or shared a file.</p></li>
<li><p>The system should be fundamentally capable of de-duplication: in
other words, any file that is uploaded, should not
require duplicate copies in back-end storage. This is essential
for the economics of (highly) scalable cloud storage.</p></li>
<li><p>It should be possible for administrators of a snackabra service
to “take down” any file, that they determine
violates their policy, including in particular the ability to
take down clearly illegal content.</p></li>
<li><p>Any file should end up with a ‘name’ that is globally unique,
so that it will have the same identifier on any snackabra server. <a class="footnote-reference brackets" href="#f045" id="id13" role="doc-noteref"><span class="fn-bracket">[</span>19<span class="fn-bracket">]</span></a></p></li>
</ul>
<p>This becomes a heavily parameterized problem. This has been a major
challenge for us to solve. To our knowledge, nobody has solved this
complete set of requirements.</p>
<p>The design described above accomplishes most of these criteria,
but we have not addressed the cost-tracking (budgeting) aspect.</p>
<p>There is a lot to unpack in this diagram, bear with us:</p>
<p>First, there are four “account balances” involved:</p>
<p><strong>[A]</strong> The budget of the total service.</p>
<p><strong>[B]</strong> The current budget of the room.</p>
<p><strong>[C]</strong> The amount spent in total on storage.</p>
<p>‘[A]’ starts as the total budget for a service - let’s say 100 TB for a
multi-user host. Upon creation of any room, an initial balance of
(say) 1 GB is allocated to the room, ergo 1 GB goes [A] => [B]. When a room
“spends” this, it requests the ledger to transfer it from the room’s “account”
to the global storage [C]. (On a personal server this is much simpler:
the admin simply sets [B] to whatever on a per room basis, and there
is no global [A] nor [C].)</p>
<p>The idea is that we step-wise anonymize parts of the overall transaction (namely:
store an object): generation of identifying information for the object is
kept separate from the path to receive permission to store that amount
of data, for example. You’ll probably need to re-read this section
a few times to see how it all hangs to gether.</p>
<p>Second, there are three important datastores involved, “D1”, “D2”, “D3”
used in this process (not counting the actual storage of data):</p>
<ul class="simple">
<li><p>D1: LEDGER - separate server in multi-owner setup,
internalized to the room in a personal server setup.
These keeps current “account balances” of everything.</p></li>
</ul>
<ul class="simple" id="ledgernamespace">
<li><p>D2: LEDGER_NAMESPACE - tracks spending of approved <a class="reference internal" href="glossary.html#term-TID"><span class="xref std std-term"><TID></span></a>.
To spend storage space, you’re “issued” a kind of token,
which is simply a reference into D2, which in turn
will track if it’s been “cashed” or not.</p></li>
<li><p>D3: RECOVERY_NAMESPACE - tracks details to allow for
anonymous recovery - garbage collection - of revoked
storage etc. This is a bit complex,
but it’s only relevant for multi-owner paid
membership management, for a personal server you
don’t need to worry much about it.</p></li>
</ul>
<p>Now we can untangle the diagram a bit (you can follow along in the code <a class="footnote-reference brackets" href="#f046" id="id14" role="doc-noteref"><span class="fn-bracket">[</span>20<span class="fn-bracket">]</span></a>):</p>
<ol class="arabic simple">
<li><p>The client requests to store a <a class="reference internal" href="glossary.html#term-file"><span class="xref std std-term">file</span></a>.
It generates the first “half” of the name, and sends it
to the storage server. What it needs is help to
“construct” the “true name” for the <a class="reference internal" href="glossary.html#term-object"><span class="xref std std-term">object</span></a>.</p></li>
<li><p>Storage server checks if the data exists already.
Regardless, it replies with the assigned salt and iv
to be used for the corresponding <a class="reference internal" href="glossary.html#term-object"><span class="xref std std-term">object</span></a>.</p></li>
<li><p>The client encrypts the full set of data
and sorts out padding. The blob is ready
to save, and client has the “true name” of
the object (”<a class="reference internal" href="glossary.html#term-FN"><span class="xref std std-term"><FN></span></a>:”).</p></li>
<li><p>The client next requests from the room server
permission to store the amount of data needed. <a class="footnote-reference brackets" href="#f050" id="id15" role="doc-noteref"><span class="fn-bracket">[</span>21<span class="fn-bracket">]</span></a></p></li>
<li><p>The room checks if it has budget:
it asks the Ledger to “spend” storage bytes:
it generates a transaction of class
“token”, with properties “size, random id, used”,
and asks the ledger for an identifier (”<a class="reference internal" href="glossary.html#term-TID"><span class="xref std std-term"><TID></span></a>”.</p></li>
<li><p>The ledger spends ‘size’ from the room’s
budget ([B]->[C]), and generates <a class="reference internal" href="glossary.html#term-TID"><span class="xref std std-term"><TID></span></a>.
The key details are the approved
size, and if it’s been “spent” yet.
This is stored with a one-way
hash in ‘D2’ - thus “h(<TID>)”
If all is well and good,
responds with <TID>.</p></li>
<li><p>On a personal server, step 5/6 is done
locally instead, self-generating a <a class="reference internal" href="glossary.html#term-TID"><span class="xref std std-term"><TID></span></a>.</p></li>
<li><p>The Room now creates a special object,
sort of a “token”: <code class="docutils literal notranslate"><span class="pre"><hash(<TID>),</span> <span class="pre">R(<TID>),</span> <span class="pre">R(h<TID>)></span></code>.
This bundle is encrypted (and padded),
and returned with h(<TID>) to the client.</p></li>
<li><p>The client is now empowered to actually request
the store to be done. It sends the “magical
token” along with the blob of data.</p></li>
<li><p>Storage now checks with the Ledger (‘D2’):
the hash of the <a class="reference internal" href="glossary.html#term-TID"><span class="xref std std-term"><TID></span></a> (“h(<TID>)”), checks
that the ‘size’ is correct, and
“spends” it (finalizing [B]->[C]).</p></li>
<li><p>Storage now updates ‘D3’ with some special info:
<code class="docutils literal notranslate"><span class="pre">h(R(<room>)_R(<TID>),</span> <span class="pre">h(<TID>)_<FN>,</span> <span class="pre"><FN>_h(<TID>)</span></code>
for offline recovery / garbage collection.
(you can see the keys stowed away
by <code class="docutils literal notranslate"><span class="pre">handleStoredata()</span></code> <a class="footnote-reference brackets" href="#f048" id="id16" role="doc-noteref"><span class="fn-bracket">[</span>23<span class="fn-bracket">]</span></a>).
The <code class="docutils literal notranslate"><span class="pre">R()</span></code> notation shows it has been encrypted
by the <a class="reference internal" href="glossary.html#term-LEDGER_KEY"><span class="xref std std-term">LEDGER_KEY</span></a> <a class="footnote-reference brackets" href="#f049" id="id17" role="doc-noteref"><span class="fn-bracket">[</span>24<span class="fn-bracket">]</span></a> .</p></li>
<li><p>Finally, the storage server will generate a random
<a class="reference internal" href="glossary.html#term-verification"><span class="xref std std-term">verification</span></a> number - unique for every <a class="reference internal" href="glossary.html#term-FN"><span class="xref std std-term"><FN></span></a>.
When the client receives it, it can <em>finally</em>
construct the control message with all
the details about the object, which
altogether we loosely refer to as
the <a class="reference internal" href="glossary.html#term-manifest"><span class="xref std std-term">manifest</span></a>. This is sent to
all chat room participants.</p></li>
</ol>
<p>Various things to note:</p>
<ul class="simple">
<li><p>The room server manages it’s own “budget”;
you can think of it as a “bucket” or almost
as a directory. On a personal server that
you run yourself, you can modify this
budget directly for any room. On a multi-user
service, there’s a separate “Ledger Server”
which manages storage budgets and accounting
across all accounts and users.</p></li>
<li><p>A new room is initialized with an initial
total budget - current default is 1 GB.
It can “independently” authorize messages
and files up to that total amount.
Once that’s exceeded, then on a personal
server you need to directly change the
budget using the <a class="reference internal" href="glossary.html#term-CLI"><span class="xref std std-term">CLI</span></a>, on a multi-user
server it needs to request more allocation
from the <a class="reference internal" href="glossary.html#term-Ledger-Backend"><span class="xref std std-term">Ledger Backend</span></a>.</p></li>
<li><p>Note that in around step [5], neither the Room
nor the Ledger actually need to know <em>what</em>
object is being stored, just it’s size
(which is padded to specific set of size
options to further obfuscate correlation
between specific objects and coresponding
storage budget “spend”).</p></li>
<li><p>You can think of part of the transactions
around <a class="reference internal" href="glossary.html#term-TID"><span class="xref std std-term"><TID></span></a> as a sort of local cryptocurrency,
a “token” in the old-fashioned sense:
it’s a thing that can be “printed” by
asking the Ledger to approve [B]->[C],
and cashed in by “spending” it with
the storage server ([C]).</p></li>
<li><p>The <a class="reference internal" href="glossary.html#term-manifest"><span class="xref std std-term">manifest</span></a> can be used anywhere:
command line, other clients, etc. There’s two
versions of it - one that is share with
everybody, and one that includes the additional
bit of information that enables
future revocation of storage budget(s). <a class="footnote-reference brackets" href="#f047" id="id18" role="doc-noteref"><span class="fn-bracket">[</span>22<span class="fn-bracket">]</span></a></p></li>
</ul>
</section>
<section id="storage-revocation">
<h2>Storage Revocation<a class="headerlink" href="#storage-revocation" title="Permalink to this heading">#</a></h2>
<p>[To be Written]</p>
</section>
<section id="group-security">
<h2>Group Security<a class="headerlink" href="#group-security" title="Permalink to this heading">#</a></h2>
<p>For a number of security-oriented messaging apps, the “group” aspect
has been a challenge. See for example:</p>
<ul class="simple">
<li><p>More is Less: On the End-to-End Security of Group Chats in Signal,
WhatsApp, and Threema
<a class="reference external" href="https://eprint.iacr.org/2017/713.pdf">https://eprint.iacr.org/2017/713.pdf</a></p></li>
<li><p>(A number of Wired articles, to be added)</p></li>
<li><p>Attack of the Week: Group Messaging in WhatsApp and Signal (blog) -
<a class="reference external" href="https://blog.cryptographyengineering.com/2018/01/10/attack-of-the-week-group-messaging-in-whatsapp-and-signal/">https://blog.cryptographyengineering.com/2018/01/10/attack-of-the-week-group-messaging-in-whatsapp-and-signal/</a></p></li>
</ul>
<p>The Signal app and protocol being the most common, we’ll comment in
relation to it’s design. The group chat capability was design while
moxie0 was still with Open Whisper. <a class="footnote-reference brackets" href="#f036" id="id19" role="doc-noteref"><span class="fn-bracket">[</span>11<span class="fn-bracket">]</span></a> Some issues:</p>
<ul class="simple">
<li><p>The “every client broadcasts” nature of group communication is still
going through the Signal servers; this leaves enough metadata
available (whether collected or not) to easily reconstruct group
membership, and in addition, because of the authentication model,
all the phone numbers of participants. Even though the server
“notionally” doesn’t know group membership (there is no DB that
explicitly tracks it), the data necessary is unavoidably generated
in the normal course of the service. <a class="footnote-reference brackets" href="#f037" id="id20" role="doc-noteref"><span class="fn-bracket">[</span>12<span class="fn-bracket">]</span></a> Anonymity bestowed
in principle by the broadcast model does not in fact exist if the
service has a monopoly on delivering the messages.</p>
<ul>
<li><p>Our design in contrast unashamedly sets up a websocket addressable
worker to receive and re-transmit messages. This in fact puts
control at the hands of the client with respect to how it connects
to the server - it can “pop up” from a VPN or ToR or any manner
that allows it to connect.</p></li>
</ul>
</li>
<li><p>… next point was about random number generation … took a while to
figure out actually entropy in Signal groupId … in the end I think
their 2014 generation was just 31 bits, namely Java’s max integer
value (signed 32 bit) .. new system is 128, but it’s generated in
the client, so that’s not super great, more on that soon …</p></li>
</ul>
</section>
<section id="binary-serialized-format">
<h2>Binary Serialized Format<a class="headerlink" href="#binary-serialized-format" title="Permalink to this heading">#</a></h2>
<p>Images are generally stored in a binary serialized format. We may also
use this format for binary protocols (web socket), where a more
correct term might be “wire transfer format.”</p>
<p>For storage, data in key-value stores in various Cloudflare services
typically support either a string (JSON) object, or a purely binary
object. <a class="footnote-reference brackets" href="#f038" id="id21" role="doc-noteref"><span class="fn-bracket">[</span>13<span class="fn-bracket">]</span></a> If we were to Base64 encode these, it would cause a
8/6 (1.33) factor expansion, which matters less for a <a class="reference internal" href="glossary.html#term-thumbnail"><span class="xref std std-term">thumbnail</span></a>, but on
full sized images starts adding up. We’ve therefore designed <a class="footnote-reference brackets" href="#f039" id="id22" role="doc-noteref"><span class="fn-bracket">[</span>14<span class="fn-bracket">]</span></a>
a simple manner in which to store a more or less arbitrary JS data
structure in binary format.</p>
<p>Format: The first four bytes (32 bits) stores the size of the
“metadata”, which in turn is a (JSON.stringified) dictionary in the
form of “{ key1: dataSize1, key2: dataSize2, …}”. The data parts are
all Uint8Array objects of arbitrary size, as given by dataSize1 etc,
which are all assembled (concatenated) and stored as a single binary
blob. The reverse (extract) operation first extracts the size of the
metadata, allowing JSON.parse() to run against a well-formed object,
and then assembles back into the dictionary the same keys, but with
matching binary objects. This binary format does not limit the size of
objects that can be included in any practical sense. <a class="footnote-reference brackets" href="#f040" id="id23" role="doc-noteref"><span class="fn-bracket">[</span>15<span class="fn-bracket">]</span></a> The net
effect is that a JS dictionary of the form ‘{key1: arrayBuffer1,
key2:arrayBuffer2, …}’ can be assembled and extracted, essentially a
pickler that works for our specific use case.</p>
</section>
<section id="static-room-ui-local-client">
<h2>Static Room UI (“Local Client”)<a class="headerlink" href="#static-room-ui-local-client" title="Permalink to this heading">#</a></h2>
<p>In the plan and the design, but not finalized, is the intent to
provide an open source, static, single-html-page web application
version of the client. We refer to this as a “local client” and also
“static client”. This would allow any user to join any room by loading
from local storage a static page, then loading a previously exported
set of keys, and join any rooms detailed in those keys.</p>
<p>Also to be implemented is support for full export of all messages, in
a manner that can be synchronized (merged) upon joining any other
server (see <a class="reference internal" href="install.html#personal-server"><span class="std std-ref">Stand-Alone Server</span></a>). UPDATE: this
now works!</p>
<p>An important use case is for participants to always be able to join a
room (starting with the first time) by copy-pasting the room name and
server address into the static client, and thereby have greater
confidence that the keys they’re using were truly generated locally.</p>
<p>A perhaps more obscure use case is the option for participants in a
room to use local clients as a part of the strict locking-down
process, to account for any possible combinations of compromised
clients amongst any of the participants. This process is currently
under design. UPDATE: this is almost fully in place!</p>
<p>A simpler, likely more common, use case is a room with a small number
of participants, where the owner has locked the room, and all
participants including Owner have exported their respective sets of
keys. Then, they should <em>all</em> be able to rejoin the room, from
respective systems, all loading from static files.</p>
<p>Below is a demonstration that current design works for this usage
model. It shows connecting straight to a chat room endpoint from the
command line, using ‘curl’ for API endpoints and ‘wscat’ for websocket
connection. Ergo, users can script their own tools. We believe this
approach remedies many of the historically observed problems with any
web-based UI. <a class="footnote-reference brackets" href="#f041" id="id24" role="doc-noteref"><span class="fn-bracket">[</span>16<span class="fn-bracket">]</span></a> was largely addressed by development of the
subtle.crypto standard )</p>
<img alt="_images/curl_example_01.png" src="_images/curl_example_01.png" />
</section>
<section id="command-line-tools">
<span id="command-line"></span><h2>Command Line Tools<a class="headerlink" href="#command-line-tools" title="Permalink to this heading">#</a></h2>
<p>To Be Written.</p>
</section>
<section id="rooms-technical-details">
<span id="id25"></span><h2>Rooms: Technical Details<a class="headerlink" href="#rooms-technical-details" title="Permalink to this heading">#</a></h2>
<p>All communication is centered around a <a class="reference internal" href="glossary.html#term-Room"><span class="xref std std-term">Room</span></a>.</p>
<p>The <a class="reference internal" href="glossary.html#term-Room-Name"><span class="xref std std-term">Room Name</span></a> is a 48-byte <a class="footnote-reference brackets" href="#f011" id="id26" role="doc-noteref"><span class="fn-bracket">[</span>26<span class="fn-bracket">]</span></a> URI <a class="footnote-reference brackets" href="#f012" id="id27" role="doc-noteref"><span class="fn-bracket">[</span>27<span class="fn-bracket">]</span></a> encoded in 64
characters of b64 <a class="footnote-reference brackets" href="#f013" id="id28" role="doc-noteref"><span class="fn-bracket">[</span>28<span class="fn-bracket">]</span></a>. This name is globally unique <a class="footnote-reference brackets" href="#f014" id="id29" role="doc-noteref"><span class="fn-bracket">[</span>29<span class="fn-bracket">]</span></a>, including
across servers.</p>
<p>An example would be:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">Raih2xfY6D8aKVIlkIeDLIbSpt0qNmU2mUTXYiJQoNSU</span><span class="o">-</span><span class="n">SgyTLC0FReui0OhX1Q8</span>
</pre></div>
</div>
<p>We variously refer to this identifier as “<room>”, “<roomId>”, “room
identifier”, or “room name”.</p>
<p>This URI is generated using (Python) <code class="docutils literal notranslate"><span class="pre">secrets.token_urlsafe</span></code>
<a class="footnote-reference brackets" href="#f010" id="id30" role="doc-noteref"><span class="fn-bracket">[</span>25<span class="fn-bracket">]</span></a> in the <a class="reference internal" href="#command-line"><span class="std std-ref">command line tools</span></a>.
(UPDATE: this is changing to be derived from a generated owner public key.)</p>
<p>The location (server) where a room is created is called the
<a class="reference internal" href="glossary.html#term-Origin-Server"><span class="xref std std-term">Origin Server</span></a>. Note that the identity of the origin server is
in no manner reflected in the name of a room. <a class="footnote-reference brackets" href="#f015" id="id31" role="doc-noteref"><span class="fn-bracket">[</span>30<span class="fn-bracket">]</span></a></p>
<p>The same <room> identifier is used on different servers: if you
have a staging environment for your front end code, for example,
but share backend databases.</p>
<p>The main MI service hosts rooms here:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">https</span><span class="o">//</span><span class="n">s</span><span class="o">.</span><span class="n">privacy</span><span class="o">.</span><span class="n">app</span><span class="o">/<</span><span class="n">room</span><span class="o">></span>
</pre></div>
</div>
<p>Or you can run a <a class="reference internal" href="glossary.html#term-Personal-Room-Server"><span class="xref std std-term">Personal Room Server</span></a> if you like.</p>
<p id="owner-definition">An <a class="reference internal" href="glossary.html#term-Owner"><span class="xref std std-term">Owner</span></a> is a verified user, authenticated in some manner;
for a public server typically through an <a class="reference internal" href="glossary.html#term-SSO"><span class="xref std std-term">SSO</span></a></p>
<p>Our baseline use case is an MI Member
<a class="reference external" href="https://privacy.app">https://privacy.app</a> where owners are authenticated with an MI
membership number, an assigned Yubico hardware security key, and PIN
code. However, the intent of this design is that it should be
stand-alone from any particulars of authentication, but it does assume
the presence of some such authentication.
For example, for a <a class="reference internal" href="glossary.html#term-Personal-Room-Server"><span class="xref std std-term">Personal Room Server</span></a>, it’s a server secret
that the admin of the site has.</p>
<p>Room identifiers are unique, global, and persistent. <a class="footnote-reference brackets" href="#f016" id="id32" role="doc-noteref"><span class="fn-bracket">[</span>31<span class="fn-bracket">]</span></a> They are
transportable in a natural way: no matter how many people host a
service using this design, the probability of any new rooms from any
two services colliding is practically zero. Both our current web app and iOS clients
support export / import of conversations. See our discussion on
<a class="reference internal" href="discussion.html#micro-federation"><span class="std std-ref">micro-federation</span></a>.</p>
<p>The <a class="reference internal" href="#command-line"><span class="std std-ref">command line tools</span></a> include support
for generating “business cards” with QR codes for a room, the
idea being that you can share these with people you meet, instead
of <a class="reference internal" href="glossary.html#term-PII"><span class="xref std std-term">PII</span></a> like emails or phone numbers.</p>
</section>
<section id="room-and-user-keys">
<span id="roomuserkeydetails"></span><h2>Room and User Keys<a class="headerlink" href="#room-and-user-keys" title="Permalink to this heading">#</a></h2>
<p>The two basic key <em>types</em> are:</p>
<ul class="simple">
<li><p>Public keys:</p>
<ul>
<li><p>RSA public-key pair, using SECG curve secp3841 (aka NIST P-384)
This pair is generated by <code class="xref py py-func docutils literal notranslate"><span class="pre">snackabra.crypto.gen_p384_pair()</span></code>.</p></li>
<li><p>Python library: <code class="docutils literal notranslate"><span class="pre">ec.generate_private_key(ec.SECP384R1())</span></code> <a class="footnote-reference brackets" href="#f021" id="id33" role="doc-noteref"><span class="fn-bracket">[</span>36<span class="fn-bracket">]</span></a></p></li>
<li><p>NSS (Firefox) currently supports NIST curves P-256, P-384,
P-521. However, apparently P-521 is not widely supported <a class="footnote-reference brackets" href="#f017" id="id34" role="doc-noteref"><span class="fn-bracket">[</span>32<span class="fn-bracket">]</span></a> as it is
not part of “NIST Suite B”. <a class="footnote-reference brackets" href="#f018" id="id35" role="doc-noteref"><span class="fn-bracket">[</span>33<span class="fn-bracket">]</span></a></p></li>
<li><p>ECDH is used to agree on a key <a class="footnote-reference brackets" href="#f019" id="id36" role="doc-noteref"><span class="fn-bracket">[</span>34<span class="fn-bracket">]</span></a>
between two parties <a class="footnote-reference brackets" href="#f020" id="id37" role="doc-noteref"><span class="fn-bracket">[</span>35<span class="fn-bracket">]</span></a>.</p></li>
</ul>
</li>
<li><p>Encryption keys:</p>
<ul>
<li><p>AES 256-bit (A256GCM) symmetric key</p></li>
<li><p>Currently generated using jwk <a class="footnote-reference brackets" href="#f022" id="id38" role="doc-noteref"><span class="fn-bracket">[</span>37<span class="fn-bracket">]</span></a>
library (it’s just 256
random bits). Generated by <code class="xref py py-func docutils literal notranslate"><span class="pre">snackabra.crypto.gen_aes_key_jwk()</span></code>.</p></li>
</ul>
</li>
</ul>
<p>Key instances, the following are pre-generated and stored by the <a class="reference internal" href="glossary.html#term-SSO"><span class="xref std std-term">SSO</span></a>:</p>
<ul class="simple">
<li><p><room>_ownerKey - <em>[Public key pair]</em></p>
<ul>
<li><p>public room key, used to claim ownership of the room, and to verify anything signed by the Owner</p></li>
<li><p><em>(existence of this does not imply that the Durable Object for the
room has been created, but it means it will be created when
accessed)</em></p></li>
<li><p>private half of this is stored in Owner (SSO) data only</p></li>
<li><p>When owner joins room, private half stored in Owner’s
local_storage as <room>_room</p></li>
<li><p>The public half of this key is also stored in the localstorage</p></li>
<li><p>Owner can secure key management by generating a new key pair and
saving the public half as a new entry in the KV_global as
<room>_ownerKey<ts>, where ts is the timestamp when they updated
the key. The private half of this newly generated pair will be
saved only in their localstorage. <a class="footnote-reference brackets" href="#f023" id="id39" role="doc-noteref"><span class="fn-bracket">[</span>38<span class="fn-bracket">]</span></a></p></li>
</ul>
</li>
<li><p><room>_signKey - <em>[Public key pair]</em></p>
<ul>
<li><p>The private room signing key, used by visitors to sign back and
forth (or more accurately, to derive a unique signing key).</p></li>
<li><p>All participants have access to this key (both halves).</p></li>
</ul>
</li>
<li><p><room>_encryptionKey - <em>[Encryption key]</em></p>
<ul>
<li><p>The durable object keeps an encryption key, used for end to end
encryption <a class="footnote-reference brackets" href="#f024" id="id40" role="doc-noteref"><span class="fn-bracket">[</span>39<span class="fn-bracket">]</span></a> unless the Owner has taken
control of their key management</p></li>
</ul>
</li>
<li><p><room>_authorizationKey - <em>[Public key pair]</em></p>
<ul>
<li><p>used to prove ownership of a room (SSO backend->Chat CF backend)</p></li>
<li><p>Only SSO backend has private key; SSO can verify it’s authority to
the Chat CF backend by signing a cookie (or in future, have other
admin APIs)</p></li>
</ul>
</li>
</ul>
<p>The following keys may eventually populate the room (KV_local):</p>
<ul class="simple">
<li><p><room>_lockedKey - <em>[Public key pair]</em></p>
<ul>
<li><p>Generated if the owner “restricts” the room and stored in local
storage of accepted guests</p></li>
<li><p>Used to send an end-to-end encrypted message in restricted rooms.</p></li>
</ul>
</li>