You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: model-writing-tutorial.md
+17-3Lines changed: 17 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -166,9 +166,9 @@ If you're training a custom model with your custom dataset, keep in mind that yo
166
166
167
167
Now the model is fully quantized. Specifically, `mnist_.json` and `mnist_.params` are your quantized models, stored under `~/mrt_model` (assuming you have not altered the path in the `python/mrt/conf.py` file). Create a separate folder named `data` and rename your `mnist_.json` to `symbol` and your `mnist_.params` to `params`. Run `ls -l` to check your operating system is not hiding the file name extension, as the full file names need to be correct for successful upload.
168
168
169
-
We're now ready to upload them! Like most PoW blockchains, transaction data are broadcast out, received by different full nodes and relayed to propagate the entire network; miners then include these transactions in a block later mined into the blockchain. In our case, model file (which is mostly matrices of weights) is the transaction data; to upload this model file, we broadcast it out by seeding a torrent (which is a file including metadata about the model file that enables its reception and relay by other full nodes).
169
+
We're now ready to upload them! Like most PoW blockchains, transaction data are broadcast out, received by different full nodes and relayed to propagate the entire network; miners then include these transactions in a block later mined into the blockchain. In our case, model file (which is mostly matrices of weights) is the transaction data; to upload this model file, we broadcast it out by seeding it with a torrent file (which is a file including metadata about the model file that enables its reception and relay by other full nodes).
170
170
171
-
Let's first go the [TestNet Cerebro Explorer](https://cerebro.test.cortexlabs.ai/) to generate a torrent file for our model file. (When you deploy to MainNet, you need to go to the MainNet Cerebro Explorer [Cerebro Explorer](https://cerebro.cortexlabs.ai/))
171
+
Let's first go the [TestNet Cerebro Explorer](https://cerebro.test.cortexlabs.ai/) to generate a torrent file for our model file, which we will later use to broadcast (upload) the model file to the network. (When you deploy to MainNet, you need to go to the MainNet Cerebro Explorer [Cerebro Explorer](https://cerebro.cortexlabs.ai/))
172
172
173
173
In the menu bar at the top, find "upload" under "AI Contract"
174
174
@@ -190,7 +190,21 @@ Now make sure you've signed in to your TestNet wallet and don't have any other t
190
190
191
191
Type in the model description, click `save` and then hit `upload`. A new transaction confirmation window will pop out. (If it doesn't, it's probably due to the two reasons above. Close your browser and try again). Click confirm.
192
192
193
-
We're not done yet - all the upload button does is to generate a torrent file which then you will need in order to broadcast out by seeding it along with your model file.
193
+
We're not done yet - recall that we're only generating a torrent file here, which includes metadata about the model file that enables its broadcast to other full nodes.
194
+
195
+
To finish the upload, we now need to seed(upload/broadcast) the model file along with the torrent file (model file's metadata). In this example, we will use qBittorrent to do the seeding, but feel free to use whichever BitTorrent client you prefer.
196
+
197
+
After confirming the upload transaction on Cerebro above, a torrent file is generated and downloaded. We rename it as "torrent". We create a new folder named "test" in which we put the "torrent" file alongside the "data" folder containing the model files that we created above. The resulting directory should have this structure:
198
+
199
+

200
+
201
+
Next, we open qBittorrent and drag this "test" folder to the window.
202
+
203
+

204
+
205
+
Configure as above (usually by default) and click "Create Torrent" and let it broadcast for ~6 hours to ensure the file propagate to the entire network.
206
+
207
+
To prevent spams, the Cortex protocol requires large model file uploaders to manually push the progress by consuming gas. We need to go to the transaction page to manually send a few more transactions - otherwise, the full nodes will ignore our seed instead of relaying it.
0 commit comments