Root filesystem is low on space

[SyntaxColoring]

This issue follows up on #143 by aggregating further ideas to free up space on the filesystem. (Other Opentrons folks, feel free to edit this list.)

---

• This PR added git, curl, and wget.

  • Do we really need git? Do we really need both curl and wget, or can we just pick one?
  • "i do not remember at all why we added git, let's not"
  • Pretty sure git was a user request - @mcous

• Busybox has its own implementations of curl and/or wget (can't remember which, exactly). So we have two implementations installed. Can we get rid of one? Are we accidentally doing this with any other commands?

  • "if busybox has a curl let's use that; wget can still be useful, but i don't really care whether it happens or not"
  • Looks like something similar for i2ctools and e2fsprogs if I'm reading this right?

• Don't need busybox crond and crontab because we have systemd for that

  • Maybe same with halt, etc.

• Should we strip test files from Python packages, sort of automating the workaround in bug: robot's filesystem is out of space,make push sometimes fails opentrons#8184?

  • "Stripping as much non-critical content from the python stuff is absolutely a good idea. We may also find it useful to install them as gzipped wheels to save disk space."
  • Test files from pandas and numpy stripped by Strip tests from numpy and pandas #150

• Strip out all the bluetooth stuff. this was a temporary experiment that at this point I [@sfoster1] think is not going to happen

  • Removed in Remove bluetooth #149

• i [@sfoster1] doubt we need screen or tmux but they're also tiny so whatever

  • We should not remove these; power users use them because long-running SSH is unreliable - @mcous

---

See also Opentrons/opentrons#8184, which has some investigation for what counts as "full."

[arogozhnikov]

git is very useful for users like me - so that code of utilities working on opentrons is taken/stored at git repo, not just laying on the robot

[SyntaxColoring]

@arogozhnikov Thanks for your feedback.

Can you tell us a little more about how you're using Git on the robot? Are you doing something like this?

# SSH into the robot.
$ ssh -i $MY_SSH_KEY root@$MY_ROBOT_IP_ADDRESS

# Now that we're on the robot, use Git to download files from a remote repository.
cd /data/user_storage/my_repo
git pull

If so, could you replace that with something like this?

# On your PC, use Git to get the latest files.
cd ~/my_repo
git pull

# Delete any old files on the robot.
ssh -i $MY_SSH_KEY root@$MY_ROBOT_IP_ADDRESS rm -rf /data/user_storage/my_repo
# Use scp to push the latest files from your PC to the robot.
scp -i $MY_SSH_KEY -r ~/my_repo root@$MY_ROBOT_IP_ADDRESS:/data/user_storage/my_repo

[arogozhnikov]

@SyntaxColoring it is a two-way sync (notebooks from robot stored to the repo)

[SyntaxColoring]

Would rsync work instead of git?

The robot doesn't currently have rsync installed. I don't know if it would actually be smaller than git, but it might be.

[arogozhnikov]

I think rsync would do.

But

May I ask why you say 'low on space'?

E.g. it looks I have 16Gb on robot and I'm happy to spend 3-4Gb to get more tools rather than suffering from sh (that doesn't even support readline)... or from missing nano/git ... or missing ssh client. Example: your support recommends using reverse tunnel, but there is no ssh client on robot 🤷

[SyntaxColoring]

May I ask why you say 'low on space'?

E.g. it looks I have 16Gb on robot [...]

It has to do with the way the SD card is partitioned. The whole card does indeed have plenty of space. That's the 16 GB you're seeing. But the root partition only has something like 312 MB.

Our robot software updates work by atomically overwriting the whole root partition, basically leaving the rest of the SD card alone. So all of our executables and libraries live on the root partition. This can and has filled up its 312 MB beyond the conventional 95% limit, to the point where it couldn't even fit a few additional Python packages.

I'm happy to spend 3-4Gb to get more tools rather than suffering from sh (that doesn't even support readline)... or from missing nano/git ... or missing ssh client. Example: your support recommends using reverse tunnel, but there is no ssh client on robot 🤷

Agreed. It's just a matter of unlocking the rest of the 16 GB.

We're pursuing an approach like this:

• Pare down the contents of the 312 MB root partition so it's just the minimal core stuff that Opentrons needs.
• Give users a suggested way to install whatever they want outside the root partition, and provide pre-built packages for common Python packages that are otherwise difficult to configure properly for our system.

[arogozhnikov]

Let's see how it goes. Userspace binaries should work (though registry with binaries would help).

As for python packages - I've tried scipy / pillow, but neither worked (unsurprisingly). Not that it I need it right now, but I have to lower my expectations from 'I have a machine that can orchestrate things in the lab with flexibility of python' to 'I shouldn't expect more than liquid handling'.