Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use sqlite #124

Draft
wants to merge 31 commits into
base: canon
Choose a base branch
from
Draft

Use sqlite #124

wants to merge 31 commits into from

Commits on Apr 19, 2024

  1. fix(shell.nix): explicitly use rust tools from rustup

    It’s just a better environment that way … sigh
    Profpatsch committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    6046cac View commit details
    Browse the repository at this point in the history
  2. refact(ops/stream_events): directly construct json value

    Instead of having complex generic types internally that we map over,
    just create the json in a macro.
    
    Removes `ProjectAdded` from `Reason`, because it was not used
    anywhere. Changes the daemon format, so make sure to run the daemon
    with this new lorri version or you get serialization errors.
    Profpatsch committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    adfc561 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    ba27237 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    6630b36 View commit details
    Browse the repository at this point in the history
  5. refact(ops/shell): Rephrase Arc code in terms of thread::race()

    This new function makes it easy to race two threads, instead of
    syncing on shared state. This also simplifies the duration code, since
    all we are doing is waiting on timeouts and the stop channel.
    Profpatsch committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    d6f7d49 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    cfe2f6c View commit details
    Browse the repository at this point in the history
  7. Configuration menu
    Copy the full SHA
    e6aad7a View commit details
    Browse the repository at this point in the history
  8. refact(ops/gc): generate json via macro

    Instead of using serde, defining a clear API boundary (preventing
    breakage on refactors).
    Profpatsch committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    47b1f91 View commit details
    Browse the repository at this point in the history
  9. refact(*): Remove ContentAddressable from Project & pass manually

    We only pass the `cas` through the `Project` constructor, never
    actually using it in the project `impl`!. This means we don’t need to
    burden `Project` with the cas, since it’s only used by the builder in
    the end. Insead, put it on the build loop for now.
    
    Funnily enough, `StartUserShell` does not need the project init
    anymore, cause it only wanted the `cas` from it.
    Profpatsch committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    2208ddc View commit details
    Browse the repository at this point in the history
  10. refact(ops/op_gc): don’t quit on first error for non-json

    Collect errors and continue deleting when json is not on.
    Profpatsch committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    3258e3e View commit details
    Browse the repository at this point in the history
  11. Configuration menu
    Copy the full SHA
    e001762 View commit details
    Browse the repository at this point in the history
  12. Configuration menu
    Copy the full SHA
    9249133 View commit details
    Browse the repository at this point in the history
  13. Configuration menu
    Copy the full SHA
    dd33b50 View commit details
    Browse the repository at this point in the history
  14. feat(sqlite): start importing the gcroots data

    This is super crappy at the moment, but we want to ideally save most
    of the data in the database instead of indirectly in the filesystem.
    This gives us better leverage for storing other fun things, like which
    files we last watched for a project, or how long changes to the files
    took.
    Profpatsch committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    c3a4945 View commit details
    Browse the repository at this point in the history
  15. fix: allow dead code for structs we only use for lifetimes

    These two structs are only used for preventing automatic cleaning of
    these resources, so their content is never used. Newer rust will warn
    about that for structs as well.
    Profpatsch committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    846e3fa View commit details
    Browse the repository at this point in the history
  16. Configuration menu
    Copy the full SHA
    0fda515 View commit details
    Browse the repository at this point in the history
  17. Configuration menu
    Copy the full SHA
    43b3c7d View commit details
    Browse the repository at this point in the history
  18. refact(builder): hide shell_gc_path, add better interface

    Yay, information hiding!
    Profpatsch committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    93902ba View commit details
    Browse the repository at this point in the history
  19. fix(thread): slightly increase message timeout

    The 5ms difference would sometimes trigger this channel first, so
    let’s give it 20ms difference.
    Profpatsch committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    486f474 View commit details
    Browse the repository at this point in the history
  20. feat(ops/shell): print a nice warning for unsupported shells

    We’ve only really used `shell` for the first few months, then expected
    people to switch to `direnv`. So we only have real support for `bash`
    and `zsh`. We should note that.
    
    By not shadowing `project` in the main function, I uncovered that we
    missed out on using the nested logger for `Info`.
    Profpatsch committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    2c1cb9b View commit details
    Browse the repository at this point in the history
  21. refact(tests/gc): repair the gc integration test

    What an amazing piece of code.
    
    Apparently it wasn’t even worth the work to remove all `dbg!` macros
    before submitting it upstream.
    
    Get rid of adding `stderr` to the checked output.
    
    I have a very trigger-happy finger that was very close to simply
    deleting this whole file without a second thought, but I suppose
    having some kind of integration test for this is at least a little
    useful.
    Profpatsch committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    483683d View commit details
    Browse the repository at this point in the history
  22. refact(ops,project): move list_roots to project

    Every thing related to roots should be in that module.
    Profpatsch committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    ecb1ee9 View commit details
    Browse the repository at this point in the history
  23. Configuration menu
    Copy the full SHA
    3179439 View commit details
    Browse the repository at this point in the history
  24. Configuration menu
    Copy the full SHA
    7ca377d View commit details
    Browse the repository at this point in the history
  25. Configuration menu
    Copy the full SHA
    edecd77 View commit details
    Browse the repository at this point in the history
  26. fix(project): create project for each gc root

    Instead of manually listing the files and doing some ad-hoc reading of
    files to list every gc dir, use our existing project setup to fetch
    this information.
    
    This switches around the logic a bit, because I couldn’t be bothered
    to keep it all the same. Especially around the liveness of nix files.
    Basically there might be a nix_file symlink file, but there might also
    not. And if it’s there, there might be a nix_file behind it or not.
    Same for the timestamp and the gc root symlink.
    Profpatsch committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    98696a6 View commit details
    Browse the repository at this point in the history
  27. refact(op/gc,project): move gc_deletion into project

    We create a function which consumes the project and deletes its GC
    root.
    Profpatsch committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    525eb58 View commit details
    Browse the repository at this point in the history

Commits on Apr 20, 2024

  1. Configuration menu
    Copy the full SHA
    d8e2514 View commit details
    Browse the repository at this point in the history
  2. feat(ops/gc): print the “time ago” down to the second

    Previously it would say “0 days” for anything that’s younger than one
    day, now it will use the smallest unit that’s not 0.
    Profpatsch committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    eaf9cb9 View commit details
    Browse the repository at this point in the history
  3. feat(project): start saving nix_file in database

    In addition to updating the backlink to nix_file, we also update the gc_roots
    table in our database to contain the same info.
    
    The next step is to read everything from the database instead of from the
    symlinks.
    
    Currently, the gc_roots table is deleted at every start (to make changes to the
    schema easier), after everything works through the database, we have to set up a
    one-time migration (and write a test for it …) and clean up the symlinks.
    Profpatsch committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    d44b6fc View commit details
    Browse the repository at this point in the history
  4. refact(project): add connection to struct

    This was a lot more complicated than I expected, because you cannot share a
    rusqlite `Connection` object across unwrap boundaries (you can share it across
    threads, but our `Async` does automatic unwrapping …).
    
    So what I do here is that whenever we want to clone a `Project`, I create
    something I call “skeleton”, which is a clone of the project without the
    connection object (just the sqlite file path), then on the other side of the
    boundary I turn the skeleton back into a Project by connecting to the database
    again. I think this is quite an okay solution.
    Profpatsch committed Apr 20, 2024
    Configuration menu
    Copy the full SHA
    3db14ab View commit details
    Browse the repository at this point in the history