-
-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[REVIEW]: ollamar: An R Package for running large language models #7211
Comments
Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks. For a list of things I can do to help you, just type:
For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:
|
Software report:
Commit count by author:
|
Paper file info: 📄 Wordcount for ✅ The paper includes a |
|
License info: 🟡 License found: |
👋 @hauselin, @elenlefoll, and @KennethEnevoldsen - This is the review thread for the paper. All of our communications will happen here from now on. Please read the "Reviewer instructions & questions" in the first comment above. Both reviewers have checklists at the top of this thread (in that first comment) with the JOSS requirements. As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines. The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, the reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention #7211 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package. We aim for the review process to be completed within about 4-6 weeks but please make a start well ahead of this as JOSS reviews are by their nature iterative and any early feedback you may be able to provide to the author will be very helpful in meeting this schedule. |
Review checklist for @KennethEnevoldsenConflict of interest
Code of Conduct
General checks
Functionality
Documentation
Software paper
|
|
Thanks @KennethEnevoldsen for your review/comments! I've made changes to the repo/doc to address your comments, which definitely clarified things a lot. Let me know if they've addressed your comments (see responses below). Your other comments relate to the paper itself, which I'll address later.
The R community has workflows/package structures that produce multiple licenses that usually aren't combined into one—ollamar is following closely the conventions followed by the R community. For example, see the multiple licenses in ggplot and dplyr, two of the most used R libraries.
Glad you think it's valuable! There are many workflows in place. First, the package already has github continuous integration and deployment, so it will be tested on macOS/linux/windows whenever there are changes to the repository (there are a lot of test cases, which are also being run whenever the repo updates). Second, because it's hosted on R's CRAN, the same tests are also being run regularly on CRAN's servers to ensure maintainability and compatibility (see regular test results here; note that on 2024-09-10, a few CRAN servers are down, resulting in failed tests on certain linux machines and test results page might not load). Note also that for a library to be hosted on CRAN, it has to satisfy many strict requirements regarding maintainability and compatibility (otherwise, CRAN will inform the author and take it down).
I've restructured the site so the home page focuses on installation and basic usage (also added a table of contents on the right). I've also added a Get started page that uses foldout menus to allow for ease of navigation. The old "Notes" section no longer exist and has been integrated into the rest of the documentation. There's a new Community section on the right sidebar that links to contributing guide and code of conduct.
I've updated the installation instructions so they are clearer, especially for different OS. |
thanks for the quick fixes!
Thanks for the clarification. It has been a while since I did packages in R and they were only for internal projects so didn't know CRAN regularly ran tests (def. nice to know). Just to clear up any worries I have: What happens if the Ollama community pushes an update with breaking changes? As I understand it would require an update from you. It might be ideal to add information about compatible versions to allow users to resolve compatibility issues.
I totally agree, makes navigation noticeably easier. Once the updates for the paper are in I will do a full run-through of code examples in the docs and run the tests. Optional:
|
I've added the versions that have been tested in the updated README (https://github.com/hauselin/ollama-r/blob/4fca9c0546b45e7ea998e600e8112de17e028340/README.md?plain=1#L32C1-L36C16). Let me know if this is good, @KennethEnevoldsen. Ollama should be relatively stable (86k stars and almost 7k forks) so it's unlikely they'll introduce breaking changes. But if they do, the (official) Python and JS libraries (and the hundreds of apps/tools that have already been built on top of it) will break too—ollamar's design philosophy is similar to these two libraries' and is very modular/follows good software design practices, so it should not be difficult to update. Regarding your two optional comments:
It actually already is in their list of libraries (but in a different section). I think the section you referred to is for their official libraries (other libraries are listed lower down on the same page).
There is a citation file here (again, it's located in this directory because I'm following R package development conventions. It is picked up by github though (if you go to the main page, you can see "Cite this repository" in the right sidebar). If I add a |
👋 @hauselin, @elenlefoll, and @KennethEnevoldsen - just checking in to see how things are going with this review. Could you each post a short update here? Also, @elenlefoll I don't see that you have created your checklist yet. Are you still able to conduct this review? Thanks! |
I am waiting for the updated article draft, made clear here:
Notably the missing state-of-the-field and design considerations as it allow me to evaluate whether the code lives up to the intent However, I understand that @hauselin was waiting for the second review before making too many changes. |
Yes @crvernon I've already revised the codebase based on @KennethEnevoldsen's suggestions. What's left are changes to the paper itself, and waiting for the second review before revising the paper makes more sense to me (but @crvernon, if you think it makes sense for me to revise the paper at this point too, let me know). |
Submitting author: @hauselin (Hause Lin)
Repository: https://github.com/hauselin/ollama-r
Branch with paper.md (empty if default branch): joss
Version: v1.2.0.9000
Editor: @crvernon
Reviewers: @KennethEnevoldsen, @elenlefoll
Archive: Pending
Status
Status badge code:
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
Reviewer instructions & questions
@KennethEnevoldsen & @elenlefoll, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:
The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @crvernon know.
✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨
Checklists
📝 Checklist for @KennethEnevoldsen
The text was updated successfully, but these errors were encountered: