-
Notifications
You must be signed in to change notification settings - Fork 777
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Aggregate, update and curate Gibson Les Paul Studio knowledge submissions #1099
Conversation
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Beep, boop 🤖, Hi, I'm @instructlab-bot and I'm going to help you with your pull request. Thanks for you contribution! 🎉 I support the following commands:
Note Results or Errors of these commands will be posted as a pull request check in the Checks section below Note Currently only maintainers belongs to [[taxonomy-triagers taxonomy-approvers taxonomy-maintainers labrador-org-maintainers instruct-lab-bot-maintainers]] teams are allowed to run these commands. |
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added a fix or two before the linter catches them.
@jjasghar Appreciate your attention and help, but I had already fixed these locally but was not able to commit them yet... this is only 1/10 of what will be in the final submission and it is marked "draft" so that I do not waste anyone's time until it reached a point where I want dedicated reviews ;) |
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Assessment of PR #948 (created_by: apomponio) Les Paul Studio - Knowledge (Q&A) analysis
Wikipedia:
Wikipedia delta:
Wikipedia:
Wikipedia delta:
Merlinite answer
Wikipedia: Has answer in "History" section...
NOTE: This recent contribution to Wikipedia may be inaccurate; the merlinite model provides the correct, more accurate answer of a single, differerence of 5mm (or 0.19685 inches) Wikipedia delta: The eight and quarter inch dimensions exited in the trained model, with a slight "typo" fixed to account for proper character encoding of the "quarter" symbol:
Merlinite answer: Does not include dimensions from "History" in listed differences under "body shape":
If the Question is asked specifically about "body thickness", you get the following accurate response:
Merlinite answer: Which specifically asks about body thickness:
Wikipedia delta:
Merlinite answer:
Recommend: Improve question to ask about wood from all parts of the guitar and supply the totality of the delta provided in paragraph form.
For example:
OR: Merlinite answer
Wikipedia: includes this info. in the "History" section:
Wikipedia delta:
Merlinite answer: was truncated!
Merlinite answer: improved using an improved question:
|
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Beep, boop 🤖, Hi, I'm @instructlab-bot and I'm going to help you with your pull request. Thanks for you contribution! 🎉 I support the following commands:
Note Results or Errors of these commands will be posted as a pull request check in the Checks section below Note Currently only maintainers belongs to [[taxonomy-triagers taxonomy-approvers taxonomy-maintainers labrador-org-maintainers instruct-lab-bot-maintainers]] teams are allowed to run these commands. |
…cing approach Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Assessment of PR #956 contributions:
Wikipedia:
Wikipedia delta:
Merlinite response:
Conclusion
Wikipedia:
Wikipedia delta:
Merlinite responses:
Conclusion
Wikipedia:
Wikipedia delta:
Merlinite responses:
Conclusion
Wikipedia:
Wikipedia delta:
Merlinite responses: Conclusion
|
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Analysis of submission by
Wikipedia:
Wikipedia delta:
Merlinite responses:
we get the base model arguing that none exists (and it looks it is from a chatbot response!):
Conclusion
Recommend we improve both parts of the prompt to provide an answer directly including the Wiki paragraph from the section shown above and that does not argue (hopefully overrides due to new weights) that the model even exists.
Wikipedia:
Wikipedia delta:
Merlinite responses:
Conclusion
Wikipedia:
Wikipedia delta:
Merlinite responses:
Conclusion
Wikipedia:
Wikipedia delta:
Merlinite responses:
Conclusion
Wikipedia:
Wikipedia delta:
Merlinite responses:
Conclusion
|
Analysis of submittsion by
Conclusion
Conclusion
Conclusion
Conclusion
Conclusion
|
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
analysis of peterclauterbach
Merlinite responses:
Conclusion
Merlinite responses:
Conclusion
Conclusion
Wikipedia:
Wikipedia delta:
Merlinite responses:
Conclusion
Wikipedia:
Wikipedia delta:
Merlinite responses:
Conclusion
|
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
Signed-off-by: Matt Rutkowski <mrutkows@us.ibm.com>
@jjasghar Ready for review (finally) |
@instructlab-bot precheck |
Beep, boop 🤖, Generating test data for your PR with the job type: precheck. Your Job ID is 364. The results will be presented below in the pull request status box. This may take several minutes... |
Results for job ID: 364 using the model instructlab/granite-7b-lab! Results can be found here. |
Beep, boop 🤖, Hi, I'm @instructlab-bot and I'm going to help you with your pull request. Thanks for you contribution! 🎉 I support the following commands:
Note Results or Errors of these commands will be posted as a pull request check in the Checks section below Note Currently only maintainers belongs to [[taxonomy-triagers taxonomy-approvers taxonomy-maintainers labrador-org-maintainers instruct-lab-bot-maintainers]] teams are allowed to run these commands. |
This pull request has been automatically marked as stale because it has not had activity within 15 days. It will be automatically closed if no further activity occurs within 31 days. |
This pull request has been automatically closed due to inactivity. |
Describe the contribution to the taxonomy
This PR seeks to aggregate (including fixing linting issues) knowledge submissions from the following separate PRs as part of a triager-requested effort to do so as part of a post-event "bounty cleanup". All of the PRs aggregated seek to update Wikipedia knowedge around the topic of "Gibson Les Paul Studio" (see: https://en.wikipedia.org/w/index.php?title=Gibson_Les_Paul_Studio&direction=prev&oldid=1133072009).
These include:
Input given at the prompt
Response from the original model
Response from the fine-tuned model
I will detail the prompts submitted (as well as describe the "delta" from what the current models provide) one-by-one below. Specifically, the Llama 2-based models used by instructLab are based upon WIkipedia information as-of October 2020. For this topic (i.e., "Gibson Les Paul Studio"), The informations delta can easily be obtained/distilled using Wikipedia APIs.
Here are the "diffs" or "deltas" which should guide the curation of "updated content" following an aggregation of the Q&A contained within all PRs referenced and their respective
qna.yaml
files.Note: Llama 3 will update all Wikipedia knowledge based upon its data "cutoff" dates (i.e.,
The knowledge cutoff is March 2023 for Llama 3 8B and December 2023 for Llama 3 70B.
); so not that these skill submissions will only be meaningful for changes after those dates once our project's models are rebased on llama3.Contribution checklist
ilab generate
ilab generate
qna.yaml
file contains at least 5seed_examples
qna.yaml
file was linted and prettified (yaml-validator can do both)attribution.txt
file in the same folder as theqna.yaml
file