Skip to content

GUI + Installer #135

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 35 commits into from
Jun 15, 2023
Merged

GUI + Installer #135

merged 35 commits into from
Jun 15, 2023

Conversation

0dm
Copy link
Collaborator

@0dm 0dm commented May 11, 2023

Resolves #134 (my own issue) - Created a GUI with tkinter
brew install python-tk@3.10

Contains:

  • Record Button with pop-up prompt for recording title:
    image

  • Visualize Button

  • Replay Button

  • Clear Database Button

  • Log on the right side of the window (currently, Record does not work for this feature - keep getting SIGTRAP when I call record from the app, which just translates to an unhandled exception on MacOS

image

Please take a look, and let me know if this has any future. I didn't spend too long on it, so if it gets scrapped - no big deal, I actually find it quite useful for my own testing. The code may not be the most optimal, I'm not super good with tkinter but it gets the job done.

--
Instructions to run (latest - 2023/05/26):
python3 -m openadapt.app.main
To build:
python3 openadapt/app/build.py

requires tkinter
@0dm 0dm requested a review from abrichr May 11, 2023 05:30
@0dm
Copy link
Collaborator Author

0dm commented May 11, 2023

Ah, just realized I based this off the "renamed all instances of puterbot to openadapt" branch.. I guess that's okay for now.

@0dm
Copy link
Collaborator Author

0dm commented May 11, 2023

@abrichr
Had an idea to make the logo a button too, the future of this would be to tell OpenAdapt to perform an action using replays for context. This is using code from #98 , with hardcoded keyboard inputs instead of InputEvents/ActionEvents, only because it is significantly faster for the demo video.

replay quick demo (hardcoded but under 1 minute):
https://streamable.com/1e7xnu

EDIT:
I gave replays a try, it was actually much faster than usual, maybe mouse inputs slow it down a lot more than keyboard..

in-depth:
https://streamable.com/0xq6bd

^ This is a good one. I teach through a recording, and have it replayed with a twist. At the end, I demo what visualization looks like. It's still a NaiveReplay but at least this one uses our API.

view the branch here (gui+demo): https://github.com/0dm/OpenAdapt/tree/gui+demo

added logo button
@abrichr
Copy link
Member

abrichr commented May 11, 2023

Excellent work @0dm !

For the gui, is there a reason you went with tk? What do you think about using NiceGUI with PyInstaller instead?

@0dm
Copy link
Collaborator Author

0dm commented May 11, 2023

I've used tkinter before so I wanted to give it a shot with this project. One benefit is that it comes installed with most python distributions, so less setup.

@0dm
Copy link
Collaborator Author

0dm commented May 12, 2023

Ok.. nicegui looks nice. I will try it out.

@abrichr abrichr changed the title #134: Created a GUI GUI May 23, 2023
@abrichr abrichr marked this pull request as draft May 23, 2023 15:15
@abrichr abrichr changed the title GUI GUI + Installer May 23, 2023
@0dm 0dm marked this pull request as ready for review May 23, 2023 15:43
@0dm
Copy link
Collaborator Author

0dm commented May 23, 2023

some screenshots:
image

image

image
image
image

bugs: ctrl + c closes the app, I'd like to prevent this during recordings so we don't have to reopen it.

@abrichr
Copy link
Member

abrichr commented May 26, 2023

@0dm is this ready for re-review?

@0dm
Copy link
Collaborator Author

0dm commented May 26, 2023

Yes @abrichr

@dianzrong dianzrong mentioned this pull request May 29, 2023
3 tasks
from nicegui import ui


class local_file_picker(ui.dialog):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use UpperCamelCase convention for class names 🙏

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay

"OPENAI_MODEL_NAME": "gpt-3.5-turbo",
# may incur significant performance penalty
"RECORD_READ_ACTIVE_ELEMENT_STATE": False,
# TODO: remove?
"REPLAY_STRIP_ELEMENT_STATE": True,
"IGNORE_WARNINGS": True,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@0dm what do you think about setting this to False by default?

Copy link
Collaborator Author

@0dm 0dm Jun 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's better to ignore warnings by default if we use the GUI, since the logger will just fill up with warnings (until we add extra support for ObjC objects). Maybe I could have it set IGNORE_WARNINGS to true within the app though.

@abrichr
Copy link
Member

abrichr commented Jun 13, 2023

@0dm please fix merge conflicts 🙏

@0dm
Copy link
Collaborator Author

0dm commented Jun 13, 2023

@abrichr resolved

@abrichr abrichr merged commit 3e1ebb9 into OpenAdaptAI:main Jun 15, 2023
@0dm 0dm mentioned this pull request Jun 16, 2023
R-ohit-B-isht pushed a commit to R-ohit-B-isht/OpenAdapt that referenced this pull request Jun 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create a GUI for OpenAdapt
2 participants