Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: interact with screen element #1182

Draft
wants to merge 41 commits into
base: develop
Choose a base branch
from

Conversation

pixel-shock
Copy link
Contributor

@pixel-shock pixel-shock commented Oct 20, 2023

Hi @sds100,

this is the follow up for PR #1168 ...

I created this branch before my vacation so solve all merge conflicts and to test the features in my vacation on my tablet which I am using on my motocycle with the silverfox controller.
Everything went well and I was able to control my tablet with the controller on my bike flawlessly.

I just merged the latest develop branch from you into this branch to keep in in sync

So if you have some time please do a review of this one - thank you very much 👍

TODO

  • show button prompting the user to enable the accessibility service and disable the Select UI Element button rather than the "Start record" button.
  • Add "Interact with screen element" title to the top of the screen.
  • Show placeholder text for Element ID and package name if no element is selected yet.
  • The list of UI elements should have top gravity rather than be in the center.
  • Reduce jank by disabling the list while it is recording
  • Make search more fuzzy by matching words with any case. E.g "system ui" should match "System UI".

Closes #257

- TODO: check which events are really needed to fetch view ids
- TODO: check database structure (remove autoincrement)
- TODO: Add search bar to screen with list of view ids
- TODO: Add dropdown to select which action the user wants to perform with the view element (click, long click etc)
…Item" to search within the fully qualified view ID of the element
# Conflicts:
#	app/src/main/java/io/github/sds100/keymapper/actions/ActionDataEntityMapper.kt
#	app/src/main/java/io/github/sds100/keymapper/actions/BaseActionUiHelper.kt
#	app/src/main/java/io/github/sds100/keymapper/actions/CreateActionViewModel.kt
#	app/src/main/java/io/github/sds100/keymapper/actions/PerformActionsUseCase.kt
#	app/src/main/java/io/github/sds100/keymapper/actions/swipescreen/SwipePickDisplayCoordinateFragment.kt
#	app/src/main/java/io/github/sds100/keymapper/actions/swipescreen/SwipePickDisplayCoordinateViewModel.kt
#	app/src/main/java/io/github/sds100/keymapper/data/db/AppDatabase.kt
#	app/src/main/java/io/github/sds100/keymapper/system/accessibility/IAccessibilityService.kt
#	app/src/main/java/io/github/sds100/keymapper/system/accessibility/MyAccessibilityService.kt
#	app/src/main/java/io/github/sds100/keymapper/util/MathUtils.kt
#	app/src/main/java/io/github/sds100/keymapper/util/ui/NavigationViewModel.kt
#	app/src/main/res/values/strings.xml
…merge-pinch-and-interact-with-screen-element

# Conflicts:
#	app/src/main/java/io/github/sds100/keymapper/actions/ActionData.kt
#	app/src/main/java/io/github/sds100/keymapper/actions/ActionDataEntityMapper.kt
#	app/src/main/java/io/github/sds100/keymapper/actions/ActionId.kt
#	app/src/main/java/io/github/sds100/keymapper/actions/ActionUtils.kt
#	app/src/main/java/io/github/sds100/keymapper/actions/BaseActionUiHelper.kt
#	app/src/main/java/io/github/sds100/keymapper/actions/CreateActionViewModel.kt
#	app/src/main/java/io/github/sds100/keymapper/actions/PerformActionsUseCase.kt
#	app/src/main/java/io/github/sds100/keymapper/actions/swipescreen/SwipePickDisplayCoordinateFragment.kt
#	app/src/main/java/io/github/sds100/keymapper/actions/swipescreen/SwipePickDisplayCoordinateViewModel.kt
#	app/src/main/java/io/github/sds100/keymapper/data/entities/ActionEntity.kt
#	app/src/main/java/io/github/sds100/keymapper/system/accessibility/IAccessibilityService.kt
#	app/src/main/java/io/github/sds100/keymapper/system/accessibility/MyAccessibilityService.kt
#	app/src/main/java/io/github/sds100/keymapper/util/Inject.kt
#	app/src/main/java/io/github/sds100/keymapper/util/MathUtils.kt
#	app/src/main/java/io/github/sds100/keymapper/util/ui/NavDestination.kt
#	app/src/main/java/io/github/sds100/keymapper/util/ui/NavigationViewModel.kt
#	app/src/main/res/navigation/nav_app.xml
#	app/src/main/res/values/strings.xml
…" button, remove unnessesary accessibility service flags
… prevent cancelation of the action through android because of "out of display bounds"
@sds100
Copy link
Collaborator

sds100 commented Oct 22, 2023

Thanks! I'm taking a look now.

How come we need to use a database to record UI elements? Can it not just be stored in memory temporarily?

@sds100
Copy link
Collaborator

sds100 commented Oct 22, 2023

Can you also add the description as a text field in the configuration screen instead of a dialog. Look at fragment_config_intent for an example

@pixel-shock
Copy link
Contributor Author

Can you also add the description as a text field in the configuration screen instead of a dialog. Look at fragment_config_intent for an example

Done ✅

@pixel-shock
Copy link
Contributor Author

@TechMan205 I agree that @sds100 maybe not know how much impact this app has ... I would also be willing to pay for this app, but the problem here is that Seth isn't doing this as his main job and if you are charging for an app or a service than you have to be always on point and you have to deliver, you know? And that's maybe the main reason for him to not do so.

@sds100
Copy link
Collaborator

sds100 commented Feb 7, 2024

Yeah, that's the main reason I'm not working on this anymore. I've been considering how to make this a worthwhile endeavour in terms of cash but I don't think it's possible. It's not the kind of app that gets a lot of screen time. Also, the fact it is free and FOSS is an attractive feature for a lot of people. And you're right, once you start charging people for something you've got to deliver and keep delivering.

@pixel-shock
Copy link
Contributor Author

Agree! So ... if I would have a better understanding for android development, then I would love to be part of this ... but as you can see based on my PRs I am not as fit in Android Dev and Kotlin as I should be to do more maintenance ... even if I am kind of relient to this app because I am using it to control my tablet via the SilverFox handlebar controller on my motocycle ... that was also the reason why I tried to implement those features (I mean 2 got merged 🥳 🥳 🥳) ...

But maybe there are some proper Android Devs who want to be part of this and to like create a team behind this app?
Would it be an option to do like a hint on the website and here on github that you're "searching" for maintainers?

@pixel-shock
Copy link
Contributor Author

PS: For any non-devs

It is really exhausting to do a main job and then to do a "after work work" to maintain such big projects ... I did it myself for a couple of years but here is the thing: you're running into a burnout as long as you have other things to do like family or non-tech hobbies ... I had it and I had to stop immediately any private projects to get healthy again ...

@sds100
Copy link
Collaborator

sds100 commented Feb 7, 2024

I think it would still be a lot of work to find someone willing to takeover. Most people have the same problem I am having right now. I could only work on this so much because I was still in high school and when COVID hit i had nothing else to do since I was on a gap year before uni 😅 I would love there to be some sustainable way to monetize this but I'm not sure it is competitive with working on the side at uni as a software developer. Also, I really want the core functionality to still be open source so maybe I can keep some more advanced features closed source and behind a paywall. Another issue is that the majority of my users are actually young people using it for Minecraft and they love it because it is free. I wonder if adding ads would be too disruptive.

I've gone in circles multiple times about this and always come to the conclusion that there is no way it could be competitive with working. Many people thought I was crazy at the time to not monetize this initially but I didn't want to be overburdened with being a business/entrepreneur. Also, it might never have been as successful as it is for my public image if it wasn't free and open source. I've landed multiple jobs and requests from people because of this project being all over the internet (which was the hope I had).

Edit: if this was a game or something with more screen time then one could make a lot of money with these user numbers.

@pixel-shock
Copy link
Contributor Author

Dude! Calm down and please use some formatting. It is nearly unreadable.

If you have bugs then open a proper bug report with as much info as you can on how to reproduce.

@pixel-shock
Copy link
Contributor Author

pixel-shock commented Feb 12, 2024

Sorry do say that, but people like you and comments like those from you are the reason why devs are running into burnouts and don't continue developing apps ...

I don't know what you're trying here but if this is driving you nuts then try to learn kotlin and android dev and develop by your own ... I am also sad about the "dev end" of this app but what you're doing here isn't helping anyone ...

@sds100 I am closing this PR because of @TechMan205 running crazy here ... if you have time I would be happy about merging this PR so feel free to reopen it

@sds100 sds100 added the enhancement New feature or request label Jun 5, 2024
@sds100
Copy link
Collaborator

sds100 commented Jun 5, 2024

@pixel-shock Do you want to continue working on this?

@pixel-shock
Copy link
Contributor Author

@sds100 as far as I remember I was kind of done but you had some changes mind which you wanted to do. I think you did not like my db integration.
I am currently renovating my house. So I can work on this after that (4-6 weeks hopefully) 🤔😉

@sds100 sds100 reopened this Jun 5, 2024
@sds100 sds100 marked this pull request as draft June 5, 2024 17:39
@sds100
Copy link
Collaborator

sds100 commented Jun 5, 2024

I've pushed my changes, in the PR description I put a TODO of what still needs fixing. Whenever you're free you can pick them up.

@pixel-shock
Copy link
Contributor Author

Awesome man 🙏🙌
I'll let you know when I am done with it.

And by the way: your app is in use every day while riding my bike and conrolling my apps with my silverfox controller. Still the best app available for android. Much kudos for you 🍻🍻🍻

@GfEW
Copy link

GfEW commented Jun 7, 2024

I think it would still be a lot of work to find someone willing to takeover.
[...]
I would love there to be some sustainable way to monetize this but
[...]
the majority of my users are actually young people using it for Minecraft and they love it because it is free. I wonder if adding ads would be too disruptive.
[...]
I've gone in circles multiple times about this and always come to the conclusion that there is no way it could be competitive with working.
[...]
Edit: if this was a game or something with more screen time then one could make a lot of money with these user numbers.

Just stumbled across this. If your main question is to "charge or not to charge", I'm not sure that's all there is. Have you considered communicating this dysbalance openly to every keymapper user, and actively inviting donations? Whilst most likely that's not a way to making "a lot of money", it might help to balance the (limited) time you're continuing to dedicate to this precious app, and to feel better about doing so. I'm not suggesting donations could replace a proper job, but there are open source projects that, for the time share they require, more or less get by on donations.

@pixel-shock
Copy link
Contributor Author

I think it's more about the fact that with money comes a certain obligation. And when there is little time left, as is currently the case with Seth, then it is difficult to reconcile. But I understand what you mean, I would also donate €25 or more straight away.

@sds100
Copy link
Collaborator

sds100 commented Jun 7, 2024

Well, asking for donations is also easier said than done. I think it breaks google play's terms of service unless they also get a cut, and asking for donations might not be allowed anyway? I think the best route at this point of maturity would be to start making closed-source and paid "pro" features that can be unlocked with In-app-purchases. That way I only have an obligation to maintain and support users that use those specific features.

Also, the vast majority of my user base seems to be using it in minecraft and are very vocal for better or worse, so would never donate or pay.

…reen-element_together

# Conflicts:
#	app/src/main/java/io/github/sds100/keymapper/actions/BaseActionUiHelper.kt
#	app/src/main/java/io/github/sds100/keymapper/constraints/ChooseConstraintViewModel.kt
#	app/src/main/java/io/github/sds100/keymapper/constraints/Constraint.kt
#	app/src/main/java/io/github/sds100/keymapper/data/entities/ActionEntity.kt
#	app/src/main/java/io/github/sds100/keymapper/system/accessibility/AccessibilityServiceController.kt
#	app/src/main/java/io/github/sds100/keymapper/system/accessibility/AccessibilityUtils.kt
#	app/src/main/java/io/github/sds100/keymapper/system/accessibility/MyAccessibilityService.kt
#	app/src/main/java/io/github/sds100/keymapper/system/apps/ChooseAppViewModel.kt
#	app/src/main/java/io/github/sds100/keymapper/system/apps/PackageUtils.kt
#	app/src/main/java/io/github/sds100/keymapper/util/Event.kt
#	app/src/main/java/io/github/sds100/keymapper/util/Inject.kt
#	app/src/main/java/io/github/sds100/keymapper/util/MathUtils.kt
#	app/src/main/java/io/github/sds100/keymapper/util/ui/NavDestination.kt
#	app/src/main/java/io/github/sds100/keymapper/util/ui/ResourceProvider.kt
@pixel-shock
Copy link
Contributor Author

JFYI: I did not forget about this PR ... I am just still busy with my house renovation, due to sickness everything took longer than expected :-/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature request: New action(s) - click/tap items in other apps
3 participants