mlapptools.getWebElements() fails for axes or uiaxes

[xiangruili]

Summary

Besides some warning, mlapptools.getWebElements() fails for axes or uiaxes due to struct(hUIElement) has no Controller field. I wonder if there are any other ui objects with this problem.

Problem Description

Reference to non-existent field 'Controller'.

Error in mlapptools/getDataTag (line 683)
dataTag = char( struct(hUIElement).Controller.ProxyView.PeerNode.getId() );

Error in mlapptools.getWebElements (line 321)
hWin, mlapptools.getDataTag(hUIElement), descendType);

Steps to Reproduce the Problem

ax = uiaxes(uifigure); % or ax = axes(uifigure);
mlapptools.getWebElements(ax)

[Dev-iL]

Hi @xiangruili, can you please mention which MATLAB version you're using?

[xiangruili]

Hi @xiangruili, can you please mention which MATLAB version you're using?

I tried both 2020b and 2019b, with the same issue.

Edit: Just tried 2018b, and it works with warning.

[Dev-iL]

I'll look into this as soon as I can, but for now please try the following fork: https://github.com/Khlick/mlapptools where I think this is fixed.

[xiangruili]

@Dev-iL Thanks for the prompt reply!

[Dev-iL]

@xiangruili So I had a few minutes and took a look at the code. That's when I remembered that modifying uiaxes objects wasn't really supported by the tool since they were (at least as of the last time I looked at their HTML) webgl objects, which are significantly more difficult to manipulate than "regular" web elements.

Regardless, there shouldn't be arbitrary MATLAB errors when running the tool. In case axes objects are still similarly complex entities, the code should result in a warning (which already appears in the code).

Sent using FastHub

[xiangruili]

What I am trying to do is to get a unique ID from Matlab side, and identify the element in JS. Then in JS, all I need is the bounding box information for the element. From mlapptool, it seems data-tag is a good candidate, but unfortunately it is not available fox axes element. As shown in the code, I am now sending the bounding box information from Matlab to JS. It works fine, except that the documentation of getpixelposition says it is for figure/GUIDE only (it looks fine for uifigure).
BTW, I use your trick of a fake button to trigger Matlab callback from JS.
Any idea is welcome. Thanks for your tool and post.

[Dev-iL]

@xiangruili I can tell you what I did in previous MATLAB versions, and you can give it a try and see what works:

Create a uifigure with some axes. After the figure is completely loaded, use getHTML to dump the HTML code of the uifigure. Save this as an html file and open with a browser or simply use your favorite text editor. Find a node that represents the uiaxes and try to identify some parent\child node that has a property containing some unique ID (such as a data-tag). From there it's a matter of using dojo.query to locate ths object using information we can access from the MATLAB side. And... that's pretty much it - I hope this information is useful to you.

I'm happy to hear that you found the button trick useful 🙂, although I think I saw something in the changelog of one of the recent releases that adds a supported form of this functionality.

[xiangruili]

For the supported form of trigger, did you mean using uihtml? The assignment of the Data property at either of uihtml or JS htmlComponent will trigger callback at the other side. The first version of my uiFileDnD did use that feature. But there are other issues with uihtml, so I switched to the fake button trick.

For axes issue, if it is only for axes, I can try to find another unique ID if possible. That is why I asked you if there are any other elements that do not have data-tag .

[Dev-iL]

@xiangruili Please try the latest version I just uploaded. If that doesn't work for you, you can try working with the outputs of the mlapptools.getWidgetList function (this function returns a lot of data, which will surely contain the information you want; unfortunately, there's a lot of information returned by the function, so it might be be tricky to find it).

[xiangruili]

@Dev-iL It works great now. Thanks.
But for my specific purpose, I will stick with the current scheme: get Position in Matlab, and send it to JS (rather sending an ID). The reason are, (1) there may be an object type that has no convenient unique ID, and (2) the JS element bounding rect may be inconsistent with Position. For example, my fake button has size of [0,0] in Matlab, while it gives [6,6] in JS. For axes/uiaxes, JS gives the same size as figure size (probably need to dig into child element for correct axes size).
Anyway, you solved the issue, and I am closing it.