AutomationElement shows up using Inspect.exe but does show not up when using UIAutomationCore.dll or System.Windows.Automation
Multi tool use
Clash Royale CLAN TAG#URR8PPPAutomationElement shows up using Inspect.exe but does show not up when using UIAutomationCore.dll or System.Windows.Automation
TL;DR: What am I doing wrong that is causing the workspace pane to show up in Inspect Objects but not show up in my custom code?
I am trying to write some UI automation to a 3rd party program. I am using Inspect.exe that came with the Windows SDK, and I have tried both System.Windows.Automation and direct COM Calls (using the wrapper library from UIA Verify).
Process processes = Process.GetProcessesByName("Redacted Client");
if (processes.Length == 0) throw new Exception("Could not find "Redacted Client" process");
PropertyCondition parentFileCond = new PropertyCondition(AutomationElement.ProcessIdProperty, processes[0].Id);
PropertyCondition workspaceCond = new PropertyCondition(AutomationElement.NameProperty, "Workspace", PropertyConditionFlags.IgnoreCase);
PropertyCondition documentCond = new PropertyCondition(AutomationElement.NameProperty, "Untitled3", PropertyConditionFlags.IgnoreCase);
var parentElement = AutomationElement.RootElement.FindFirst(TreeScope.Children, parentFileCond);
var workspaceElement = parentElement.FindFirst(TreeScope.Children, workspaceCond); //Also does not work with TreeScope.Descendants
var documentElement = workspaceElement.FindFirst(TreeScope.Children, documentCond);
When I try the above code, parentElement does have the correct reference to the main program window, but workspaceElement is null.
parentElement
workspaceElement
If I change my documentElement code to:
documentElement
var documentElement = parentElement.FindFirst(TreeScope.Descendants, documentCond);
I will get the correct element returned. I can use this as a workaround as the document window is the one I really wanted anyway, but I would like to know why the Workspace pane would not show up so I can improve my skills in case I run into this in the future with a situation I cannot work around.
UPDATE: I tried MrGomez's suggestions
PropertyCondition parentFileCond = new PropertyCondition(AutomationElement.ProcessIdProperty, 5872);
PropertyCondition panelCond = new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Pane);
var parentElement = AutomationElement.RootElement.FindFirst(TreeScope.Children, parentFileCond);
var panels = parentElement.FindAll(TreeScope.Children, panelCond);
I get 3 results, unfortunately, I have 4 panels, and the one that did not show up was the panel named Workspace.
Workspace
I also tried to use a TreeWalker
PropertyCondition parentFileCond = new PropertyCondition(AutomationElement.ProcessIdProperty, 5872);
PropertyCondition workspaceCond= new PropertyCondition(AutomationElement.NameProperty, "Workspace");
var walker = new TreeWalker(workspaceCond);
var parentElement = AutomationElement.RootElement.FindFirst(TreeScope.Children, parentFileCond);
var workspaceElement = walker.Normalize(parentElement);
but that also returns null for workspaceElement
workspaceElement
Finally, in desperation, I tried the current value of "NativeWindowHandle" from Inspect and started the walking from the root node.
PropertyCondition workspaceCond = new PropertyCondition(AutomationElement.NativeWindowHandleProperty, 0x110906);
var walker = new TreeWalker(workspaceCond);
var workspaceElement = walker.Normalize(AutomationElement.RootElement);
Workspace element is STILL null.
I finally did get Workspace to show up, but I had to perform
PropertyCondition workspaceCond = new PropertyCondition(AutomationElement.NativeWindowHandleProperty, 0x110906);
var test = AutomationElement.RootElement.FindFirst(TreeScope.Subtree, workspaceCond);
and it took quite a while to run.
Here is screenshots from Inspect.exe showing the tree view.
Here are the properties of the main window of the program.
How found: Selected from tree...
RuntimeId: "[42.2557552]"
BoundingRectangle: {l:75 t:1 r:1311 b:1003}
ProcessId: 8160
ControlType: UIA_WindowControlTypeId (0xC370)
LocalizedControlType: "window"
Name: "Redacted"
AccessKey: ""
HasKeyboardFocus: false
IsKeyboardFocusable: true
IsEnabled: true
ClassName: "C:Program Files (x86)RedactedProgramFiles7RedactedClientFolder"
HelpText: ""
IsPassword: false
NativeWindowHandle: 0x270670
IsOffscreen: false
FrameworkId: "Win32"
ProviderDescription: "[pid:4000,hwnd:0x270670 Main:Nested [pid:8160,hwnd:0x270670 Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Nonclient:Microsoft: Non-Client Proxy (unmanaged:uiautomationcore.dll); Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]"
Window.CanMaximize: true
Window.CanMinimize: true
Window.WindowVisualState: Normal (0)
Window.WindowInteractionState: ReadyForUserInteraction (2)
Window.IsModal: false
Window.IsTopmost: false
Transform.CanMove: true
Transform.CanResize: true
Transform.CanRotate: false
LegacyIAccessible.ChildId: 0
LegacyIAccessible.DefaultAction: ""
LegacyIAccessible.Description: ""
LegacyIAccessible.Help: ""
LegacyIAccessible.KeyboardShortcut: ""
LegacyIAccessible.Name: "Redacted"
LegacyIAccessible.Role: client (0xA)
LegacyIAccessible.State: focusable (0x100000)
LegacyIAccessible.Value: ""
IsDockPatternAvailable: false
IsExpandCollapsePatternAvailable: false
IsGridItemPatternAvailable: false
IsGridPatternAvailable: false
IsInvokePatternAvailable: false
IsLegacyIAccessiblePatternAvailable: true
IsMultipleViewPatternAvailable: false
IsRangeValuePatternAvailable: false
IsScrollPatternAvailable: false
IsScrollItemPatternAvailable: false
IsSelectionItemPatternAvailable: false
IsSelectionPatternAvailable: false
IsTablePatternAvailable: false
IsTableItemPatternAvailable: false
IsTextPatternAvailable: false
IsTogglePatternAvailable: false
IsTransformPatternAvailable: true
IsValuePatternAvailable: false
IsWindowPatternAvailable: true
IsItemContainerPatternAvailable: false
IsVirtualizedItemPatternAvailable: false
IsSynchronizedInputPatternAvailable: false
FirstChild: "Workspace" pane
LastChild: "Application" menu bar
Next: "Inspect (HWND: 0x01700F06)" window
Previous: "Sandbox Console (Debugging) - Microsoft Visual Studio (Administrator)" window
Other Props: Object has no additional properties
Children: "Workspace" pane
(null) title bar
"Application" menu bar
Ancestors: "Desktop" pane
[ No Parent ]
Here are the properties of the problem "Workspace" pane.
How found: Selected from tree...
RuntimeId: "[42.34146524]"
BoundingRectangle: {l:83 t:51 r:1303 b:995}
ProcessId: 8160
ControlType: UIA_PaneControlTypeId (0xC371)
LocalizedControlType: "pane"
Name: "Workspace"
AccessKey: ""
HasKeyboardFocus: false
IsKeyboardFocusable: true
IsEnabled: true
ClassName: "MDIClient"
HelpText: ""
IsPassword: false
NativeWindowHandle: 0x20908DC
IsOffscreen: false
FrameworkId: "Win32"
ProviderDescription: "[pid:4000,hwnd:0x20908DC Main:Nested [pid:8160,hwnd:0x20908DC Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]"
LegacyIAccessible.ChildId: 0
LegacyIAccessible.DefaultAction: ""
LegacyIAccessible.Description: ""
LegacyIAccessible.Help: ""
LegacyIAccessible.KeyboardShortcut: ""
LegacyIAccessible.Name: "Workspace"
LegacyIAccessible.Role: client (0xA)
LegacyIAccessible.State: focusable (0x100000)
LegacyIAccessible.Value: ""
IsDockPatternAvailable: false
IsExpandCollapsePatternAvailable: false
IsGridItemPatternAvailable: false
IsGridPatternAvailable: false
IsInvokePatternAvailable: false
IsLegacyIAccessiblePatternAvailable: true
IsMultipleViewPatternAvailable: false
IsRangeValuePatternAvailable: false
IsScrollPatternAvailable: false
IsScrollItemPatternAvailable: false
IsSelectionItemPatternAvailable: false
IsSelectionPatternAvailable: false
IsTablePatternAvailable: false
IsTableItemPatternAvailable: false
IsTextPatternAvailable: false
IsTogglePatternAvailable: false
IsTransformPatternAvailable: false
IsValuePatternAvailable: false
IsWindowPatternAvailable: false
IsItemContainerPatternAvailable: false
IsVirtualizedItemPatternAvailable: false
IsSynchronizedInputPatternAvailable: false
FirstChild: "Untitled3" window
LastChild: "Letters (32638 of 32638):" window
Next: (null) title bar
Previous: [null]
Other Props: Object has no additional properties
Children: "Untitled3" window
"Letters (32638 of 32638):" window
Ancestors: "Redacted" window
"Desktop" pane
[ No Parent ]
Here are the properties of the "Working" document window.
How found: Selected from tree...
RuntimeId: "[42.9505096]"
BoundingRectangle: {l:85 t:53 r:651 b:491}
ProcessId: 8160
ControlType: UIA_WindowControlTypeId (0xC370)
LocalizedControlType: "window"
Name: "Untitled3"
AccessKey: ""
HasKeyboardFocus: false
IsKeyboardFocusable: true
IsEnabled: true
AutomationId: "10"
ClassName: "ProToolsSubMDIWndClass"
HelpText: ""
IsPassword: false
NativeWindowHandle: 0x910948
IsOffscreen: false
FrameworkId: "Win32"
ProviderDescription: "[pid:4000,hwnd:0x910948 Main:Nested [pid:8160,hwnd:0x910948 Annotation(parent link):Microsoft: Annotation Proxy (unmanaged:uiautomationcore.dll); Main:Microsoft: MSAA Proxy (unmanaged:uiautomationcore.dll)]; Nonclient:Microsoft: Non-Client Proxy (unmanaged:uiautomationcore.dll); Hwnd(parent link):Microsoft: HWND Proxy (unmanaged:uiautomationcore.dll)]"
Window.CanMaximize: true
Window.CanMinimize: true
Window.WindowVisualState: Normal (0)
Window.WindowInteractionState: ReadyForUserInteraction (2)
Window.IsModal: false
Window.IsTopmost: false
Transform.CanMove: true
Transform.CanResize: true
Transform.CanRotate: false
LegacyIAccessible.ChildId: 0
LegacyIAccessible.DefaultAction: ""
LegacyIAccessible.Description: ""
LegacyIAccessible.Help: ""
LegacyIAccessible.KeyboardShortcut: ""
LegacyIAccessible.Name: "Untitled3"
LegacyIAccessible.Role: client (0xA)
LegacyIAccessible.State: focusable (0x100000)
LegacyIAccessible.Value: ""
IsDockPatternAvailable: false
IsExpandCollapsePatternAvailable: false
IsGridItemPatternAvailable: false
IsGridPatternAvailable: false
IsInvokePatternAvailable: false
IsLegacyIAccessiblePatternAvailable: true
IsMultipleViewPatternAvailable: false
IsRangeValuePatternAvailable: false
IsScrollPatternAvailable: false
IsScrollItemPatternAvailable: false
IsSelectionItemPatternAvailable: false
IsSelectionPatternAvailable: false
IsTablePatternAvailable: false
IsTableItemPatternAvailable: false
IsTextPatternAvailable: false
IsTogglePatternAvailable: false
IsTransformPatternAvailable: true
IsValuePatternAvailable: false
IsWindowPatternAvailable: true
IsItemContainerPatternAvailable: false
IsVirtualizedItemPatternAvailable: false
IsSynchronizedInputPatternAvailable: false
FirstChild: "" thumb
LastChild: (null) title bar
Next: "Letters (32638 of 32638):" window
Previous: [null]
Other Props: Object has no additional properties
Children: "" thumb
(null) title bar
Ancestors: "Workspace" pane
"Redacted" window
"Desktop" pane
[ No Parent ]
TreeScope.Descendants
Do me a favor, since you have this code in your IDE? Do a quick
autoElement.GetCurrentPropertyValue(AutomationElement.NameProperty, true) on your Workspace pane and see what it comes up with. If this returns "Workspace" correctly, something else is clearly wrong.– MrGomez
Apr 18 '12 at 19:58
autoElement.GetCurrentPropertyValue(AutomationElement.NameProperty, true)
We can further refine this by seeing if
PropertyCondition workspaceCond = new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Pane); returns non-null when evaluated against your tree. If this corrects the traversal, then the trick will be to figure out why your name property isn't evaluating properly. From a cursory documentation walk, your code should just work, but it clearly isn't.– MrGomez
Apr 18 '12 at 20:06
PropertyCondition workspaceCond = new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Pane);
It will be a few as I have a long running process running in the software (one of the things I was trying to automate was so I don't need to baby sit error dialogs that must be dismissed for the program to continue during this operation that can pop up randomly). Once it is done I will update my question.
– Scott Chamberlain
Apr 18 '12 at 20:10
@MrGomez I got a chance to test it,
ropertyCondition workspaceCond = new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Pane); returned null, also your TreeWalker suggestion in your answer also did not work (but it let me get rid of my LINQ queries for other parts that where working, thanks for that). I did get it to show up by searching all nodes under the root node but that will not work for practical use. See my updated question for details.– Scott Chamberlain
Apr 19 '12 at 20:02
ropertyCondition workspaceCond = new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Pane);
TreeWalker
1 Answer
1
Very nice question. Based upon the problem you've documented, it's clear that your conditional:
PropertyCondition workspaceCond = new PropertyCondition(
AutomationElement.NameProperty, "Workspace", PropertyConditionFlags.IgnoreCase);
... fails evaluation. Why?
The answer is how your conditional is evaluated. Looking at your element tree, we notice this property for Workspace:
Workspace
IsWindowPatternAvailable: false
And for the main window and Untitled3:
Untitled3
IsWindowPatternAvailable: true
From MSDN:
UIA_IsWindowPatternAvailablePropertyId 30044
Identifies the IsWindowPatternAvailable property, which indicates whether the Window control pattern is available for the automation element. If TRUE, a client can retrieve an IUIAutomationWindowPattern interface from the element.
IsWindowPatternAvailable
TRUE
IUIAutomationWindowPattern
We find a repro in this thread, which implies the same failure pattern as the one you are currently experiencing. We also note the lack of Window properties present for this element because IUIAutomationWindowPattern is inaccessible.
Window
IUIAutomationWindowPattern
A workaround is available from the aforelinked thread. Instead of PropertyCondition, one might use:
PropertyCondition
public class ConditionMatcher : IMatchConditions
{
public bool Matches(AutomationElement element, Condition condition)
{
return new TreeWalker(condition).Normalize(element) != null;
}
}
Or, alternately, one might use the workaround you've given, provided your tree structure is guaranteed to be shallow (and thus, appropriate to the name of this site, will not trigger a stack overflow).
Admittedly, this wasn't the most obvious issue. In the perfect world, MSDN should have better documentation on this topic.
Wow thanks! it still is doing that the export from earlier (it will be several hours before it completes) but I will give this a test once I can re-create the situation and test it out.
– Scott Chamberlain
Apr 18 '12 at 20:52
@ScottChamberlain No problem! It's still unclear if the traversal is being outright blocked by the inaccessibility of
IUIAutomationWindowPattern, but it's the most plausible regression. Should this fail to be the case, I believe we have more than enough test cases to poke this with until we find the root cause.– MrGomez
Apr 18 '12 at 20:54
IUIAutomationWindowPattern
The
TreeWalker did not work either. I did get it to show up if I search all nodes under the root node and look for the window handle. but that will not work for Live use.– Scott Chamberlain
Apr 19 '12 at 20:05
TreeWalker
@ScottChamberlain Nice. Your repro tells us why the conditional is failing -- it can't query the name property exactly as we'd surmised. That leads me to this thread on MSDN, which tells a plausible story. Note that Inspect.exe is using the native UIA API, while you're using the managed API. Therefore, the next test becomes the following: what does UISpy have to say about your windows? If it's missing the name property, then using the native API is the right fix.
– MrGomez
Apr 19 '12 at 20:33
@ScottChamberlain Neat! For the benefit of others seeking out this information, please do let us know what worked for you. :)
– MrGomez
Apr 23 '12 at 19:58
By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.
Nit: I assume you meant
TreeScope.Descendantsin your comment above?– MrGomez
Apr 18 '12 at 19:42