IntroductionMicrosoft UI Automation is the new accessibility framework for Microsoft Windows. It addresses the needs of assistive technology products and automated test frameworks by providing programmatic access to information about the user interface (UI). In addition, UI Automation enables control and application developers to make their products accessible.
UI Automation provides programmatic access to most user interface (UI) elements on the desktop, enabling assistive technology products such as screen readers to provide information about the UI to end users and to manipulate the UI by means other than standard input. UI Automation also allows automated test scripts to interact with the UI.
UI Automation provides full functionality in Windows Vista, Microsoft Windows XP, and Windows Server 2003.
Who will use UI Automation?The latest research study, commissioned by Microsoft and conducted by Forrester Research, shows that 57 percent (74.2 million) of working-age computer users (aged 18-64) in the U.S. are likely to benefit from using accessible technology. The research also confirms that the number of people who could benefit from accessible technology is increasing rapidly as the global population grows older and more people experience age-related difficulties and impairments related to vision, hearing and dexterity.
UI Automation is divided into two branches: UI Automation Providers and UI Automation Clients. UI Automation providers are applications such as MS Word, Excel, or any other program which can be used by users who need accessible technology support. UI Automation Clients are assistive technology applications, such as screen readers, screen enlargers,alternative input or others. Automated test scripts can use UI Automation for automated testing and are also considered clients in the UI Automation model. How will this help assistive technology developers?With current technology, assistive technology developers are required to use many different approaches to obtain and present information about the UI to computer users who rely on assistive technology products. These developers therefore spend an inordinate amount of time and resources gathering the UI information required for their application instead of spending this time innovating in their feature set. UI Automation reduces the cost for these developers by delivering the most complete accessibility model in the industry. It will become the primary source of UI information for the next generation of assistive technology products.
UI Automation ModelUI Automation exposes every piece of the UI to client applications as an AutomationElement. Elements are contained in a tree structure, with the desktop as the root element. Clients can filter the raw view of the tree as a control view or a content view. (These standard views of the structure can be easily seen using the UI Spy application). Applications can also create custom views.
AutomationElement objects expose common properties of the UI elements they represent. One of these properties is the control type, which defines its basic appearance and functionality as a single recognizable entity: for example, a button or check box. There are special helper classes and methods, that allow to obtain any UI element as AutomationElement object. So you can find any element you need by name, type, id, etc. walking through the tree.
In addition, elements expose control patterns that provide properties specific to their control types. Control patterns also expose methods that enable clients to get further information about the element and to provide input.
There is no one-to-one correspondence between control types and control patterns. A control pattern may be supported by multiple control types, and a control may support multiple control patterns, each of which exposes different aspects of its behavior. For example, a combo box has at least two control patterns: one that represents its ability to expand and collapse, and another that represents the selection mechanism.
UI Automation also provides information to client applications through events. Unlike WinEvents, UI Automation events are not based on a broadcast mechanism. UI Automation clients register for specific event notifications and can request that specific UI Automation properties and control pattern information be passed into their event handlers. In addition, a UI Automation event contains a reference to the element that raised it. Providers can improve performance by raising events selectively, depending on whether any clients are listening.
Read more: ArtfulBits
QR:
UI Automation provides programmatic access to most user interface (UI) elements on the desktop, enabling assistive technology products such as screen readers to provide information about the UI to end users and to manipulate the UI by means other than standard input. UI Automation also allows automated test scripts to interact with the UI.
UI Automation provides full functionality in Windows Vista, Microsoft Windows XP, and Windows Server 2003.
Who will use UI Automation?The latest research study, commissioned by Microsoft and conducted by Forrester Research, shows that 57 percent (74.2 million) of working-age computer users (aged 18-64) in the U.S. are likely to benefit from using accessible technology. The research also confirms that the number of people who could benefit from accessible technology is increasing rapidly as the global population grows older and more people experience age-related difficulties and impairments related to vision, hearing and dexterity.
UI Automation is divided into two branches: UI Automation Providers and UI Automation Clients. UI Automation providers are applications such as MS Word, Excel, or any other program which can be used by users who need accessible technology support. UI Automation Clients are assistive technology applications, such as screen readers, screen enlargers,alternative input or others. Automated test scripts can use UI Automation for automated testing and are also considered clients in the UI Automation model. How will this help assistive technology developers?With current technology, assistive technology developers are required to use many different approaches to obtain and present information about the UI to computer users who rely on assistive technology products. These developers therefore spend an inordinate amount of time and resources gathering the UI information required for their application instead of spending this time innovating in their feature set. UI Automation reduces the cost for these developers by delivering the most complete accessibility model in the industry. It will become the primary source of UI information for the next generation of assistive technology products.
UI Automation ModelUI Automation exposes every piece of the UI to client applications as an AutomationElement. Elements are contained in a tree structure, with the desktop as the root element. Clients can filter the raw view of the tree as a control view or a content view. (These standard views of the structure can be easily seen using the UI Spy application). Applications can also create custom views.
AutomationElement objects expose common properties of the UI elements they represent. One of these properties is the control type, which defines its basic appearance and functionality as a single recognizable entity: for example, a button or check box. There are special helper classes and methods, that allow to obtain any UI element as AutomationElement object. So you can find any element you need by name, type, id, etc. walking through the tree.
In addition, elements expose control patterns that provide properties specific to their control types. Control patterns also expose methods that enable clients to get further information about the element and to provide input.
There is no one-to-one correspondence between control types and control patterns. A control pattern may be supported by multiple control types, and a control may support multiple control patterns, each of which exposes different aspects of its behavior. For example, a combo box has at least two control patterns: one that represents its ability to expand and collapse, and another that represents the selection mechanism.
UI Automation also provides information to client applications through events. Unlike WinEvents, UI Automation events are not based on a broadcast mechanism. UI Automation clients register for specific event notifications and can request that specific UI Automation properties and control pattern information be passed into their event handlers. In addition, a UI Automation event contains a reference to the element that raised it. Providers can improve performance by raising events selectively, depending on whether any clients are listening.
Read more: ArtfulBits
QR: