The initial prototype AR system that will be used for engaging key stakeholders for this investigation is
operational at the University of Akron in PI Sastry’s Lab. This system has been successfully demonstrated at the NASA Johnson Space Center through the NASA SUITS Competition in 2018, 2019 and virtually in 2020.
We have implemented Participatory Action Research and studied its effectiveness in engaging students to address and improve problems in the Francis College of Engineering at UMASS Lowell and bring this experience to engage AR users in co-creating and contributing to the human-factor design of AR systems in workforce training.
The following videos demonstrate the AR system with Guidance and Annotation capabilities for the table-top conveyor cyber physical system. With Guidance, AR system describes the operation of the conveyor system with multiple visual and audio cues designed by domain experts and stakeholders. With Annotation, AR system displays information of object of interest.
AR and Neurmorphic Vision Sensors fusion towards human centric system design
Neuromorphic Vision Sensors (NVSs) are cameras inspired by the structure and function of the human eye. They leverage the asynchronous properties of human vision to output a stream of event that represents motion or a change in lighting intensity. This change in intensity is done at a pixel level and is stored in a tuple with the time, location, and sign of brightness change of the recorded event. Events can be on/off depending on whether the scene goes brighter/darker.
These figures (on the right)
show the use of NVS in a hand tracking application. The left panel is the view from a frame camera, whereas the right panel shows the NVS view which only outlines the moving hand.
This research aims to leverage this technology to create minimally invasive human centric designs for various applications. This includes positional awareness in a virtual environment, tracking, or physical rehabilitation. We envision that the integration of AR and NVS can support in tracking and quantifying human/object mobility through gestures, postures, and performance.
Participatory Design Process for Co-Creating Augmented Reality Based Education and Training Systems
Emerging technology advances, such as the Electrification of Vehicles, demand creative and effective approaches for reskilling the industry workforce by harnessing their current skills and expertise and providing new digital skills.
While AR Systems have matured in their capabilities, creating effective AR content remains challenging. This investigation explores how human factors that provide measurable outcomes of the learning and training experience can be integrated to create AR content that enables inter-generational communication and knowledge sharing among future workers with diverse backgrounds.
Utilize a prototype AR system to conduct Participatory Action Research (PAR) Focus Groups with representative ‘future workers’. Extract themes from the PAR data and revise AR System iteratively with input from key stakeholders (Delphi Method) drawn from manufacturing, social-sciences, education and engineering.
Our models and processes for AR integration will be anchored in participatory design and universal design learning principles where future workers will collaboratively co-create educational content in emerging media formats while working closely with experts from multiple disciplines.
We are recruiting project participants for AR system usability studies which will be conducted on the University of Massachusetts Lowell, North Campus. Help us evaluate state-of-the-art augmented reality systems such as the Microsoft Hololens 2 and the Varjo VR/XR systems. We will provide initial training on the use of AR systems and have you experiment on the use of these devices in assembling a mechanical system. You will also engage with researchers in a few focus-groups to help us refine our design.