Data Science Holodeck

This project is about revealing and making widely available the spectacular opportunities the virtual worlds provide to enhancing our professional experience as educators and business knowledge managers.  

The project team is challenged to discover how the novel visualization and interaction techniques, emerging with the virtual reality (VR), can improve the comprehension of data complexity.  

Our main objective is to create a value for cognitive activities and operations related to learning, such as understanding, remembering, reasoning, by accommodating virtual reality (VR) technologies. Therefore, we direct our professional strengths towards solving the following tasks: 

1. Creating knowledge by collecting empirical evidence about the impact of selected VR components and features, such as 3D user interfaces, visual metaphors, and collaborative interaction techniques, on comprehension of complex data. 

1. Applying this knowledge in designing and developing software engine that can facilitate academic and business professionals in creating virtual environments for collaborative data exploration. 

1. Formulating specifications and recommendations for practical implementation of such virtual environments, in support of solving tasks with comprehension and manipulation of complex data and abstract concepts, in either educational or business context. 

1. Transferring expertise to data-driven businesses, other academic institutions, to researchers, and other educators at Cphbusiness 

Over the recent years the world is exposed to a widespread explosion of data, generated by various digital sources and activities. Data-driven business becomes a norm and a feature of success. At the same time, the quantity and the dynamics of collected data, as well as the complexity of the algorithms for processing it increase. The modern data analytics and business intelligence (BI) practices, which involve methods from the artificial intelligence (AI), machine learning (ML), and deep learning (DL), can be challenging for many companies and individuals. As our pre-study reveals, the traditional diagraming visualisation techniques and dashboards, do not work well anymore. Overloaded with data and information, the consumers nowadays demand more efficient ways of data presentation – clearer, understandable, in context – to save them time from processing, to create comfort and trust. 

At another side of the IT specter, we observe fast evolvement of virtual visual technologies, such as virtual reality (VR), augmented reality (AR), and mixed reality (MR). Increasing number of industries worldwide, as in Denmark, apply these technologies, typically for modelling and simulation of real objects and processes, which are otherwise impossible or difficult to work with. 

It is noticeable that non-material concepts, such as data, information, knowledge, and algorithms remain still underrepresented. At present, there are no standards or specifications, neither recommended approach of researching, just too little empirical data is collected. 

It could be difficult to identify proper VR/AR/MR use case with just abstract numbers, key words, or algorithms, more complicated visualisation methods are required, and the efficiency of those visualisations is not clearly proved.  

The Holodeck team shares the idea the virtual technologies bring significant potential for facilitating humans in operations with abstract concepts and non-material objects, which are common for data science, BI/AI/ML, but also in education. 

The research proposed hereby aims at revealing this potential and then creating instruments for utilizing it into benefit and support for human functions, such as perception, comprehension, analytical reasoning, and abstract thinking.  

We intend to address the challenge by focusing on novel models for creating visuals of abstraction, trying novel techniques for interacting with these visuals. We also plan to explore the power of advanced algorithms, like in augmented intelligence, to exploit advanced data structures, such as graphs, and to text novel concepts, like ‘graph thinking’.  

This way we expect to achieve reducing complexity, revealing meaning, inferring new knowledge, and promoting action, in data-related task with either educational or business context. 

Our new empirical findings would contribute to researching the emerging, but still under-developed field of immersive technologies.   

We would also become the first team, implementing the new competences in developing usable software instruments, simple procedures, and clear recommendations, empowering other educators and business users with new opportunities for efficient and rewarding experience in their domain. 

The project proposal, its objectives, activities, and expected outputs go in line with the following framing recommendations: 

• international and European research and development guidelines, as defined by Frascati Manual1 
• national research priority areas, as pointed in FORSK2025 catalogue2 
• research recommendations for professional academies in Denmark3 
• CPH Business strategy4 
• current state-of-the-art and trends in the subject area5 
• best professional practices (Reference available on request) 

This project continues, extends and upgrades the research done within the related pre-study, https://innotechspace.dk/projects/data-science-holodeck/, held as a prove-of-concept in spring, 2020. 

Design Thinking  

As a foundation of our research methodology we adopted the design thinking process, which drives us from inspiration through exploration and application of our ideas. 

Based on our previously accumulated knowledge, teaching and researching experience, and collaboration with students and enterprises, we were able to identify some of the challenges the students and business users experience, while learning and implementing complex data structures and processing algorithms.  

To discover more, as part of the pre-study, we undertook an extensive review of published resources and best practices in the interdisciplinary area, where big data, data science, and data-driven business meet computer technology, artificial intelligence, and immersive visualization.  The review helped us to get knowledgeable about the current state-of-the-art and the major trends in our field.  As a result of it, we managed to specify our new goals and to narrowed down the scope of our follow-up research and development initiative (https://innotechspace.dk/projects/holodeck/exploration/ideation/). 

We choose to base our software development and integration platform on Unity and Oculus technologies. 

Prototyping 

Adopting the agile approach, during the pre-study we created our initial prototypes of interactive software application, running in VR (https://innotechspace.dk/projects/holodeck/exploration/prototyping/) 

It is a Unity application, able to  

• input (big) data set, one typically used for training models in ML,  
• convert its numeric and text features into virtual visuals 
• present the visuals in VR, ready for interaction with a user. 

The user then opens a VR space, where is possible to explore the data from quite different perspective, being able to move around, to compare, filter, touch and move virtual objects. 

The experiment illustrates various advantages of the approach, such as faster solving a clustering and recognition tasks, easier conceptualization of the space, much higher motivation and satisfaction. 

The prototype was tested and evaluated as feasible and proving the concept. Further it was used as a basis for planning continuation and extension of our research and developments. 

Research Experiments 

More experiments are planned, as a part of the follow-up research. Work hypotheses, design concepts, and code prototypes will be tested experimentally, both in created laboratory conditions and during real teaching and learning sessions. Cphbusiness teachers and students, as well as business partners would be invited to participate. 

Modelling and Simulation 

Representation and processing of complex data requires implementation of methods from mathematics, statistics, computer graphics and visualization, ML and DL algorithmics. 

HCI Evaluation 

Usability is a major feature of our products, therefore a range of methods for usability evaluation will be implemented. Factors that have impact on the users’ performance will be explored, as well. 

Technology 

Our research and development process require adaptation of computer technologies, such as: 

• Computer hardware with graphics-processing power (GPU) 
• Unity development platform and related extensions 
• VR/MR headsets and 3D interaction and controlling tools 
• Data sources, connectors, and middleware 
• GitHub for storing, sharing, and versioning the software 

Project Outcome 

The expected outcome consists of: 

1. Empirical evidence for the impact of selected 3D modelling, visualisation and interaction techniques on the comprehension of complex data. 

1. Novel specifications of virtual environments and 3D user interfaces for modelling, visualisation and collaborative interaction with complex data 

1. Software platform for immersive collaborative exploration of complex data. 

Data collected in our experiments contributes to extending the limited sources of knowledge about VR implementations in education and virtual data analytics. 

The specifications, defined by the empirical analysis, performed in our research provide methodological support to teachers, software developers, researchers, and other business professionals, interested in augmenting human abilities by use of advanced digital technologies in broad range of data-related tasks. 

The software platform, which applies the newfound knowledge, integrates advanced programming and VR technologies, such as virtual analytics, immersive collaboration, and augmented intelligence, in benefit of academics, data scientists, and business analysts. It  

• enables teamwork in virtual classroom or office,  
• promotes action and interaction between VR and participants 
• displays complex data structures and algorithms by understandable visuals 
• involces use of AI/ML methods visually 
• provides intelligent support to participants, when and where they need it. 

The software can be implemented directly   

• in teaching and learning at Cphbusiness and other educational and business institutions 
• in academic research and pedagogical experiments, aiming at creating reliable prove of lectors’ qualification 
• as a valuable asset and significant contribution of our school to larger projects, professional networks, and integrated software systems 
• as a platform for further research and exploration of 3D VR data structures and processing algorithms in both academic and business context. 

The project outcomes would be especially helpful in situations, where the remote teaching and learning are the best forms of educating. 

Expected Effect 

Our methodology and software create a multidimensional, multisensory and intelligent virtual environment, which provides the participants with a composition of multiple perspectives and degrees of freedom for personalization, navigation, interaction and collaboration. 

For our business partners, it would enable creating meaningful insights and intuition, speeding up the contextualization of raw data and the data-based decision-making process, while keeping the data processing transparent and the solutions ethical. 

For our students, it would facilitate cognitive functions and promote active learning, with better results in shorter time and higher motivation. 

Contribution to the improvement is expected also from the associated necessity of revising and updating the existing teaching models and learning practice. 

The teachers would benefit from the diversity of opportunities for teaching and researching, as well as from exciting inspiration, higher satisfaction and extended experience. 

Measurement 

There are numerous factors that can impact the positive effect of the project results, and each of them can be measured differently.  First, we need to identify the most important factors, then to define appropriate measures, and then to apply the measures for assessing the impact. The selection of variables and measures is task of designing the experiments.  

The overall effect of applying the project results will be validated in practice and measured with usability criteria for effectiveness, efficiency, and usefulness.