This wiki service has now been shut down and archived

From EscienceEnvoyWiki

Click to return to the main event page - Monday - Tuesday - Wednesday - Thursday - Friday - Biographies - Abstracts - eSI Workshop wiki

Web cast of plenary talks presentations and other material used at the DIR workshop

Data-Intensive Research Workshop: Monday 15th March 2010

Topic: Introduction - the interplay between challenges and technology in data-intensive research

Day organiser: Malcolm Atkinson

Introduction

The use of data in research is growing rapidly; the digital revolution generates more and more data, and policies encourage more data to be published. Expectations for openness, repeatability and evidence quality increase the data-use imperative.

Data use includes the data-curation lifecycle from creation or collection, through cleaning, integration, analysis, annotation, citation and presentation to preservation or discard. It includes finding data, developing an understanding of a body of data (often drawn from multiple sources), working out how to get evidence from data or what the newly available collections of data may now enable, deciding how to extract evidence from data possibly in combination with other data and presenting results so that evidence is understood, trusted and used.

There are many challenges in the wide range of data uses; for example, coping with complexity, with variable or poor data quality, with high volumes, with sophisticated analytic and presentation requests, with high data rates, with heterogeneity, with user numbers and diversity and so on. These challenges are addressed through multiple forms of iteration, progressively discovering and understanding data, progressively developing and understanding analytic methods, progressively refining the processes used to obtain particular forms of evidence, progressively improving the software, progressively adapting the computational platforms, and so on. Communities of data users build expertise around data and as they do they change requirements, patterns of use and modes of acceptable behaviour.

Data-intensive research is both research in any domain that has to pay serious attention to the ways in which it uses data in order to succeed, and research that improves our ability to use data. These co-evolve.

Monday’s programme explores that co-evolution. Talks will start from an understanding of a class of research challenges that need data-intensive methods and report how those methods are developing and exploiting advances in data-intensive technology. Other talks will introduce an approach to data-intensive methods and show how that approach is developing power and delivering results in multiple disciplines.

During the research village groups who have been making progress with any aspect of data-intensive research will display their wares. The participants will move around the village for 15-minute presentations. After each 15 minutes there will be a 5-minute interval to choose a new presentation at a different stand. This should provide a good opportunity for meeting fellow researchers, finding potentially useful ideas, methods and software and for starting new working relationships. Use the coffee breaks and the reception to follow up leads.

We also introduce three of the many ways of thinking about data-intensive problems: (1) programming paradigms, include workflows and bespoke high-level languages, (2) database paradigms, include parallel and distributed query as a framework for research methods, and (3) data analysis paradigms, combine statistical and algorithmic methods to extract knowledge from data. Other ways of thinking about data use will arise in the sessions and breakouts, as this list is by no means comprehensive. Also, a given technology might be suitable for more than one track, e.g. MapReduce is both a programming paradigm and a data-analysis paradigm. In other words, the tracks are not independent. We hope they will stimulate thought but not inhibit other lines of thinking.

The day, which is open to all (tell your friends), will form an introduction to data-intensive research, to current work and to key projects and products. It will form a framework for the rest of the week’s discussions.

The day will conclude with a drinks reception to give you a chance to have follow-up conversations and to start new ones.

Timetable

Discussion Pages

You can read more and discuss Monday's activities at these places: Monday's talks and the Research Village