Data Virtualization and Services

Posted on 14. Dec, 2010 by vbrown+ in Enterprise Architecture, SOA

Data Services has been an increasingly hot topic over the past few months, and for good reason. Data Services are a key to enterprise-enabling the information provided by an Enterprise Information Management (EIM) function. And, providing data as a service is an effective strategy for introducing the benefits of a service oriented architecture.

I’ve occasionally heard the concern that Data Virtualization and Services (DV&S) will provide uncontrolled access to data, compromising security and data quality. In fact, DV&S allows an EIM organization to more effectively manage and control the data that’s served to consumers, and to shape data to suit the consumers’ requirements.

A robust and cost-effective DV&S platform can be created using Open Source components.

Concept and Approach

Although data virtualization and data services can be implemented independently of each other, the greatest benefits come from incorporating both into a cohesive service platform.  Data virtualization enables the IT (or EIM) organization to provide federated views of data — composed of data from virtually any source, including other services. Data services provide a delivery mechanism that has all of the benefits offered by an agile, flexible, loosely-coupled integration architecture. So the approach is to:

• Provide data as a service
• Abstract the physical data layer to provide domain-specific views
• Enable data federation – including data warehouses and ODSs with other sources
• Establish the infrastructure necessary to support the goals and approach
• Incorporate security at the point of access
• Implement a tool that supports robust virtualization and performance

Goals/Benefits

A few of the strategic goals and most tangible benefits that a DV&S platform offers include:

• Position for maximum leverage of an EIM platform
• Improve levels of service and expand breadth of service to the business
• Provide rapid response to requests for data (requirements)
• Isolate physical data sources from consumption requirements and patterns
• Enable development of enterprise domain objects (shared data services)
• Enable rapid ad hoc views of data
• Support access to data via any standard method: SOAP, REST, JDBC, ADO.Net, etc.
• Ensure acceptable levels of performance when serving data (run-time performance)
• Position for integration with services in the Cloud

Roadmap

Basic requirements for implementing a data services capability and beginning to realize immediate value, and then even greater incremental value over time, include:

• Identify an initial project/application
• Establish a data services mechanism
• Define criteria, patterns, and standards for using the data services — including security
• Establish the use of data services as a solution design best practice (ideally, under Enterprise Architecture governance)

Platform

The following diagram, courtesy of Composite Software, is for illustration and depicts their commercial data virtualization product with a rich set of supporting tools. Actual solutions could follow various approaches.

As mentioned above, if a more conservative route to DV&S is required, a robust and cost-effective DV&S platform can be created using Open Source components.

Tags: data, data service, Enterprise Architecture, integration, LinkedIn, service, SOA, virtualization