All About Data Virtualization
Data Virtualization allows users to access various sources of data without knowing or caring where the data actually resides. Data is found and stored in various locations and formats, from database systems and applications to hardware devices and most often in text files located within the personal computers of end users. A wealth of business information is contained within this data, and the ability to unify all these sources into a common pool of knowledge can deliver huge benefits to any organization.
Data Virtualization presents information as a virtual service and promises to provide a single access point to manage and view all enterprise data regardless of data source or physical location. Data Virtualization is increasingly being recognized as a better way to integrate data when data consumers need real-time data, faster time to solution and lower costs.
As a proven data integration technique, data virtualization is being increasingly adopted by enterprises and government agencies to complement their earlier data consolidation and data synchronization investments. Moreover, data virtualization usage has recently accelerated as a way to gain tangible revenue, productivity, compliance and cost benefits.
Data virtualization middleware integrates data from anywhere across the extended enterprise in a unified, logically virtualized manner for consumption by nearly any front-end business solution including ad-hoc reports, portals, applications, enterprise-wide search and more. By accessing the data from both original and already consolidated data warehouse sources, data virtualization avoids the need for additional physical consolidation and replicated storage of source data, making it faster to build and lowering the cost to operate when compared to data-warehouse-only integration approaches.
Developers can create semantic abstractions in the form of relational views or data services using the vEnterprise VirtualDBA utility. At run time, user-level applications, reports or portals call upon these on-demand data services to provide the requested data. A high-performance Virtual Database Server optimizes access to the data, as well as queries, federates, abstracts and delivers this data to the consuming applications.
Operational transparency is a key feature, enabling the data virtualization middleware to fit easily into any existing IT environment. Rapid time to solution, lower total cost of ownership, and effective leveraging of existing staff and technology resources are the primary benefits that virtualization delivers.
In traditional Business Intelligence tools, data must be extracted, transformed and stored in proprietary data structures such as cubes prior to being accessible to end users. Most tools perform this task in a disk-based fashion which impacts on performance.
Newer generation tools provide in-memory processing capabilities which improve performance, but the unfortunate side-effect of implementing a Business Intelligence-only solution is the fact that after implementation, a new information silo has been created - a well integrated, Bi-specific knowledge repository.
Consumption of the integrated data, before it is transformed into 'Business Intelligence' cannot easily be achieved by other potential data consumers across the organization.
Data Virtualization promotes the concept of creating a data foundation that all potential consumers can build upon, including developers, data miners, management and other traditional users of Business Intelligence.
The vEnterprise architecture stack is displayed in the following image, clearly highlighting the key difference between Data Virtualization and traditional Integrate-and-Consume application-specific solutions, such as Business Intelligence tools.
vEnterprise allows you to bring together data from virtually any source, including Database Management Systems, Text Files, Spreadsheets and XML into a uniform, logical layer that can then be consumed but virtually any number of applications.
As part of the vEnterprise solution, Large Scale Systems provides a royalty-free Microsoft ASP.NET-based WebUI, with source code included to allow rapid customization and fast implementation.
vEnterprise is built on Microsoft .NET technology, ensuring a future-proof technology investment built upon industry-standard development tools, rather than introducing additional scripting and programming concepts. Your developers can rapidly implement and customize their solution using tools they are already familiar with.
What Sets vEnterprise Apart?
- Uses standard SQL for integration; no custom programming required
- High-performance Virtual Database Server with in-memory caching
- Multiple end-user interfaces including Web, SharePoint, Microsoft Office and the majority of industry-leading reporting tools
- Extendable using Microsoft .NET-based Plug-Ins
- Simply the easiest to use, most cost-effective integration platform available on the market today
Case Study: Integration between Manufacturing, ERP and Spreadsheets
A customer required integration between a leading Manufacturing platform, an ERP system and Time and Attendance data stored in Spreadsheet files. The first step was to use standard SQL queries for retrieving the individual data sets from these systems running on Oracle, Microsoft SQL Server and Microsoft Excel. Several queries retrieved the data from these individual platforms and virtualized it in-memory using a standard SQL JOIN statement. The process was completed in under 1 hour by the local Database Administrator; no programming expertise or additional tools were required.
Once the virtual database was designed, it was automatically hosted using the vEnterprise Virtual Database Server and made accessible to hundreds of users. Defining and configuring access-control and setting up the users took a further 8 hours of configuration.
Once configuration was completed, the newly created 'Manufacturing' virtual database was accessible using the vEnterprise WebUI platform.
Invidual queries were developed using the VirtualDBA utility and made available as drill-down knowledge domains within the WebUI platform.
From within the WebUI platform, users can navigate through different views, including the table view shown below, with groupable columns and expression filtering.
Data can be instantly visualized using a number of available charting views.
OLAP Cubes can be created in real-time at the click of a button. Users can create different dimensions by simply dragging and dropping the columns required to create the cube.
Users of Microsoft Office applications and any tool supporting ODBC can instantly access the data and use it within business applications using the vEnterprise ODBC driver.