PK
'Aoa, mimetypeapplication/epub+zipPK 'A iTunesMetadata.plistG
This section discusses the concepts of deploying design objects to a target schema, and executing the ETL logic defined in the deployed objects.
This section contains these topics:
After you design your ETL and data quality processes, you deploy and execute the resulting design objects to implement the design in the target schema. The Control Center Manager provides a comprehensive deployment console for viewing and managing all aspects of deployment and execution. It provides access to the information stored in the active Control Center.
The following topics provide overviews of deploying and executing design objects:
See Also: "Deploying to Target Schemas and Executing ETL Logic" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide. |
Deployment is the process of creating physical objects in a target location according to the logical objects in an Oracle Warehouse Builder workspace.
For example, when you create a table using the Design Center, the metadata for this table is stored in the workspace. If the table described in your design does not exist in the database schema referenced by the specified location, then you must create the table by deploying it. Similarly, after you design a PL/SQL mapping, you must generate code for it (which creates a PL/SQL package implementing the mapping logic), then deploy the generated code to the specified location, which loads the generated PL/SQL package to the referenced schema. You can deploy objects from within the Design Center, or use the Control Center Manager. You can also use OMB*Plus commands to deploy objects.
As soon as you define a new object in the Design Center, the object is listed in the Control Center Manager under its deployment location.
Deploying a mapping or a process flow includes these steps:
Generate the PL/SQL, SQL*Loader, or ABAP script, if necessary.
Register the required locations and deploy any required connectors. This ensures that the details of the physical locations and their connectors are available at run time.
Transfer the PL/SQL, XPDL, SQL*Loader, or ABAP script from the Design Center to the Control Center. (XPDL refers to XML Process Definition Language, a format standardized by the Workflow Management Coalition.)
Note:
|
See Also: "Deploying Objects" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide. |
After you deploy an object, Oracle Warehouse Builder assigns a deployment status to it. You can view the deployment status in the Control Center Manager.
The status represents the result of the deployment as follows:
Not Deployed: Indicates that the object has not yet been deployed to the target schema.
Success: Indicates that the object has been successfully deployed to the target schema.
Warning: Indicates that some warnings were generated during the deployment of the object. Double-click the status to view details about the warning.
Failed: Indicates that deployment of the object failed. Double-click the job in the Control Center jobs window to view information about why the deployment failed. The Job Log in the Design Center also captures details that you can view.
Execution is the process of starting the ETL logic defined in the deployed objects. For example, you define a mapping that sources data from a table, performs transformations on the source data, and loads it into the target table. When you deploy this mapping, the PL/SQL code generated for this mapping is stored in the target schema. When you execute this mapping, the job for the ETL logic is started and the data is picked up from the source table, transformed, and loaded into the target table.
During the lifecycle of a data system, you typically takes these steps in the deployment process to create your system and perform the execution process to move data into your system:
The Control Center for the selected configuration specifies the execution environment for the objects.
You can deploy them individually, in stages, or all.
If an object fails to deploy, then fix the problem and try again.
Execute the mappings or process flows that contain the ETL logic for the objects.
Accommodate user requests, changes to the source data, and so forth.
Set the modified objects to Upgrade or Replace.
Whenever you deploy and execute a design object, auditing information is generated and stored for you to view and manage.
The auditing information is specific to the type of object:
PL/SQL run time auditing is the auditing information for Oracle Warehouse Builder objects deployed using PL/SQL scripts. Data objects, mappings, and process flows use PL/SQL deployments.
Heterogeneous run time auditing refers to auditing information about objects that are deployed to an OC4J server or a heterogeneous database such as DB2 or SQL Server. This type of auditing includes auditing from multiple Control Center Agent (CCA). This type of auditing also includes information about Code Templates, Web services, and Code Template mappings.
You view and manage the audit information through these methods:
Control Center Manager. The Control Center Jobs panel displays a list of the deployment and executions jobs. When you double-click a job, you can see the job details.
Repository Browser or Heterogeneous Repository Browser. You can access audit information The Heterogeneous Repository Browser enables you to access auditing information on systems that do not have Oracle Warehouse Builder installed.
Public Views. As an alternative to using the Repository Browser, you can access the same information through the public views. To do the same, start a SQL*Plus session and query the public views. Refer to Oracle Warehouse Builder Sources and Targets Guide for a list of public views.
See Also: "Auditing Deployments and Executions" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
Oracle Warehouse Builder provides these scheduling options:
Integrated job scheduling in Oracle Warehouse Builder (This feature is available if you have the Enterprise ETL option)
Scheduling with Oracle Enterprise Manager to run ETL jobs
Integration with third party schedulers
Integration with Third-party Schedulers
Integration with third-party schedulers depends on the features of the third party scheduler. For example, mappings and process flows in Oracle Warehouse Builder are PL/SQL packages, and if there is a way in a third party scheduler to invoke a PL/SQL package, then refer to the documentation for that third party scheduler. You can also expose a mapping or a process flow as a Web Service if you are using a product like Oracle BPEL that can invoke Web services as part of orchestrating complex processes.
See Also:
|
This section summarizes the new high-value features in Oracle Warehouse Builder for this release.
This section contains the following topics:
"New Feature Highlights for Oracle Warehouse Builder 11g Release 2 (11.2)"
"New Features by Group for Oracle Warehouse Builder 11g Release 2 (11.2)"
"Complete New Feature List for Oracle Warehouse Builder 11g Release 2 (11.2)"
While the new features of Oracle Warehouse Builder for this release cover multiple areas, significant changes for new and existing customers are:
ETL support for non-Oracle databases, within and tightly integrated with the familiar flow-based ETL design paradigm
SOA integration for Oracle Warehouse Builder data integration and data quality functionality
Extensive user interface redesign for enhanced usability and developer productivity
Note: Go to the detailed sections under "Complete New Feature List for Oracle Warehouse Builder 11g Release 2 (11.2)" for links to the relevant documentation.Numerous smaller changes and improvements have been made throughout the product and, therefore, this list is not intended to be exhaustive. |
The major new features in Oracle Warehouse Builder for this release can be grouped into the following categories:
Oracle Warehouse Builder now provides extensive built-in support for non-Oracle databases. JDBC connectivity is added alongside previous support for ODBC and database gateways, and Oracle Warehouse Builder now supports in-database ELT operations on non-Oracle databases. Other enhancements improve access to data from non-Oracle sources such as mainframe and flat file data.
Features in this area include:
The ETL and data quality functionality in Oracle Warehouse Builder can now be integrated into SOA-style architectures.
Features in this area include:
The data warehousing-specific support in Oracle Warehouse Builder has improved. These improvements provide smarter dimensional object operators for ETL and support for more storage types for dimensional objects.
Features in this area include:
The Design Center user interface in Oracle Warehouse Builder has been extensively redesigned to improve developer productivity and make advanced features more accessible.
Features in this area include:
Oracle Warehouse Builder administration tasks are simplified and improved by multiple features in this release. Administration has been extended to support new feature areas such as heterogeneous database support and Web services integration.
Features in this area include:
ETL mappings have been enhanced to add new transformation capabilities and to improve the productivity of developers working with flat files and designing and debugging ETL mappings.
Features in this area include:
This section provides detailed descriptions of all major new features in Oracle Warehouse Builder for this release.
The Mapping Editor has been enhanced with advanced find capabilities to help locate and constitute dates to operators, groups, and attributes in a mapping diagram, in the Available Objects tab, and in the Selected Objects tab.
This feature enhances extraction, transformation, and loading (ETL) mapping developer productivity, especially on large and complex mappings and, for example, when working with complex data sources with large numbers of tables, views, or columns.
See Also: "Overview of the Mapping Editor" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
Oracle Warehouse Builder now supports integration with Oracle Business Intelligence Suite Enterprise Edition (OBI EE). This integration includes:
Derivation of ready-to-use physical, business model and presentation layer metadata for OBI EE from a data warehouse design
Visualization and maintenance of the derived objects from within Oracle Warehouse Builder
Deployment of the derived objects in the form of an RPD file that can be loaded into OBI EE.
Inclusion of the derived objects in Oracle Warehouse Builder data lineage and impact analysis, such that data lineage of objects in OBI EE reports can be traced down to the individual column level.
Oracle Warehouse Builder now supports OLAP cube storage in cube-organized materialized views. This brings the performance advantages of such storage to users of Oracle Warehouse Builder data warehouse design.
The new Code Template-based mapping framework enables implementation of data integration techniques and patterns for integration of content from non-Oracle databases with maximum performance and flexibility.
JDBC connectivity supports a wide variety of sources. Additionally, Oracle-supplied or user-developed Code Templates can use other native data integration techniques, such as bulk unloads and loads, for maximum performance on any platform.
Code Template mappings bring heterogeneous data integration support to the familiar flow-based data integration mapping paradigm that leverages existing developer skills with Oracle Warehouse Builder. Code Template mappings that load Oracle targets still support the full range of transformation capabilities available in other Oracle Warehouse Builder mappings.
See Also:
|
In the mapping editor, users can now copy and paste operators within a mapping or across mappings, including attribute settings.
This enhancement saves time and reduces errors in the development of complex ETL mappings that reuse common or similar elements.
See Also: "Copying Operators Across Mappings and Pluggable Mappings" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
In the Design Center, there is now a drop-down box that displays the active configuration of the user. This feature improves usability of the multi-configuration feature.
See Also: "Activating Configurations" in Oracle Warehouse Builder Installation and Administration Guide |
There are numerous improvements to support for importing flat files, including a simplified Flat File Sample Wizard, support for multi-character and hexadecimal format delimiters and enclosures, simplified support for fixed format fields, and support for bulk flat file loads into heterogeneous targets.
Flat files are frequently used for simple and high-performance data movement in ETL applications. These changes improve ETL developer productivity and provide more flexible handling of more types of flat files in more scenarios.
See Also: "Flat Files as Targets" and "Importing Definitions from Flat Files Using Sampling" in Oracle Warehouse Builder Sources and Targets Guide |
Oracle Warehouse Builder now has improved support for table functions, including importing metadata for existing table functions, a wizard for creating table functions from within Oracle Warehouse Builder, and better support for table functions in mappings.
Improved support simplifies using table functions for much more flexible and powerful transformations, such as user-defined aggregations, data mining sampling operators, and so on.
See Also: "Table Function Operator" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
It is now possible to add Oracle Warehouse Builder experts to the mapping editor menu.
This feature makes it possible to enhance and extend the functionality of the mapping editor, improving developer productivity.
See Also: "Overview of the Mapping Editor" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
Expressions associated with operator attributes can now be entered directly into an Operator Edit Dialog or Expression Editor, rather than requiring that these expressions be entered into a property in the Property Inspector.
Developers can finish more of their work in one place when creating operators in ETL mappings, thus improving their productivity.
See Also: "About Expressions" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
Platform extensibility enables users to define new platforms, represent the native data types for those platforms, and create ETL mappings that manipulate that data according to the requirements of the platform.
This feature is part of the overall improved support for heterogeneous databases in this release.
See Also:
|
Orphan management policy for dimensions or cubes refers to the process of handling source rows that do not meet the requirements necessary to form a valid dimension or cube record.
Oracle Warehouse Builder now supports the following orphan management policies:
Assign a default parent
Reject orphan rows
No maintenance
Automated orphan management policies improve ETL developer and administrator productivity by addressing an important cause of cube and dimension load failures.
See Also: "Performing ETL Using Dimensional Objects" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
You can now temporarily or permanently group objects in the Mapping Editor so that they are collapsed to a single icon. This hides complexity in mappings. Users can also spotlight a single operator, which temporarily hides all objects in the mapping except for those objects that connect directly to the operator.
These features improve productivity for developers working with complex mappings with large numbers of operators.
See Also: "Grouping Operators in Mappings and Pluggable Mappings" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
Auditing and reporting on run time jobs have been enhanced to show execution of all jobs required to support heterogeneous connectivity.
Users receive a unified view of all Oracle Warehouse Builder job executions on both Oracle and non-Oracle platforms.
See Also: "Monitoring Quality with Data Auditors and Data Rules" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
The user interface for managing the registration of locations in control centers has been reworked to improve usability, especially when working with locations registered in multiple control centers.
This change improves productivity of Oracle Warehouse Builder administrators responsible for managing locations across control centers.
See Also: "Locations Registered in Multiple Control Centers" in Oracle Warehouse Builder Sources and Targets Guide |
The Dependency Manager, which is used to browse data lineage and impact analysis information, now includes advanced metadata searching capabilities.
Users can now more easily locate specific objects in large and complex dependency graphs. This improves productivity by making it easier to find specific objects and their lineage, and discover impacts from design changes.
See Also: "Managing Metadata Dependencies" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
The user interface for managing Oracle Warehouse Builder locations has been reworked to improve usability and support access to non-Oracle data sources using newly supported connectivity methods.
These changes improve Oracle Warehouse Builder administrator and developer productivity in heterogeneous and Oracle-only environments.
See Also: "Designing Source and Target Schemas" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
The Control Center Agent provides a Java-based run time environment that can be installed on Oracle and non-Oracle database hosts. Heterogeneous ETL mappings and Web services-related code are deployed to the Control Center Agent, and run time audit metadata is accessible within Oracle Warehouse Builder.
The Control Center Agent provides fundamental infrastructure for the heterogeneous, Code Template-based mapping support and Web services-related features of Oracle Warehouse Builder for this release.
Extensive changes have been made to the key lookup operator:
More efficient use of screen real estate.
Support for non-equality lookups.
Dynamic lookups, where the lookup table may be modified during the mapping execution.
These changes make the lookup operator more powerful in many situations, including improving Type 2 slowly changing dimension support.
There are numerous enhancements to the Oracle Warehouse Builder Mapping Editor, including:
Improved support for watch points and enabling and disabling of individual break points.
Support for user-defined type columns.
Enhanced support for numerous existing operators, such as VARRAY
, EXPAND
, and CONSTRUCT
.
Support for key lookup and table function operators.
Improved cleanup of debugger-specific objects.
These enhancements improve productivity for ETL mapping developers, especially when working with complex mappings where the mapping debugger adds the most value.
See Also: "Debugging Mappings" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
In this release, Oracle Warehouse Builder provides the ability to import metadata from COBOL Copybook definitions.
This improves developer productivity, by simplifying working with complex flat file data structures extracted from mainframe sources.
See Also: "Importing Metadata Definitions from COBOL Copybooks" under "Using Flat Files as Sources or Targets" in Oracle Warehouse Builder Sources and Targets Guide |
The Oracle Warehouse Builder user interface for viewing and managing multiple configurations has been redesigned to simplify and clarify previously complex tasks, including:
Editing the configuration values for an object across all configurations
Using configuration templates to set default configuration values for different object types
Copying and pasting of configuration attribute values
Side-by-side editing of attribute values for multiple configurations for an object
These improvements enable users to take full advantage of the flexibility provided by multiple configurations.
The Oracle Warehouse Builder Design Center user interface has been updated to use the Fusion Client Platform, the same core Integrated Development Environment (IDE) platform as Oracle JDeveloper and Oracle SQL Developer.
The advantages of this user interface framework include:
More efficient and flexible use of screen real estate.
Support for opening multiple editors of the same type, for example, editing several ETL mappings in different windows.
More consistent behavior across different parts of the Oracle Warehouse Builder user interface.
This change brings the Design Center in Oracle Warehouse Builder in line with other development tools from Oracle.
PL/SQL code generated for Oracle Warehouse Builder ETL mappings now includes detailed comments to help developers associate specific operators in a mapping with sections of the generated code.
Developers can more easily troubleshoot issues with Oracle Warehouse Builder-generated code that can only be detected when the code is deployed. This additional information enhances developer productivity.
See Also: "Creating PL/SQL Mappings" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
Users can create hierarchically nested folders to logically group related objects. Folders can be created within Oracle and non-Oracle database modules, non-Oracle database modules, application modules. User folders can be nested as necessary to organize objects further.
You can use folders to group related objects. Any object in one of the supported module types, such as a table or a mapping, can be moved into a folder.
For example, if a single database module contained tables, views, and ETL mappings for product and customer data, folders "Product" and "Customer" could be created, and the objects related to each category moved into the separate folders.
User folders can also be created to contain pluggable mappings. This allows organization of related pluggable mappings into groups.
User-created folders improve ETL developer productivity on complex projects, by making it easier to logically group and manage large numbers of objects.
See Also:
|
In this release, Oracle Warehouse Builder introduces the Mapping Connection dialog box, a spreadsheet-like "quick mapper" for connecting operators in a mapping. This functionality replaces the Auto Mapping dialog box in earlier releases.
This improvement saves developer time and reduces errors when working with operators with a large number of inputs or outputs.
See Also: "Connecting Operators, Groups, and Attributes" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
The Repository Browser has been updated to support foldering, to expose the new types of metadata associated with the release 11.2 feature set, and to support OC4J 10.3.3.
These changes improve manageability for Oracle Warehouse Builder.
See Also: "Common Repository Browser Tasks" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
Chunking in Oracle Warehouse Builder automates the use of a "divide and conquer" approach to parallelize the processing of large updates. Users enable chunking for a mapping and define chunking criteria to partition the updates. Oracle Warehouse Builder generates PL/SQL code for the mapping, and at execution time, updates are divided according to chunking criteria, a pool of threads is allocated, and the chunks are processed in parallel.
The benefits of applying chunking include:
Chunking provides the only method of automatically parallelizing PL/SQL code in Oracle Warehouse Builder.
Chunking avoids the need for large rollback segments. Set-based SQL statements for large updates require large rollback segments, because a single set-based statement does not perform intermediate commits.
Large updates can be performed incrementally, and if interrupted, chunks are processed do not have to be processed again. Without chunking, if a large update terminates for some reason, all processing must be repeated.
See Also: The section on chunking data under "Using Oracle Source and Target Operators" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
The Repository Upgrade automatically upgrades an Oracle Warehouse Builder repository to the current release with less user intervention.
This feature simplifies the task of upgrading from one release to the next.
A new process flow activity supports calling an EJB or Java program from within a process flow.
Customers benefit from being able to incorporate existing or new logic implemented in Java into their data integration processes.
See Also: "Enterprise Java Bean" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
Oracle Warehouse Builder can now generate SQL*Plus code to extract data from database schemas supporting the deprecated LONG data type, such as occurs in PeopleSoft application data sources.
Support for LONG data types used in PeopleSoft data enables Oracle Warehouse Builder users to integrate more effectively with PeopleSoft data or any other data source that uses the LONG
data type.
See Also:
|
Process flows now support an activity type for running an OMB*Plus script.
New process flow activity types increase the breadth of user-defined activities that can be incorporated into process flows and thus orchestrated and managed as part of your overall data integration process.
The JOIN
operator in Oracle Warehouse Builder now supports several new behaviors related to the use of subqueries in joins:
Specifying subqueries using EXISTS
, NOT
EXISTS
, IN
, and NOT
IN
.
Specifying outer joins using the input role instead of the + (plus) sign.
Generating ANSI SQL syntax for all join types instead of only outer joins.
More flexible handling for JOIN
operations improves developer productivity and makes possible more flexible data transformations.
See Also:
|
Oracle Warehouse Builder ETL and data quality mapping, process flows, transformations, and data auditors can be published as Web services. Oracle Warehouse Builder can also consume Web services in process flows.
This feature provides point-and-click integration of the ETL and data quality functionality of Oracle Warehouse Builder into SOA-based designs, and facilitates integration with SOA-based process orchestration technologies such as Oracle BPEL Process Manager. (SOA stands for Service-Oriented Architecture).
See Also: "Creating and Consuming Web Services in Warehouse Builder" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
Concepts
11g Release 2 (11.2)
E10581-04
September 2011
Oracle Warehouse Builder Concepts, 11g Release 2 (11.2)
E10581-04
Copyright © 2000, 2011, Oracle and/or its affiliates. All rights reserved.
Primary Author: Richa Agarwala
This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited.
The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing.
If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, the following notice is applicable:
U.S. GOVERNMENT RIGHTS Programs, software, databases, and related documentation and technical data delivered to U.S. Government customers are "commercial computer software" or "commercial technical data" pursuant to the applicable Federal Acquisition Regulation and agency-specific supplemental regulations. As such, the use, duplication, disclosure, modification, and adaptation shall be subject to the restrictions and license terms set forth in the applicable Government contract, and, to the extent applicable by the terms of the Government contract, the additional rights set forth in FAR 52.227-19, Commercial Computer Software License (December 2007). Oracle America, Inc., 500 Oracle Parkway, Redwood City, CA 94065.
This software or hardware is developed for general use in a variety of information management applications. It is not developed or intended for use in any inherently dangerous applications, including applications that may create a risk of personal injury. If you use this software or hardware in dangerous applications, then you shall be responsible to take all appropriate fail-safe, backup, redundancy, and other measures to ensure its safe use. Oracle Corporation and its affiliates disclaim any liability for any damages caused by use of this software or hardware in dangerous applications.
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.
Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group.
This software or hardware and documentation may provide access to or information on content, products, and services from third parties. Oracle Corporation and its affiliates are not responsible for and expressly disclaim all warranties of any kind with respect to third-party content, products, and services. Oracle Corporation and its affiliates will not be responsible for any loss, costs, or damages incurred due to your access to or use of third-party content, products, or services.
This section provides an introduction to Oracle Warehouse Builder, and discusses the architectural components and objects that you create.
This section contains these topics:
Oracle Warehouse Builder is a full-featured data integration, data warehousing, data quality and metadata management solution designed for the Oracle database. Oracle Warehouse Builder is an integral part of Oracle Database 11g Release 2 (11.2) and is installed as part of every database installation (other than Oracle Database XE).
The major feature areas of Oracle Warehouse Builder include:
Data modeling
Extraction, Transformation, and Load (ETL)
Data profiling and data quality
Metadata management
Business-level integration of ERP application data
Integration with Oracle business intelligence tools for reporting purposes
Advanced data lineage and impact analysis
Oracle Warehouse Builder is also an extensible data integration and data quality solutions platform. Oracle Warehouse Builder can be extended to manage metadata specific to any application, and can integrate with new data source and target types, and implement support for new data access mechanisms and platforms, enforce your organization's best practices, and foster the reuse of components across solutions.
See Also: Oracle Database Licensing Information for complete information about options for Oracle Warehouse Builder. |
Oracle Warehouse Builder can be used in a wide range of scenarios, centered on Oracle Database, and adds value as a solution for data integration, data movement, and data quality. The data systems you create with Oracle Warehouse Builder are driven by rich metadata about sources and targets, and tight integration with, and awareness of, core features in Oracle Database. The ETL and data quality features provided by Oracle Warehouse Builder add value in each of the use cases described in this section.
The most common use cases include:
Business Intelligence and Data Warehousing
Oracle Warehouse Builder can be used in the design of relational objects for your operational data store, and dimensional objects for the data warehouse performance layer. You can implement ETL processes for loading warehouses, including smart operators that simplify loading dimensional objects, even for complex loading processes required for slowly changing dimensions. Oracle Warehouse Builder can implement business intelligence applications and data marts.
Oracle Warehouse Builder can also be used to profile data sources and to develop or discover data rules. You can use data rules to measure data quality, monitor, and enforce quality requirements during loading, or as an out-of-band process. Data cleansing logic can be incorporated into the warehouse loading process.
Master Data Management
Oracle Warehouse Builder application adapters (or connectors) enable access to data stores representing critical business entities such as customers and products at a logical, rather than physical, level. This simplifies the design of data movement, data quality and data cleansing, and enrichment processes.
You can use Oracle Warehouse Builder data quality features to discover, audit, and enforce the contents of your master data stores and their compliance with your data rules. Automated data cleansing and enrichment processes are easy to implement.
Data Migration, Conversion, and Modernization
You can use Oracle Warehouse Builder design to target for any migration or conversion process and can implement data movement processes. Data quality features offer high value in such scenarios as well. Data profiling of the source systems can reveal data quality issues before they are introduced into the new system. You can use Oracle Warehouse Builder profile to source data, design the target system, and to implement and orchestrate complex data movement, transformation and cleansing processes without requiring custom code.
Data Profiling and Quality Management
After you connect to your data sources in Oracle Warehouse Builder (including Oracle databases, sources accessed through gateways, and flat file sources) you can apply full-featured data profiling to generate statistics about data quality, and to discover complex patterns, foreign key relationships, and functional dependencies. You can then design complex data rules and create data auditors to monitor compliance with those rules in any source or target system in your landscape, regardless of whether those sources are loaded using Oracle Warehouse Builder or other ETL tools.
For customers who have selected solutions other than Oracle Warehouse Builder for data profiling and data quality, these can be applied independently of Oracle Warehouse Builder ETL and design features.
Note: Depending on how you use Oracle Warehouse Builder, you may require licenses for additional database options and technologies. Refer to Oracle Database Licensing Information for complete details about Oracle Warehouse Builder options. |
After Oracle Database is installed, you must not take additional actions other than to unlock the OWBSYS
and OWBuilderSYS_AUDIT
accounts, and run the Repository Assistant. This section provides
Oracle recommends that you start with the following steps to learn about using Oracle Warehouse Builder:
"Configure a Project in the Oracle Warehouse Builder Design Center"
"Deploy the Design and Execute the Data Integration Solution"
The first time you start Oracle Warehouse Builder, the Start Page is displayed with links to get you started using the product.
Note: Standalone software for Oracle Warehouse Builder is available with Oracle Database. Use the Oracle Warehouse Builder standalone software to host the Oracle Warehouse Builder repository on an earlier release of Oracle Database. Also, use the standalone software to install Oracle Warehouse Builder on client computers. See, "Working with the Oracle Warehouse Builder Standalone Install Package" in Oracle Warehouse Builder Installation and Administration Guide. |
Before you can use any of the Oracle Warehouse Builder client components, first ensure you have access to an Oracle Warehouse Builder workspace.
To begin using Oracle Warehouse Builder:
Install the Oracle Warehouse Builder software and create the necessary workspaces as described in "Installing Oracle Warehouse Builder on the Server" in Oracle Warehouse Builder Installation and Administration Guide.
If an administrator has previously completed the installation, contact that person for the required connection information.
On a Windows platform, from the Start menu, select Programs. Select the Oracle home in which Oracle Warehouse Builder is installed, then Oracle Warehouse Builder, and then Design Center.
On a Linux platform, run Oracle Warehouse Builderclient.sh
located in the Oracle Warehouse Builder/bin/unix
directory in the Oracle home for Oracle Warehouse Builder.
Use the Projects Navigator to manage design objects for a given workspace. The design objects are organized under a project, which provide a means for structuring the objects for security and reusability. Each project contains nodes for each type of design object that you can create or import.
Use the Connections Navigator to establish connections between the Oracle Warehouse Builder workspace and databases, data files, and applications.
Use the Globals Navigator to manage objects that are common to all projects in a workspace and to administer security.
Note: The Security node is visible to users who have an administrator role. |
In this procedure, you configure your project and access source and target data.
To configure a project in the Design Center:
In the Projects Navigator, identify the project to be used.
If you are satisfied with the single default project, MY_PROJECT, continue with the next step.
Alternatively, you can rename MY_PROJECT or define more projects. Each project you define is organized in the same fashion with nodes for databases, files, applications, and so on. See the procedure "To create a project" For a different organization, consider creating optional collections as described in "Collections".
Create locations in order to connect to source and target data objects.
To create a location, right-click the appropriate node and select New. Fill in the requested connection information and select Test Connection. In this step, you establish connections to sources and targets. You do not move data or metadata until subsequent steps.
In the Connections Navigator, establish these connections by defining locations. Expand the Location node and the nodes within it to gain a general understanding of the types of source and targets you can access from Oracle Warehouse Builder.
For more information about locations see "Locations Navigator".
Identify the target schema.
Although you can use a flat file as a target, the most common and recommended scenario is to use the Oracle Database as the target schema.
To define the target schema, begin by creating a module. Modules are grouping mechanisms in the Projects Navigator that correspond to locations in the Connections Navigator. The Oracle target module is the first of several modules you create in Oracle Warehouse Builder.
In the Projects Navigator, expand the Databases node. Right-click Oracle and select New. The Create Module Wizard appears. Set the module type to Warehouse Target and specify whether the module is be used in development, quality assurance, or production. This module status is purely descriptive and has no bearing on subsequent steps you take.
When you complete the wizard, the target module displays with nodes for mappings, transformations, tables, cubes and the various other types of objects you use to design the target warehouse.
Create a separate Oracle module for the data sources. (Optional)
At your discretion, you can either create another Oracle module to contain Oracle source data or proceed to the next step.
Identify the execution environment.
Under the Connections Navigator, notice the Control Centers node. A Control Center is an Oracle Database schema that manages the execution of the ETL jobs you design in the Design Center in subsequent steps.
During installation, Oracle Warehouse Builder creates the DEFAULT_CONTROL_CENTER
schema on the same database as the workspace.
If you choose to use the default execution environment, continue to the next step. Alternatively, you can define new control centers at any time. For more information and procedures, see "Deploying to Target Schemas and Executing ETL Logic" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide.
Prepare development, test, and production environments. (Optional)
Thus far, these instructions describe the creation of a single project corresponding to a single execution environment. You can, however, reuse the logical design of this project in different physical environments such as testing or production environments.
Deploy a single data system to several different host systems or to various environments, by creating additional configurations. See, "Managing Configurations" in Oracle Warehouse Builder Installation and Administration Guide.
Adjust the client preference settings as desired or accept the default preference settings and proceed to the next step.
From the main menu in the Design Center, select Tools and then Preferences.
As a new user, you may be interested in setting the "Environment Preferences" and the naming mode under "Naming Preferences". For information on all the preferences, see "Oracle Warehouse Builder Design Center and Runtime Preferences".
In this section, you create modules for each type of design object into which you intend to import metadata.
In the Projects Navigator, select a node such as Files.
For the selected node, determine the locations from which you intend to ultimately extract data.
Then create a module for each relevant location by right-clicking on the node and select New.
Import metadata from the various data sources: right-click the module and select Import to extract metadata from the associated location. Oracle Warehouse Builder displays a wizard to guide you through the process of importing data.
For an example and additional information on importing data objects, see "Importing Warehouse Builder Data into Business Intelligence Applications" in Oracle Warehouse Builder Sources and Targets Guide.
For the metadata you imported, profile its corresponding data. (Optional)
The next step uses the Data Profiling Option to ensure data quality as described in "Overview of Data Profiling" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide.
Data can only be transformed into actionable information when you are confident of its reliability. Before you load data into your target system, you must first understand the structure and the meaning of your data, and then assess the quality.
Consider using the Data Profiling Option to better understand the quality of your source data. With the Data Profiling Option, you can correct the source data and establish a means to detect and correct errors that may arise in the loading of transformed data.
See Also:
|
In this section, you create and design the data objects for the Oracle target module. In previous steps, you may have imported existing target objects.
To design the target schema:
To create data objects, you can either start the appropriate wizard or use the Data Object Editor. To use a wizard, right-click the node for the desired object and select New. After using a wizard, you may want to modify the object in the editor. In that case, right-click the object and select Open Editor.
For additional information, see "Designing Schemas" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide.
As you design objects, be sure to frequently validate the design objects.
You can validate objects as you create them, or validate a group of objects. In the Projects Navigator, select one or more objects or modules, then click the Validate icon.
Examine the messages in the Validation Results window. Correct any errors and try validating again.
To redisplay the most recent validation results at a later time, select Validation Messages from the View menu.
For additional information, see "Validating Data Objects" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide .
Configure the data objects.
To configure a data object, select the data object in the Projects Navigator and click the Configure icon. Or right-click the data object in the Projects Navigator and select Configure.
Configuring data objects specifies the physical properties of the object. You must not generate and deploy data objects without specifying the physical property values.
When you create data objects, Oracle Warehouse Builder assigns default configuration property values based on the type of object. In most cases, these default values are appropriate. You can edit and modify the configuration property values of objects according to your requirement. For example, you configure a table to specify the name of the tablespace in which it is created.
When satisfied with the design of the target objects, generate the code.
In the Projects Navigator, select one or more objects or modules, then click the Generate icon. Examine the messages in the Generation Results window. To redisplay the most recent generation results at a later time, select Generated Scripts from the View menu.
Alternatively, in the Data Object Editor, you can generate code for a single object by clicking the Generate icon.
You can save the generated script as a file and optionally deploy it outside Oracle Warehouse Builder.
Generation produces a DDL or PL/SQL script to be used in subsequent steps to create the data objects in the target schema. For more information about generation, see "Generating Data Objects" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide.
This procedure describes how to design mappings that define the flow of data from a source to target objects.
To design ETL logic:
In the Projects Navigator, expand the Oracle target module, right-click the Mappings node and select New.
The Mapping Editor enables you to define the flow of data visually. You can drag-and-drop operators onto the canvas, and draw lines that connect the operators. Operators represent both data objects and functions such as filtering, aggregating, and so on.
See detailed procedures in "Defining Mappings" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide , concluding with generating the code for the mapping.
Manage dependencies between mappings. See "Designing Process Flows" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide.
Deployment is the process of copying the relevant metadata and code you generated in the Design Center to a target schema. This procedure is necessary to enable the target schema to execute ETL logic such as mappings.
To deploy and execute the generated code:
Deploy objects from either the Design Center or Control Center Manager.
In this step, you define the objects in the target schema.
The simplest approach is to deploy directly from the Design Center by selecting an object and clicking the Deploy icon. In this case, Oracle Warehouse Builder deploys the objects with the default deployment settings.
Alternatively, if you want more control and feedback on how Oracle Warehouse Builder deploys objects, from the Design Center menu select Tools, then Control Center Manager.
Whether you deploy objects from the Design Center or the Control Center Manager, be sure to deploy all associated objects. For example, when deploying a mapping, also deploy the target data objects such as tables that you defined and any associated process flows or other mappings.
For more information, see "Deploying to Target Schemas and Executing ETL Logic" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide.
Execute the ETL logic to populate the target warehouse.
In this step, you move data for the first time. Repeat this step each time you want to refresh the target with new data.
You have two options for executing the ETL logic in mappings and process flows. You can create and deploy a schedule as described in "Defining Schedules", or you can execute jobs manually as described in "Starting ETL Jobs" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide.
It is essential to ensure the quality of data entering your data warehouse over time. Data auditors enable you to monitor the quality of incoming data by validating incoming data against a set of data rules and determining if the data confirms to the business rules defined for your data warehouse.
See Also: "Monitoring Quality with Data Auditors and Data Rules" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide. |
Although the Control Center Manager displays histories for both deployment and execution, the Repository Browser is the preferred interface for monitoring and reporting on Oracle Warehouse Builder operations.
See Also: "About the Repository Browser" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide. |
Oracle Database provides Oracle Warehouse Builder as part of the standard software when the database is installed. Oracle Warehouse Builder is an integral part of Oracle Database. Oracle Warehouse Builder runs on all versions (Standard Edition, Standard Edition One, Enterprise Edition) and typically all platforms that Oracle Database is certified on and ported to.
The basic Oracle Warehouse Builder architectural components on the server side are:
The main Oracle Warehouse Builder components on the client or desktop side, which are discussed in Chapter 3, "User Interface Tour", are:
Figure 2-1 illustrates the components that comprise Oracle Warehouse Builder and where they reside and run on clients and servers.
Figure 2-1 Oracle Warehouse Builder Components
See Also: "Overview of Installation and Configuration Architecture" in Oracle Warehouse Builder Installation and Administration Guide for diagrams of additional configurations. |
A major feature of the architecture in Oracle Warehouse Builder is the single, unified Oracle Warehouse Builder Repository for the database instance, which is pre-seeded with a schema and database objects. The run time environment and the design environment reside in this single repository. The repository schema, named Oracle Warehouse BuilderSYS
, gets created when you install Oracle Database. After the database is installed, you must not perform additional actions, other than unlocking the Oracle Warehouse BuilderSYS
and Oracle Warehouse BuilderSYS_AUDIT
accounts.
Note:
|
See Also: These topics in Oracle Warehouse Builder Installation and Administration Guide: |
To start using Oracle Warehouse Builder, you create at least one, new workspace. Users access their respective workspaces, instead of the repository as a whole. Thus, if you are Oracle Warehouse Builder administrator, instead of granting users access to a repository, you grant them access to one or more workspaces. Because all workspaces are stored in a single repository schema, creating workspaces is simplified.
In defining the repository, an administrator creates one or more workspaces, with each workspace corresponding to a set of users working on related projects. For example, a common practice is to create separate workspaces for Development, Testing, and Production. This practice provides team focus in addition to security. Users such as developers can have access to the Development and Testing workspaces, and can be restricted from the Production workspace. Later in the implementation cycle, you can use the Repository Assistant in Oracle Warehouse Builder to manage existing workspaces or to create new ones.
See Also:
|
Each workspace has a default Control Center that points to itself, and it is started and stopped with its corresponding Control Center Service. A Control Center stores detailed information about every deployment and execution, which you can access either by object or by job.
You can use the default Control Center to deploy to the local system, or you can create additional Control Centers for deploying to different systems as needed. Only one Control Center is active at any given time, and this is the Control Center associated with the current active configuration.
The Control Center Agent (CCA) runs on the Oracle Containers for J2EE (OC4J) server. Some capabilities of Oracle Warehouse Builder related to accessing non-Oracle data, such as Code Templates and Web services, depend on Java code that executes outside the database, in an OC4J server called the Java or J2EE Run time. For some heterogeneous data access scenarios, you must install a standalone Java Run time on hosts where there is no Oracle database installed.
You start the Control Center Agent with ccastart
from the command line. Oracle Warehouse Builder provides the cca_admin
utility to enable dynamic changes to Control Center Agent settings, without the requirement to shut down and subsequently restart the run-time environment.
Note: A Code Template (CT) contains the knowledge required by Oracle Warehouse Builder to perform a specific set of tasks against a specific technology, system, or set of systems. You must start the Control Center Agent before you deploy Code Templates.Refer to "About Code Templates" in Oracle Warehouse Builder Sources and Targets Guide and "About Prebuilt Code Templates Shipped with Warehouse Builder" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide. |
The data in your Oracle Warehouse Builder project is stored in a target schema within the server. This data is in the form of data objects such as tables, views, and dimension and cube objects. In a traditional data warehousing implementation, there is typically only one target schema, which is the data warehouse target. You can design both relational and dimensional target schemas. To design a target schema, you first create the target module that contains all the data objects. A target module is a container that holds the metadata definitions of all your data warehouse objects. Each target module corresponds to a target location that represents the physical location where the objects are stored.
In addition to context-sensitive help available with the F1 key, Oracle Warehouse Builder provides a Help Menu with links to utilities, training, the discussion forum, Oracle Technology Network and more. The Help Menu also contains the "Help Center", the online version of the complete Oracle Warehouse Builder documentation library.
The Help menu available from the Design Center contains these menu items:
Search. Provides a shortcut to the search facility for the online Help Center.
Table of Contents. Opens the "Help Center" with the Table of Contents selected.
Help Favorites. Opens the Help Center with your favorites selected, if you have configured favorite Help topics.
Dynamic Help.
Start Page. Accesses the Start Page after the first time you have run Oracle Warehouse Builder.
Extensions. A link to Oracle Warehouse Builder Utility Exchange on OTN. The purpose of the Oracle Warehouse Builder Utility Exchange is to provide the user community with a forum where utilities, code samples, and tips and tricks can be exchanged. The utilities posted here are not part of any Oracle production release and are, therefore, free-of-charge and not supported.
Training. Provides a shortcut to Oracle University where you can find out about training for Oracle Warehouse Builder.
Discussion Forum.
Oracle Technology Network. Provides a shortcut to Oracle Warehouse Builder on OTN for this release.
Check for Updates. Checks for Oracle Warehouse Builder product updates. When grayed out, no updates are available.
Session Properties. Displays information about the current workspace session such as workspace owner and name; connection properties like host name, service name and database version; and roles granted to the workspace owner.
About. Displays version information about the current software release.
The Help Center contains the complete Oracle Warehouse Builder documentation set in HTML format, available for online reading and searching. The Help Center opens with the Contents tab active. Click the plus symbol to expand the contents. Use the Search facility to enter topics on which to search.
Oracle Warehouse Builder provides the documentation described in Table 2-1.
Table 2-1 Oracle Warehouse Builder Documentation Library
Title | Description and Use |
---|---|
Oracle Warehouse Builder Installation and Administration Guide |
You use "Part I: Installing and Configuring Oracle Warehouse Builder" to perform any necessary installation tasks and to configure the Oracle Warehouse Builder repository. "Part II: Administering Oracle Warehouse Builder," starting with the chapter "Managing Configurations" provides detailed procedures for configurations, the Control Center and Repository, Control Center Agent, managing content, using the Metadata Loader, and managing security. |
Oracle Warehouse Builder Release Notes |
The release notes contain any late-breaking information about corrections, troubleshooting, and known issues. You can scan through the release notes during the set up processes to see the last-minute notes about this release. |
Oracle Warehouse Builder Concepts |
Similar to Oracle Database Concepts, this book provides a high-level explanation of the architecture, user interface, and components within Oracle Warehouse Builder. It provides a user interface tour chapter and overviews of the processes and steps used to perform typical tasks within Oracle Warehouse Builder. This book provides links to more detailed information and procedures within the other books. |
Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
This book provides comprehensive procedures for designing target schemas, performing data transformations, generating code, and doing all the tasks for optimizing and managing data quality. |
Oracle Warehouse Builder Sources and Targets Guide |
This book lists all of the supported sources and targets and provides procedures for importing from sources and deploying to targets. |
Oracle Database 2 Day + Data Warehousing Guide |
This book is part of the Oracle Database 2 Day + series and provides a good starting place to understand the data warehousing features offered with the database in addition to Oracle Warehouse Builder. |
Oracle Warehouse Builder API and Scripting Reference |
This book describes the scripting language available with Oracle Warehouse Builder and provides a complete language reference. |
Oracle Warehouse Builder Help(Only available as online help within Oracle Warehouse Builder.) |
The comprehensive help system that provides online versions of the complete documentation library, and context-sensitive help for all UI objects. |
This section discusses data sources and targets, and how Oracle Warehouse Builder provides solutions for accessing and moving data among disparate systems with simple to complex requirements.
This section contains the following topics:
Metadata is the data that describes the contents of a given object in a data source or target. For example, metadata for a table indicates the column names and data types for each column.
Before you import source metadata into Oracle Warehouse Builder, first create a module that contains these metadata definitions. The type of module you create depends on the source from which you are importing metadata. For example, to import metadata definitions from an Oracle database, you create or use an Oracle module. To import metadata definitions from flat files, you create a flat file module.
See Also: "Modules and Locations" in this guide, and "General Steps for Importing Metadata from Sources" in Oracle Warehouse Builder Sources and Targets Guide. |
Oracle Warehouse Builder must have metadata for any source or target object that can be manipulated in your project. The most basic metadata needed by Oracle Warehouse Builder can be created or derived in several ways:
Oracle Warehouse Builder can directly extract existing metadata from most database sources or targets. For example, when connecting to an Oracle database, Oracle Warehouse Builder queries the database dictionary to extract all needed metadata on tables, views, sequences, dimensions, cubes, data types, PL/SQL packages, and so on.
You can define and use SQL- or XML-based custom metadata stores to retrieve definitions of source and target objects such as tables and views.
When you design data objects that do not exist, the metadata that describes the object is created by the design process.
For data files extracted from some mainframe sources, Oracle Warehouse Builder can interpret Cobol Copybook files that describes the structure of the data file, and create its source metadata based on that.
Oracle Warehouse Builder application adapters or application connectors provide additional metadata about ERP and CRM application sources.
The metadata management and reporting features in Oracle Warehouse Builder, and data lineage and impact analysis, depend on, and leverage the metadata about the sources and targets and transformations that move data among them, which accumulates in your Oracle Warehouse Builder projects over time.
See Also: "Connecting to Sources and Targets in Oracle Warehouse Builder" in Oracle Warehouse Builder Sources and Targets Guide. |
The Import Metadata Wizard automates importing metadata from a database into a module in Oracle Warehouse Builder. You can import metadata from Oracle Database and non-Oracle databases. Each module type that stores source or target data structures has an associated Import Wizard, which automates the process of importing the metadata to describe the data structures. Importing metadata saves time and avoids keying errors, for example, by bringing metadata definitions of existing database objects into Oracle Warehouse Builder.
The Welcome page of the Import Metadata Wizard lists the steps for importing metadata from source applications into the appropriate module. The Import Metadata Wizard for Oracle Database supports importing of tables, views, materialized views, dimensions, cubes, external tables, sequences, user-defined types, and PL/SQL transformations directly or through object lookups using synonyms.
When you import an external table, Oracle Warehouse Builder also imports the associated location and directory information for any associated flat files.
A module is a container structure for the data objects in Oracle Warehouse Builder. Modules are equivalent to schemas from a database perspective. A location stores credentials needed to access a schema. Locations are linked to modules to provide access to metadata and the data itself. A module can have many locations associated with it, but only one can be the configured location at any point in time.
The association of a module to a location enables you to perform certain actions more easily in Oracle Warehouse Builder. For example, you can reimport metadata by reusing an existing module. Furthermore, when you deploy ETL processes in subsequent steps, modules enable you to deploy related objects, such as process flows.
Modules are created by expanding the Projects Navigator until you find the node for the data object type for which you want to create the module. For example, if the source data is stored in an Oracle Database, then you expand the Databases node to view the Oracle node. If the source data is in an SAP R/3 system, then you expand the Applications node to view the SAP node. By right-clicking the node, you can select New and start the Create Module Wizard.
See Also: "Modules" in this guide, and "Creating Modules" in Oracle Warehouse Builder Sources and Targets Guide. |
A connector is a logical link created by a mapping between a source location and a target location. The connector between schemas in two different Oracle Databases is implemented as a database link, and the connector between a schema and an operating system directory is implemented as a database directory.
You must not create connectors manually if your user ID has the credentials for creating these database objects. Oracle Warehouse Builder creates them automatically the first time you deploy the mapping. Otherwise, a privileged user must create the objects and grant you access to use them. You can then create the connectors manually and select the database object from a list.
To create a database connector, from within the Connection Navigator, expand the Locations folder and the subfolder for the target location. Right-click DB Connectors and select New. The Create Connector wizard opens with prompts for creating the connection. You can create a directory connection by right-clicking Directories and selecting New, following the same steps.
See Also:
|
This section summarizes the supported sources and targets for each Location node as displayed in the Connections Navigator.
Oracle Warehouse Builder supports sources from:
Oracle Database Releases 8.1 and later.
Any database accessible through Oracle Heterogeneous Services (Gateways), including but not limited to DB2, DRDA, Informix, SQL Server, Sybase, and Teradata.
Any data store accessible through the Code Templates (which use JDBC), including, but not limited to, DB2, SQL Server, Sybase, and Teradata.
Any data store accessible through the ODBC Data Source Administrator, including but not limited to Excel and MS Access
Delimited and fixed-length flat files.
ERP and CRM applications such as Oracle E-Business Suite, Peoplesoft and Siebel, from which data can be extracted using SQL.
SAP R/3, from which data is extracted using officially supported methods based on native ABAP code.
Oracle Warehouse Builder supports the following targets:
Third-party databases accessed through Oracle gateways or ODBC.
Comma-delimited and XML format flat files.
Oracle BI tools, such as Oracle Business Intelligence Suite Enterprise Edition (OBI EE).
Oracle Warehouse Builder can deploy or execute process flows and schedules to Oracle Enterprise Manager and Oracle Workflow. In general, you can deploy a schedule in any Oracle Database location, Release 10g or later.
See Also: "Supported Sources and Targets" in Oracle Warehouse Builder Sources and Targets Guide for a detailed and complete list. |
Oracle Warehouse Builder supports using flat files as data sources. Flat files are typically in plain text comma-delimited or tab-separated format, or proprietary binary formats, and may be stored on different types of operating systems. You first ensure that you have direct access either locally, or by creating a network connection, through TCP/IP or NFS for example. Oracle Warehouse Builder provides the Create Flat File Wizard to create a file object, which contains the imported flat-file definitions.
The Create Flat File Wizard provides intuitive prompts for importing metadata. To start the wizard, you right-click the file module and select Import. You can filter the filenames from which to import by applying wildcards. The wizard creates definitions for the files and inserts the file names under the Flat File module in the Project Navigator.
The locations that correspond to this module appear as folders on your computer's file system. The metadata is imported into a file module in Oracle Warehouse Builder and becomes visible in the workspace.
You can then sample the metadata from these flat files. The Flat File Sample Wizard enables you to view a sample of the flat file and define record organization and file properties. You can sample and define common flat file formats such as string and ASCII. The Flat File Sample Wizard also enables the importation of new data types such as GRAPHIC, RAW, and SMALLINT.
After you have created the flat-file locations and have imported the flat-file metadata, you are ready to import data. You introduce data from a flat file into an Oracle Warehouse Builder mapping either through an external table or a flat-file operator. Depending on how the data is to be transformed, use one of the following options:
See Also: "Using Flat Files as Sources or Targets" in Oracle Warehouse Builder Sources and Targets Guide for procedures. |
If the data is to be joined with other tables or requires complex transformations, then use the External Table option. An external table can be used as a source and enables data from the associated flat file to be viewed from SQL as a table. When you use an external table in a mapping, its column properties are based on the SQL properties that you defined when importing the flat file. Oracle Warehouse Builder generates SQL code to select rows from the external table. You can also get parallel access to the file through the table. You can either import an existing external table from another database or define a new external table.
You can also use an external table to combine the loading and transformation within a single set-based SQL DML statement. You do not have to stage the data before inserting it into the target table.
See Also: "Using External Tables" in Oracle Warehouse Builder Sources and Targets Guide for procedures. |
In cases where large volumes of data are to be extracted and little transformation is required, use the flat file operator. When you use a flat file operator, SQL*Loader code is generated. From the flat file operator, you can load the data to a staging table, add indexes, and perform transformations as necessary. The transformations you can perform on data introduced by a flat file operator are limited to SQL*Loader transformations only.
See Also: Oracle Database Utilities for more information about differences between external tables and SQL*Loader (flat file operators). |
Oracle Warehouse Builder achieves seamless management of JDBC-accessible data systems through its Code Template (CT) technology. Code Templates provide native heterogeneous connectivity to Oracle and JDBC-accessible data systems and disparate platforms. Code templates can be used as an alternative to Oracle Gateways for accessing other databases. In addition to Oracle Warehouse Builder being the best ETL solution for Oracle databases, with Oracle Warehouse Builder you can move data that is located in non-Oracle systems into and out of your project quickly and easily. JDBC connectivity provides an alternative to Oracle Gateways or accessing other databases.
Code Template technology in Oracle Warehouse Builder also provides direct data movement among JDBC-accessible databases, without stopping in an Oracle database in between. For example, to move data from DB/2 to SQL Server for some reason, then you can do so from Oracle Warehouse Builder without moving the data through Oracle at all.
See Also: "Using Code Templates to Load and Transfer Data" in Oracle Warehouse Builder Sources and Targets Guide. |
Note: When getting ready to generate code for a CT, you must have only the editor open for the CT on which you are focused. Otherwise, when you generate code for the CT, you get conflicting results. |
Oracle alternatively provides the generic connectivity agent and optional Oracle Database Gateways for connecting to non-Oracle databases such as SQL Server, Sybase, Informix, Teradata, DRDA, ODBC, and other sources. Oracle Warehouse Builder can communicate with non-Oracle systems using Oracle Database Heterogeneous Services and a complementary agent if you choose this route.
The generic connectivity agent is intended for low-end data integration solutions. The transfer of data is subject to the rules of specific ODBC or OLE DB drivers installed on the client computer. In this case, you must not purchase a separate transparent gateway. You use the generic connectivity agent included with Oracle Database. You must still create and customize an initialization file for your generic connectivity agent.
Oracle Database Gateways provide specific connection agents, designed and optimized for other databases, which you install and configure separately as needed. For example, for a Sybase data source, you install the Sybase-specific gateway. The non-Oracle system appears as a remote Oracle Database to which you can then create a connection and import its data into Oracle. This is especially useful for database environments that do not intend to harbor data marts or data warehouses, but that need integration with a set of other data sources.
See Also:
|
A transportable module enables Oracle Warehouse Builder to rapidly copy a group of related database objects from one database to another.
You use the Design Center to create a transportable module for which you specify the source database location and the target database location. Then you select the database objects to be included in the transportable module. The metadata of the selected objects are imported from the source database into the transportable module. The metadata is stored in the workspace. To physically move the data and metadata from source into target, you must configure and deploy the transportable module to the target location. During deployment, both data and metadata are extracted from the source database and created in the target database.
See Also: "Moving Large Volumes of Data Using Transportable Modules" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide. |
This appendix provides information about the Graphical Navigator and object editors in Oracle Warehouse Builder.
This appendix contains the following topics:
In design mode, Oracle Warehouse Builder provides the Graphical Navigator from which you can start object and document editors. Objects from the component palette and tree panel can be dropped onto this navigator. The Oracle Warehouse Builder object editors also support keyboard navigation, including tabbing and shortcuts.
When you double-click an object such as a table, a document panel opens containing one or more viewers and editors that are specific to that object type. These editors appear as tabs at the bottom of the document window. When you select an object, a property inspector and menu options are available for that object, and you can include related objects. Additionally, you can drag and drop appropriate objects from the Navigator tree to the Graphical canvas and then relate these objects with rubber bands.
This sections provides a list of the various editors in Oracle Warehouse Builder:
Cube Name. Contains fields for the name and description for the cube.
Cube Storage. Supports the specification of the storage type (ROLAP or MOLAP) and associated details. Changing the storage type from ROLAP to MOLAP, or vice versa, affects the Cube Dimensions Editor, Cube Measures Editor and Cube Aggregation Editor.
Cube Dimensions. Supports the specification of the cube's dimension references. For a MOLAP cube, the panel has an additional Advanced button, which allows sparsity, partitioning, and indexing to be specified.
Cube Measures. Supports the specification of the cube's measures. For a MOLAP cube, the panel has an additional Generate Calculated Measures button, which may be used to specify which calculations to generate for each measure.
Cube Aggregation. Supports the specification of aggregation options for the cube. The options presented depend on whether the storage type is ROLAP or MOLAP.
Physical Binding. For cubes, a physical binding tab graphically shows the object and the objects it is bound to. From here, you can open additional editors for the cube objects. Double clicking a cube and selecting Physical Bindings opens a graphical view of the cube with links to the bound objects. A property inspector shows the properties for the selected cube. (Operations on this tab are only available if the manual binding property has been set.)
Cube Data Viewer. Invokes the Data Viewer for the cube.
See Also: "Creating Cubes" and "Editing Cube Definitions" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide |
Dimension Name. Contains fields for the name and description for the dimension. It also supports specification of any dimension roles. For a time dimension, it supports additional fields specifying the start year and number of years supported by the time dimension.
Dimension Storage. Supports the specification of the storage type (ROLAP or MOLAP) and associated details.
Dimension Attributes. supports the specification of the Dimension's Attributes. The panel has an additional field to specify the Sequence to be used to populate Dimension keys.
Dimension Levels. Supports the specification of the Dimension's Levels, including the Level Attributes available at each Level.
Dimension Hierarchies. Supports the specification of the Dimension's Hierarchies, including the Levels available in each Hierarchy. The panel for a Time Dimension displays additional “Fiscal Settings” and “Create Map” buttons.
Dimension SCD. The Dimension SCD Editor supports the specification of Slowly Changing Dimensions. This panel is not present for Time Dimensions. (Also, Oracle Warehouse Builder 11.1 does not allow these details to be updated for MOLAP Dimensions.)
Physical Binding. For dimensions, the physical binding tab graphically shows the object and the objects it is bound to.
Dimension Data Viewer. Invokes the Data Viewer for the Dimension.
These tabs are available for table objects.
Table Name
Table Columns
Table Constraints
Table Indexes
Table Partitions
Table Attribute Sets
Table Data Rules
These tabs in the Data Viewer are available for viewing objects.
Structure Panel.
View Name.
View Columns
View Query
View Constraints
View Attribute Sets
View Data Rules
View Data Viewer
See Also: "Using the Data Viewer to View Data Stored in Data Objects" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide. |
These editor tabs are available for materialized view objects.
Structure Panel.
Materialized View Name Editor.
Materialized View Columns Editor
Materialized View Query Editor
Materialized View Constraints Editor
Materialized Indexes Editor
Materialized View Partitions Editor
Materialized View Attribute Sets Editor
Materialized View Data Rules Editor
Materialized View Data Viewer Editor
See Also: "Configuring Materialized Views" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide. |
Contains these tabs:
Structure Panel.
Object Type Name Editor
Object Type Columns Editor
See Also: "Defining Object Types" in Oracle Warehouse Builder Data Modeling, ETL, and data Quality Guide. |
Contains these tabs:
Structure Panel. Displays the VArray structure with its objects.
VARRAY Name Editor
VARRAY Details Editor
See Also: "Defining Varrays" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide. |
Structure Panel. Displays the nested table structure with its objects.
Nested Table Name Editor
Nested Table Details Editor
See Also: "Defining Nested Tables" in Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide. |
Create Business Area Wizard and Business Area Editor
Contains these tabs:
Structure Panel. Displays the business area structure with its objects: business name and item folders.
Contents Editor. Shows the item folders that are in the business area.
Business Area Name Editor.
Business Area Item Folders Editor.
Contains these tabs:
Structure Panel. Displays the item folder name and its contained items.
Composition Editor. Displays the objects that are used in the construction of the Item Folder
Item Folder Name Editor
Item Folder Source Items Editor
Item Folder Items Editor
Item Folder Joins Editor
Item Folder Conditions Editor
ABAP script
A script that can be generated in Oracle Warehouse Builder that extracts and loads data from SAP systems.
analytic workspace
A container within Oracle Database that stores data in a multidimensional format. Analytic workspaces provide the best support for OLAP processing.
Code Template (CT)
A cross-platform, reusable object that contains the information required to perform a specific set of tasks against a specific technology or set of technologies, for example data integration or data transformation tasks.
Code Template mapping
A mapping that contains an association with a Code Template. Typically used to extract or load data (both with and without transformations) from non-Oracle databases, such as IBM DB2 and Microsoft SQL Server.
Control Center Agent (CCA)
The agent that runs the Code Templates in the Oracle Containers for J2EE (OC4J) server. You must start the Control Center Agent before you deploy Code Templates or CT mappings. Also referred to as the J2EE Runtime.
Control Center Manager
The graphical console of the Control Center Service for centrally viewing and managing all aspects of deployment and execution. Provides access to the information stored in the active configuration. Includes update capabilities to enable management of your data system's life cycle.
Control Center Service
A service that runs outside the database, which can monitor and execute things that cannot be run directly in the database, such as: PL*SQL scripts, SQL*Loader, and shell scripts. Enables deployment of Oracle Warehouse Builder mappings and processes to targets (databases, and so on), and the execution of these mappings and processes.
cube
A data object that contains measures, and links to one or more dimensions. The axes of a cube contain dimension members, and the body of the cube contains measure values.
data auditors
Processes that provide data monitoring by validating data against a set of data rules to determine which records comply and which do not.
data rule
Metadata (as definitions) about data profiling results, which can be bound to the profiled data objects, and then be available in any context in which the profiled objects are used in ETL.
data transformation
A set of operations, which are specified in a mapping, that change source data into consistent, compatible output for a target.
DDL script
A script that can be generated in Oracle Warehouse Builder that creates or drops database objects.
deployable parameter
The parameter for an object that specifies it is to be deployed. By default this parameter is selected. To prevent an object from being deployed, clear this parameter.
deployable parameter
The parameter for an object that specifies it is to be deployed. By default this parameter is selected. To prevent an object from being deployed, clear this parameter.
deployment
The process of creating physical objects in a target location according to the logical objects defined in an Oracle Warehouse Builder workspace.
dimension
An object that contains additional metadata to identify and categorize data. Same as dimensional object. Can be a cube.
dimension attribute
A descriptive characteristic of a dimension member, having a name and a data type.
ETL
The process of extracting data from its source location, transforming it as defined in a mapping, and loading it into target objects (or schemas). ETL stands for extract, transform, and load.
execution
The process of running the code for the ETL logic that is defined in the deployed objects to instantiate the logic within the objects.
Expert
Mini-applications or task-flows that perform a specific sequence of tasks in Oracle Warehouse Builder.
flat files
Non-hierarchical, non-object-oriented file structures in plain text comma-delimited or tab-separated format, ASCII format, or proprietary binary formats.
folders
Structures in which to organize all or some objects within a target module based on specific object characteristics. For example, you may create user folders to group tables based on their functionality (sales, marketing, administration and so forth).
hierarchy
A structure that uses ordered levels to organize data. Oracle Warehouse Builder uses hierarchies to define relationships between adjacent levels in time dimensions.
location
Object that stores the connection information to the various files, databases, and applications that Oracle Warehouse Builder accesses for extracting and loading data. Locations also store connection information to ETL management tools and Business Intelligence tools.
mapping
An object that contains operations for extraction, transformation, and loading (ETL) that moves data from sources to targets.
mapping operator
The representation of an operation for a distinct task you want to perform in a mapping. For example, operations include extracting data, loading data, and transforming data.
match bin set
Relating to the match merge feature, a match bin set consists of one or more similar records.
module
A container object that appears in the Projects Navigator and that corresponds to a specific location in the Locations Navigator. A module can correspond to only one metadata location and data location at a time.
OMB*Plus
The Oracle Warehouse Builder scripting API, which is based on the Java implementation of Tcl called Jacl.
OMU
A subset of OMB*Plus that provides scripting commands for manipulating the user interface in Oracle Warehouse Builder. Also OMU-prefixed commands ('U' for UI component).
operator
Oracle Warehouse Builder contains pre-built operators for transformations, mappings, names and addresses, and so forth. Operators in Oracle Warehouse Builder are customizable and take advantage of the library of PL/SQL functions, procedures, package functions, and package procedures for Oracle Database. See also mapping operator.
Oracle Data Integrator
Oracle Data Integrator (ODI). See Oracle Fusion Middleware Developer's Guide for Oracle Data Integrator.
Oracle Warehouse Builder Repository
The single, unified repository for the database instance, which is pre-seeded with a schema and database objects. The run time environment and the design environment reside in this single repository. The repository schema, named OWBSYS
, gets created when you install Oracle Database.
pluggable mapping
A reusable grouping of mapping operators that works as a single operator. Similar in concept to a function in a programming language.
process flow
An object that describes dependencies and activities between Oracle Warehouse Builder mappings and external processes, applets, or applications. Process flows begin with a start activity and conclude with an end activity and can also start other process flows. Compare to schedule. See also activity.
project
The highest-level and largest object in Oracle Warehouse Builder workspace. Each project contains the metadata and definitions for objects in the data system that contains the sources and targets.
relational target schema
A target schema that contains relational data objects such as tables, views, materialized views, and sequences. All of the data for a data store or data warehouse is contained in these objects.
SQL*Loader control file
A file or script that can be generated in Oracle Warehouse Builder that extracts and transports data from file sources.
table function
A set of operators that enable manipulation of a set of input rows, which return another set of rows of the same or different cardinality. Can return a set of output rows that can be queried like a physical table.
target module
A container that holds the metadata definitions of all your data warehouse objects. Each target module corresponds to a target location that represents the physical location where the objects are stored.
target schema
A schema that contains the data objects that store your data warehouse data. You can design a relational target schema or a dimensional target schema.
transportable module
Type of module that enables rapid copying of a group of related database objects in one database, to be pasted or inserted into another database.
transformation operators
Prebuilt operators that enable commonly performed operations such as filtering, joining, and sorting. Oracle Warehouse Builder also includes prebuilt operators for complex operations such as merging data, cleansing data, or profiling data.
user folder
A folder you can create to organize all or some objects in a target module based on specific object characteristics. Related tables and views that must be generated or deployed can be placed under a common folder. For example, you may create user folders to group tables based on their functionality (sales, marketing, administration and so forth).
validation
The process of verifying metadata definitions and configuration parameters to ensure that data object definitions are complete and that scripts can be generated and deployed.
value-based hierarchy
A dimension in which hierarchical relationships are defined by a parent dimension attribute and a child dimension attribute.
workspace
Oracle Warehouse Builder structure that contains all the related projects and their objects. Graphically displayed as the canvas in the Design Center where Oracle Warehouse Builder windows, navigators, wizards, and dialog boxes are laid out to create a work environment that one or more users log in to.
Oracle Warehouse Builder Concepts provides an architectural and conceptual overview for features and functionality of Oracle Warehouse Builder. This document lays a conceptual foundation for much of the practical information contained in other manuals.
This preface contains these topics:
Oracle Warehouse Builder Concepts, is primarily intended for database administrators and application developers who are new to Oracle Warehouse Builder, and who develop and maintain data integration systems. Readers of this document typically perform one or more of the following tasks:
Plan for a data integration environment
Configure data integration systems
Administer a data integration environment
Perform data integration tasks
Deploy data integration systems
To use this document, you should be familiar with relational database concepts, distributed database administration, and the operating system under which you run, or plan to run, an Oracle Warehouse Builder environment.
For information about Oracle's commitment to accessibility, visit the Oracle Accessibility Program website at http://www.oracle.com/pls/topic/lookup?ctx=acc&id=docacc
.
Access to Oracle Support
Oracle customers have access to electronic support through My Oracle Support. For information, visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=info
or visit http://www.oracle.com/pls/topic/lookup?ctx=acc&id=trs
if you are hearing impaired.
For more information about Oracle Warehouse Builder, see these Oracle resources:
Oracle Warehouse Builder Installation and Administration Guide
Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide
For information about data warehousing, see these Oracle resources:
For information about licensing options, see:
The following text conventions are used in this document:
Convention | Meaning |
---|---|
boldface | Boldface type indicates graphical user interface elements associated with an action, or terms defined in text or the glossary. |
italic | Italic type indicates book titles, emphasis, or placeholder variables for which you supply particular values. |
monospace | Monospace type indicates commands within a paragraph, URLs, code in examples, text that appears on the screen, or text that you enter. |