7 Extracting Data from SAP Applications

Many companies implement the SAP Enterprise Resource Planning (ERP) system and Oracle Warehouse Builder enables easy access to the data in these SAP systems.

This chapter describes how you can extract data from SAP systems. It describes why you require an SAP connector, how to import metadata from SAP tables, use them in a mapping, generate ABAP code for the mappings, and deploy them to an SAP system. The chapter also describes the various methods by which you can extract data from the SAP system and load this data into a target table on Oracle Warehouse Builder system.

This chapter contains the following topics:

Why SAP Connector

The SAP R/3 system operates differently compared to SQL-based systems like Oracle E-Business Suite and Oracle's PeopleSoft systems.

The major differences include:

  • The native data manipulation language is ABAP, which is a proprietary SAP language.

  • Table names are cryptic compared to those in SQL-based ERP systems.

  • In addition to database tables, SAP contains logical tables called pool tables and cluster tables. These tables contain multiple physical tables and must be managed differently from SQL-based tables.

The SAP connector assists you in managing all these issues. Furthermore, the SAP connector enables you to follow the administrative and security processes of the SAP environment.

Supported SAP Versions

For information about the SAP R/3 versions supported by Oracle Warehouse Builder 11g, log in to https://support.oracle.com/, and go to the Certify link.

Overview of SAP Objects

This section provides a brief overview of the different types of tables in SAP, and how data is organized within an SAP system. The section consists of the following topics:

SAP Object Types

With the SAP connector, you can import metadata definitions for the following SAP table types:

  • Transparent: A transparent table is a database table that stores data. You can access the table from non-SAP systems as well, for example, using SQL statements. However, Oracle Warehouse Builder uses ABAP code to access transparent tables.

  • Cluster: A cluster table is usually used to store control data. It can also be used to store temporary data or documentation. Because cluster tables are data dictionary tables and not database tables, you can access these tables only by using ABAP.

  • Pooled: This is a logical table that must be assigned to a table pool in the database. A table pool consists of multiple pooled tables. A pooled table is used to store control data such as program parameters. You require ABAP code to access pooled tables.

SAP Business Domains

SAP application systems logically group tables under different business domains. In SAP, a business domain is an organizational unit in an enterprise that groups product and market areas. For example, the Financial Accounting business domain represents data describing financial accounting transactions. These transactions might include General Ledger Accounting, Accounts Payable, Accounts Receivable, and Closing and Reporting.

When you import SAP definitions, you can use a graphical navigation tree in the Business Domain Hierarchy dialog box to search the business domain structure in the SAP source application. This navigation tree enables you to select SAP tables from the SAP application server.

Overview of the Interaction Between Oracle Warehouse Builder and SAP

Moving data from the SAP system to an Oracle database using Oracle Warehouse Builder consists of the following tasks:

  1. Connecting to the SAP system.

  2. Importing metadata from SAP data objects.

  3. Creating an extraction mapping in Oracle Warehouse Builder that defines:

    • The SAP source tables from which data is to be imported.

    • The transformation operators that operate on the source tables to extract data based on certain criteria.

    • The target table in Oracle Warehouse Builder to store the data imported from the SAP source tables.

  4. Deploying the mapping.

    This creates the ABAP report for the mapping.

  5. Starting the mapping.

    This results in the following sequence of tasks, all of which are performed automatically by Oracle Warehouse Builder:

    • Transfer of the ABAP report to the SAP server.

    • Compiling of the ABAP report.

    • Execution of the ABAP report, which results in the generation of a data file (this file has a .dat extension).

    • Transfer of the data file to the Oracle Warehouse Builder server using FTP.

    • Loading data from the data file into the target table in Oracle Warehouse Builder. The loading takes place using SQL*Loader.

SAP Function Modules

To access SAP data from non-SAP systems, you typically use a function module to execute an ABAP program that extracts the data. A function module in SAP is a procedure that is defined in a special ABAP program known as a function group. After the function group is defined, the function module can then be called from any ABAP program.

SAP contains a predefined function module called RFC_ABAP_INSTALL_AND_RUN to execute ABAP report. To upload the Oracle Warehouse Builder-generated ABAP report and execute it in SAP, you need access rights to this function module.

Alternatively, you can ask the SAP administrator to create a customized function module that executes a specific ABAP program. You can then use this function module to execute the ABAP report generated by Oracle Warehouse Builder.

Data Retrieval Mechanisms

Data retrieval from the SAP system can be "Completely Managed by Oracle Warehouse Builder", "Managed by Oracle Warehouse Builder with SAP Verification", or "Manual". This depends on whether the SAP administrator provides the Oracle Warehouse Builder user with access rights to the predefined function module RFC_ABAP_INSTALL_AND_RUN or creates a customized function module to execute the ABAP report.

Completely Managed by Oracle Warehouse Builder

In this mechanism, Oracle Warehouse Builder has access to upload and execute the generated ABAP using the default function module RFC_ABAP_INSTALL_AND_RUN, and to use FTP to import the generated data file from the SAP system.

Thus the entire process of retrieving data from the SAP system and creating a target table is managed by the Oracle Warehouse Builder and can be completely automated. It is therefore the simplest method of data retrieval. See "Automated System" for more details on implementing this data retrieval mechanism.

Managed by Oracle Warehouse Builder with SAP Verification

In this mechanism, as an Oracle Warehouse Builder user, you do not have access rights to the default function module RFC_ABAP_INSTALL_AND_RUN that executes the ABAP report in the SAP system. Instead, the SAP administrator first verifies the ABAP report that you generate using Oracle Warehouse Builder, and then creates a customized function module to execute this ABAP report. You can then run the ABAP code on the SAP system using this customized function module.

See "Semi-Automated System" for more details about implementing this data retrieval mechanism.

Manual

In this mechanism, as a Oracle Warehouse Builder user, you cannot directly run the ABAP code on the SAP system. Instead, you generate the ABAP report for the mapping, and send it to the SAP administrator, who runs the code on the SAP system. You then import the generated data file using FTP and load the target table.

The tasks involved in retrieving data using FTP and creating the Oracle table are implemented using a Process Flow. See "Manual System" for more details about implementing this system.

Implementing the SAP Data Retrieval Mechanism

As a Oracle Warehouse Builder user, you must be aware of certain restrictions while trying to extract data from an SAP system.

Because the SAP and Oracle Warehouse Builder systems are totally independent systems, as an Oracle Warehouse Builder user, you may only have restricted access rights to the SAP data (especially in the production environment). You therefore have to interact with the SAP administrator to extract data from the system.

Access privilege to the SAP system is most often determined by whether it is the development, test, or the production environment. Each of the data retrieval mechanisms can be implemented in the development, test, or production environment depending on the privileges granted by the SAP system administrator.

Development Environment

Typically, in the development environment, the SAP administrator gives you access rights to use the predefined function module RFC_ABAP_INSTALL_AND_RUN. Therefore, in this environment, you can implement a completely "Automated System" for data retrieval.

Test and Production Environment

Typically, in the test and production environments, the SAP administrator may not give you access rights to use the predefined function module RFC_ABAP_INSTALL_AND_RUN. Instead, the SAP administrator verifies the ABAP report, and either creates a customized function module that you can use, or runs the ABAP report on the SAP system, and enables you to extract the resultant data. You can therefore implement either a "Semi-Automated System" or a "Manual System" for data retrieval.

A typical data retrieval system may therefore consist of any of the three mechanisms implemented in the different environments.

Scenario 1

You run the automated system in the SAP development environment. After you verify the ABAP report in this environment, you then move the ABAP report to the SAP test environment and test the code using a customized function module. You then finally move this to the SAP production environment.

This implementation is recommended by Oracle, as it automates and simplifies the data retrieval task.

Scenario 2

Depending on the access rights to the development, test, and production environments, you implement any one of the data retrieval mechanisms in each of the environments.

The following sections provide details of the tasks involved in retrieving data from an SAP system:

  1. "Connecting to the SAP System"

  2. "Importing Metadata from SAP Tables"

  3. "Creating SAP Extraction Mappings"

  4. "Deploying and Executing SAP ABAP Mappings".

Connecting to the SAP System

To connect to the SAP system from Oracle Warehouse Builder, you must use certain SAP-specific DLL files. After you establish connection, you can then import metadata from SAP tables into SAP modules in Oracle Warehouse Builder.

This section contains the following topics:

Required Files for SAP Connector

Different sets of files are required depending on whether you are working on a Windows or a UNIX system.

Files Required in Windows

The SAP connector requires a DLL file named librfc32.dll to use remote function calls on the client computer. You must copy librfc32.dll to the location specified in java.library.path on your client system.

To find this location, click MyComputer, Properties, and then click Advanced. Next click Environment Variables, and under System variables, check the locations specified for the variable Path.

You can copy the librfc32.dll file to any one of the multiple locations specified in Path. One of the locations correspond to OWB_HOME, and is therefore the preferred location. This location is usually OWB_HOME\owb\bin.

See Table 7-1 for the list of files required in Windows.

Table 7-1 Required Files for Windows

Required Files Path Description

librfc32.dll

OWB_HOME\owb\bin

This file is available on the SAP Application Installation CD.

sapjcorfc.dll

OWB_HOME\owb\bin

Copy this file to the same location where you placed librfc32.dll.

sapjco.jar

OWB_HOME\owb\lib\int

  

Restart the client after copying these files.

Files Required in UNIX

The SAP connector requires a DLL file named librfccm.so to use remote function calls on the client computer. You must copy this file to the location specified by the UNIX environment variable path LD_LIBRARY_PATH on your client system.

By default, OWB_HOME/owb/bin/admin is the location specified in LD_LIBRARY_PATH. If it is not, then ensure that you add OWB_HOME\owb\bin\admin to LD_LIBRARY_PATH.

See Table 7-2 for the list of files required in UNIX.

Table 7-2 Required Files for UNIX

Required Files Path Description

librfcccm.so

OWB_HOME\owb\bin\admin

This file is available on the SAP Application Installation CD.

libsapjcorfc.so

OWB_HOME\owb\bin\admin

Copy this file to the same location where you placed librfcccm.so.

sapjco.jar

OWB_HOME\owb\lib\int

  

Restart the client after copying these files.

Note:

Different versions of SAP R/3 might require different versions of the DLL, SO, and JAR files. The correct versions are available in the SAP installation CD. The files can also be downloaded from:

http://service.sap.com/patches

Troubleshooting Connection Errors

The most common errors while connecting to the SAP system are listed in Table 7-3:

Table 7-3 SAP Connection Errors

Error Message Possible Reason

Connection failed.You are not authorized to logon to the target system (error code 1).

Incorrect user name or password to connect to the SAP server.

Connection failed.

Connect to SAP gateway failed.

Incorrect application server, system number, or client details.

Some Location Details are missing.

Please verify the location information is completely specified.

Missing DLL files, or DLL files placed in the wrong location.

Missing saprfc32.dll

Missing saprfc32.dll file, or file placed in the wrong location.

Note:

If you create the SAP source module and import SAP tables but cannot see the columns in the tables, then you have an incompatible librfc32.dll file. Download the correct version of the DLL file from the SAP Web site.

Creating SAP Module Definitions

Use the Create Module Wizard to create the SAP source module that stores data from the SAP source.

To create a SAP Module:

  1. Right-click SAP and select New SAP.

    The Create Module Wizard is displayed.

  2. On the Name and Description page, provide a name for the SAP module. Select the module status and optionally also provide a description. Click Next.

  3. On the Connection Information page, either select from an existing location or click Edit to open the Edit SAP Location dialog box. Specify the details as described in "Connecting to the SAP System". Click Next.

  4. On the Summary page, click Finish.

    A new sap module is now available on the Projects Navigator.

Note:

Before you create a SAP location, ensure that you have all the necessary information. You can provide the location information either while creating the module or before importing metadata into the module. You require the following information to create the location: server name, user name, password, system number, and client number. Obtain these details from the system administrator.

When you set the connection information, you can choose the connection types described in the subsequent sections.

Remote Function Call (RFC)

A remote function call enables you to call a function module on a remote system. This method requires specific IP Address information for the SAP application server.

SAP Remote Function Call (SAPRFC.INI)

You can also specify the connection information in a file called SAPRFC.INI, and copy this file to the following location: OWB_HOME\owb\bin\admin.

Using the SAPRFC.INI file requires prior knowledge of ABAP parameters, because you must specify the values for certain parameters to make the SAP connection, and is not the recommended connection method if you are not familiar with ABAP.

Note:

The SAPRFC.INI file comes with the SAP installation CD.

The Create Module Wizard creates the module for you based on the metadata contained in the SAP application server.

Connecting to the SAP System

  1. Select one of the following connection types:

    • Remote Function Call (RFC)

      This is the recommended connection type, and is selected by default in Oracle Warehouse Builder.

    • SAP Remote Function Call (SAPRFC.INI)

    For more information about these connection types, see "Creating SAP Module Definitions".

  2. Enter the connection information in the appropriate fields. The fields displayed on this page depend on the connection type you choose.

    Note:

    Ensure that you have copied the DLL files to the right location. For more information, see "Required Files for SAP Connector".

    You must obtain the connection information for your SAP application server from the system administrator before you can complete this step.

    RFC Connection type requires the following connection information:

    Application Server: The alias name or the IP address of the SAP application server.

    System Number: The SAP system number. This must be provided by the SAP system administrator.

    Client: The SAP client number. This must be provided by the SAP system administrator.

    User Name: The user name with access rights to the SAP system. This name is supplied by the SAP system administrator.

    Language: EN for English or DE for German. If you select DE, the description text is displayed in German and all other text is displayed in English.

    The SAPRFC connection type requires the following connection information:

    RFC Destination: Enter the alias for the SAP connection information.

    In addition, both the connection types require the following connection information if the ABAP report is to be executed in SAP using a function module and the data file is to be transferred by FTP to Oracle Warehouse Builder:

    Host Login User Name: A valid user name on the system that hosts the SAP application server. This user must have access rights to copy the data file using FTP.

    FTP Directory: The directory in the SAP server that stores the data file generated when the ABAP report is executed. For systems where the FTP directory structure is identical to the operating system directory structure, this field can be left blank. For systems where the file system directory structure is mapped to the FTP directory structure, enter the FTP directory path that is mapped to the staging file directory in the file system directory structure. For example, on a computer that runs Windows, if the staging file directory "C:\temp" is mapped to "/" in the FTP directory structure, then enter "/" in this field.

    Execution Function Module: In the SAP instance, if a remote function module other than the SAP delivered function module RFC_ABAP_INSTALL_AND_RUN is used to remotely execute ABAP reports through RFC connections, then enter the remote function module name here.

  3. Click Test Connection to verify that the connection information you provided is correct.

  4. Click OK to go back to the Connection Information page of the Create Module Wizard.

Figure 7-1 Edit SAP Location Dialog Box

Description of Figure 7-1 follows
Description of "Figure 7-1 Edit SAP Location Dialog Box"

Importing Metadata from SAP Tables

After you establish a connection with the SAP server, you can import metadata from SAP tables.

This section contains the following topics:

Importing SAP Metadata Definitions

After creating the SAP source module, you can import metadata definitions from SAP tables using the Import Metadata Wizard. This wizard enables you to filter the SAP tables to import, verify those tables, and reimport them. You can import metadata for transparent tables, cluster tables, or pool tables.

To import SAP metadata:

  1. From the Projects Navigator, expand the Applications node.

  2. Right-click the SAP source module into which you want to import metadata and select Import.

    Oracle Warehouse Builder displays the Welcome page for the Import Metadata Wizard.

  3. Click Next.

  4. Complete the following tasks:

Filtering SAP Metadata

You can filter objects to import by business domain or by text strings. Select a filtering method and click Next.

Filtering SAP Metadata by Business Domain

  1. Select Business Domain and click Browse to display the SAP R/3 Business Domain Hierarchy dialog box.

    The Import Metadata wizard displays the Loading Progress dialog box while it is retrieving the business domains.

  2. The Business Domain Hierarchy dialog box lists the available SAP business domains.

    Note:

    It may take a few minutes to list the SAP business domains depending on factors such as the network location of the SAP application server, the type of LAN used, and the size of the SAP application database.

    Use the Business Domain Hierarchy dialog box to select the SAP business domains that contain the metadata tables you want to import.

  3. Select a folder and click Show Tables to view the tables available in a business domain.

    The Import Metadata Wizard displays a list of tables in the selected business domain in the Folder dialog box.

  4. Review this dialog box to ensure that you are selecting the required tables.

    Some business domains can contain more than 1000 tables. Importing such a large amount of metadata can take time, depending on the network connection speed and the processing power of the source and target systems.

  5. Click OK.

    The wizard displays the Filter Information page with the SAP business domain displayed in the Business Domain field.

Filtering SAP Metadata by Text String

  1. Select Text String, where object and use the Name matches or Description matches entry field to enter a string and obtain matching tables from the SAP data source.

    The Description matches field is case sensitive, the Name matches field is not.

    Create a filter for object selection by using the wildcard characters % for zero or more matching characters, and _ for a single matching character.

    For example, to search the business domain for tables whose descriptions contain the word CURRENCY, select Description matches and enter %CURRENCY%. You can also search for tables by their names.

  2. Specify the number of tables you want to import in the Maximum number of objects displayed field.

Selecting Objects for Metadata Import

The Object Selection page contains a description of the tables and enables you to select the tables you want to import into the SAP module.

To select the tables:

  1. Move the tables from the available list to the selected list.

    The Import Metadata Wizard also enables you to choose whether you want to import tables with foreign key relationships for each table that you choose to import. You can select one of the following:

    None: Import only the tables in the Selected list.

    One Level: Import the tables in the Selected list and any tables linked to them directly through a foreign key relationship.

    All Levels: Import the tables in the Selected list and all tables linked to them through foreign key relationships.

  2. Click Next.

    If you select One Level or All Levels, the Confirm Import Selection dialog box is displayed.

    Review this dialog box to ensure that you are selecting the required tables.

  3. Click OK.

    The selected tables appear in the Selected list of the Table Selection page.

  4. Click Next.

    The wizard displays the Summary and Import page.

Reviewing Import Summary

The wizard imports the definitions for the selected tables from the SAP application server, stores them in the SAP source module, and then displays the Summary and Import page.

You can edit the descriptions for each table by selecting the Description field and entering a new description.

Review the information about the Summary and Import page and click Finish.

The SAP Connector reads the table definitions from the SAP application server and creates the metadata objects in the workspace.

The time it takes to import the SAP metadata into the workspace depends on the size and number of tables and the connection between the SAP application server and the workspace. It is a best practice to import small batches of tables to allow better performance.

When the import completes, the Import Results dialog box displays. Click OK to finish importing metadata.

Reimporting SAP Tables

To reimport SAP tables, follow the importing procedure using the Import Metadata Wizard. Before starting the import, the wizard checks the source for tables with the same name as those you are importing. The tables that have been imported appear in bold in the Object Selection page. On the Summary and Import page, the Action column indicates that these tables are reimported. The wizard then activates the Advanced Synchronize Options button so that you can control the reimport options.

Note:

To undo the reimport, click Undo. This ensures that no changes are made to the existing metadata.

Analyzing Metadata Details

With SAP tables, you cannot view the data after you import the metadata from these tables. However, you can get good insight about the data that is stored in the tables by viewing the "Column Descriptions" and the "Constraints Details".

Column Descriptions

You can view the column description of each of the columns in a table. This is valuable because the column names in SAP can be nondescriptive, and difficult to interpret if you have not previously seen the data in the table.

To view the descriptions, double-click the table to open the data object editor for the table, and then click the Columns editor.

The description for the columns of the table are visible as shown in Figure 7-2.

Figure 7-2 The Columns Editor with the Description for the Columns of SAP Table

Description of Figure 7-2 follows
Description of "Figure 7-2 The Columns Editor with the Description for the Columns of SAP Table"

Constraints Details

The other benefit of the data object editor is that you can get information about the primary and foreign keys within the table. To view the key constraints, click the Constraints editor.

Note:

It is also a useful practice to display the business names of the SAP tables in the Projects Navigator. Business names provide a description of the tables and are therefore more intuitive than the physical names. To view the business names for tables in Oracle Warehouse Builder, from the main menu, click Tools, Preferences, OWB, Naming, and then select Business Names in the Naming Mode field.

Creating SAP Extraction Mappings

After importing metadata from SAP tables, you must define the extraction mapping to extract data from the SAP system.

Note:

For details of mappings in Oracle Warehouse Builder, see Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide.

Defining the SAP Extraction Mapping

You can use the Mapping Editor to create a mapping containing SAP tables. Creating a mapping with SAP tables is similar to creating mappings with other database objects. However, there are restrictions on the operators that can be used in the mapping. You can only use table, filter, joiner, and mapping input parameter mapping operators in a mapping containing SAP tables.

A typical SAP extraction mapping consists of one or more SAP source tables (transparent, cluster, or pooled), one or more filter or joiner operators, and a non-SAP target table (typically an Oracle table) to store the imported data.

This is as shown in Figure 7-3.

Note:

The source table is always an SAP table. You cannot have both SAP and non-SAP (Oracle) source tables in a mapping, but the staging table is an Oracle table.

Figure 7-3 SAP Extraction Mapping

Description of Figure 7-3 follows
Description of "Figure 7-3 SAP Extraction Mapping"

In this mapping, the input parameter holds a date value, and the data from table BKPF is filtered based on this date. Because this is defined as an input parameter, you can enter a value for the date when the mapping is run. The joiner operator enables you to join data from multiple tables, and the combined data set is stored in a staging table.

This section contains the following topics:

Adding SAP Tables to the Mapping

To add an SAP table to a mapping:

On the mapping editor, drag and drop the required SAP table onto the Mapping Editor canvas.

The editor places a table operator on the mapping canvas to represent the SAP table.

Setting the Loading Type

Use the operator properties panel of the Mapping Editor to set the SQL*Loader properties for the tables in the mapping.

To set the loading type for the SAP Source Table:

  1. On the Mapping Editor, select the SAP source table. The Table Operator Properties panel displays the properties of the SAP table operator.

  2. Select a loading type from the Loading Type list. With ABAP code as the language for the mapping, the SQL*Loader code is generated as indicated in Table 7-4.

    Table 7-4 SQL*Loader Code Generated in ABAP

    Loading Type Resulting Load Type in SQL*Loader

    INSERT

    APPEND

    CHECK/INSERT

    INSERT

    TRUNCATE/INSERT

    TRUNCATE

    DELETE/INSERT

    REPLACE

    All other types

    APPEND

Setting Configuration Properties for the Mapping

  • Use the Configuration Properties dialog box to define the code generation language as described in "Setting the Language Parameter".

  • Set ABAP specific parameters, and the directory and initialization file settings in the Configuration Properties dialog box as described in "Setting Run-time Parameters".

Setting the Language Parameter

The language parameter enables you to choose the type of code you want to generate for a mapping. For mappings containing SAP source tables, Oracle Warehouse Builder enables you to select either PL/SQL or ABAP.

If the SAP system uses a non-Oracle database to store data, then you must select ABAP to generate code. If the SAP data is stored on an Oracle database, then you can specify PL/SQL. However, with PL/SQL, you cannot extract pool or cluster tables. Therefore, in all instances it is desirable to set the language to ABAP.

Setting Run-time Parameters

With the language set to ABAP, you can expand the Runtime Parameters node in the Configuration Properties dialog box to display settings specific to ABAP code generation.

Some of these settings come with preset properties that optimize code generation. It is recommended that these settings be retained, because altering them may slow down the code generation process.

The following run-time parameters are available for SAP mappings:

  • ABAP Report Name: Specifies the name of the ABAP report generated by the mapping. This is required only when you run a custom function module to execute the ABAP report.

  • Background Job: Select this option to run the ABAP report as a background job in the SAP system. Enable this option for the longer running jobs. Foreground batch jobs that run for a long duration are considered to be hanging in SAP after a certain time. Therefore it is ideal to have a background job running for such extracts.

  • Control File Name: By default, the control file name equals the data file name specified in the Data File Name field. This implies that the ABAP code generates a single control file that contains both, the SQL*Loader control information and the data (since the log files are the same). You can assign different file names for the control file and the data file, in which case different files are generated for the control information and data, and both the files are transferred by FTP.

  • Data File Name: Specifies the name of the data file that is generated when the ABAP report is executed in the SAP system.

  • File Delimiter for Staging File: Specifies the column separator in a SQL data file.

  • Include FTP: If this is set to true, then the data file is moved to the Oracle Warehouse Builder system using FTP. If this is set to false, then the file is not transferred.

  • INSTALL ONLY: When you set this to true and run the mapping, the ABAP report gets installed on the SAP system, but does not get executed. To execute the ABAP report, you must run the mapping again and use the execution function module. Setting this option to true enables you to generate the mapping, install the ABAP report on the SAP system, and subsequently make modifications to the ABAP report and then execute it. You can use this option when you want the ABAP report to be stored on the SAP system. If this option is set to false, then the ABAP report is loaded and executed on the SAP system. However, the ABAP report is not stored on the SAP system after it is executed.

  • SAP Location: Specifies the location of the SAP instance from where the data can be extracted.

  • SAP System Version: Specifies the SAP system version number to which you want to deploy the ABAP report. The characteristics of the generated ABAP report depends on the version number. For MySAP ERP and all other versions, select SAP R/3 4.7. Different ABAP report is generated for versions before 4.7.

  • SQL Join Collapsing: If this is set to true, then it specifies the following hint, if possible, to generate ABAP report.

    SELECT < > INTO < > FROM (T1 as T1 inner join T2 as T2) ON <condition >

  • Staging File Directory: Specifies the location of the directory in the SAP system where the data file generated by ABAP report resides.

  • Timeout: This specifies the duration, in seconds, for which Oracle Warehouse Builder waits for the SAP system to execute the ABAP report and return a data file. If the SAP system completes the execution within this duration, then Oracle Warehouse Builder automatically retrieves the resultant data file from the SAP system. If the execution takes longer than the specified duration, then you must manually retrieve the data file.

Setting the Join Rank

You must set this parameter only if the mapping contains the joiner operator, and you want to explicitly specify the driving table. Unlike SQL, ABAP code generation is rule-based. Therefore, you must design the mapping in such a way that the tables are loaded in the right order. Or, you can explicitly specify the order in which the tables are to be joined. From the Configuration Properties dialog box, expand Table Operators, and then for each table, specify the Join Rank. The driving table must have the Join Rank value set to 1, with increasing values for the subsequent tables.

You can also let Oracle Warehouse Builder decide the driving table, and the order of joining the other tables. In such cases, do not enter values for Join Rank.

Deploying and Executing SAP ABAP Mappings

After designing the extraction mapping, you must validate, generate, and deploy the mapping, as you do with all mappings in Oracle Warehouse Builder.

To generate the script for the SAP mapping:

  1. Right-click the SAP mapping and select Generate.

    The Generation Results window is displayed.

  2. On the Script tab, select the script name and select View Code.

    The generated code is displayed in the code viewer.

    You can edit, print, or save the file using the code editor. Close the code viewer to return to the Generation Results window.

  3. To save the file, click Save as File and save the ABAP program to the hard drive.

After you generate the SAP mapping, you must deploy the mapping to create the logical objects in the target location.

To deploy an SAP mapping, right-click the mapping and select Deploy. You can also deploy the mapping from Control Center Manager.

For detailed information about deployment, see Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide.

When an SAP mapping is deployed, an ABAP mapping is created and stored in the Oracle Warehouse Builder run time schema. Oracle Warehouse Builder also saves the ABAP file under OWB_HOME\owb\deployed_files, where OWB_HOME is the location of the Oracle home directory of your Oracle Warehouse Builder installation.

If you are using the Oracle Warehouse Builder installation that comes with Oracle Database, then this equals the database home.

Depending on whether data retrieval from the SAP system is fully automated, semi-automated, or manual, you must perform the subsequent tasks described in the following sections:

Automated System

In a completely automated system, as an Oracle Warehouse Builder user you have access to the predefined function module in the SAP system. It enables you to execute any ABAP report and extract data directly from the SAP system without being dependent on the SAP administrator, as shown in Figure 7-4.

Figure 7-4 Automated Data Retrieval

Description of Figure 7-4 follows
Description of "Figure 7-4 Automated Data Retrieval"

You can automate the process of transferring the ABAP report to the SAP system and generating the data file on the SAP system. After the data file is generated, Oracle Warehouse Builder uses FTP to transfer the data file to the Oracle Warehouse Builder system. The data file is then loaded into the target file by using SQL*Loader.

An automated system works as follows:

  1. You design the extraction mapping and generate the ABAP report for this mapping.

  2. Before deploying the mapping, you must ensure that the following configuration properties are set for the mapping:

    • ABAP Report Name: The file that stores the ABAP code generated for the mapping.

    • SAP Location: The location on the SAP system from where data is extracted.

    • Data File Name: The name of the data file to store the data generated by the execution of ABAP report.

    Also ensure that you have provided the following additional connection details for the SAP location:

    • Execution Function Module: Provide the name of the predefined SAP function module. When executed, this function module takes the ABAP report name as the parameter, and executes the ABAP code.

    • FTP Directory: A directory on the SAP system. The data file generated upon the execution of the function module is saved to this directory. Oracle Warehouse Builder uses FTP to transfer the file from this directory to the Oracle Warehouse Builder system. This requires an FTP server to be located in the SAP system.

    • Also provide a user name with Write permissions on the FTP directory.

  3. You then start the mapping, after which the following tasks are automatically performed:

    • Oracle Warehouse Builder deploys the ABAP report and the function module RFC_ABAP_INSTALL_AND_RUN is used to load and execute the ABAP report in the SAP system.

      The ABAP report is sent to the SAP system using a "Remote Function Call (RFC)".

  4. In the SAP system, the ABAP report extracts data from the source tables and creates a data file.

    This data file is stored in the location specified by the Staging File Directory. See "Setting Run-time Parameters" for details of the staging file directory.

  5. Oracle Warehouse Builder uses FTP to transfer this data file back to the Oracle Warehouse Builder system.

    The file is stored in the location specified in the FTP Directory field.

  6. Using SQL*Loader, Oracle Warehouse Builder loads the target table specified in the mapping with the data from the data file.

The advantage of this system is that you can create a fully automated end-to-end solution to extract SAP data. As a user, you create the extraction mapping, and run it from Oracle Warehouse Builder, which then creates the ABAP report, sends it to the SAP system, extracts the resultant data file, and loads the target table with the extracted data.

Semi-Automated System

In a semi-automated system, as a Oracle Warehouse Builder user, you do not have access to the predefined function module RFC_ABAP_INSTALL_AND_RUN, and therefore cannot use this function module to execute ABAP report. Instead, you must create an extraction mapping, deploy it, and then send the ABAP report to the SAP administrator who verifies the code before allowing you to run it in the SAP system, as shown in Figure 7-5.

Figure 7-5 Semi-Automated Implementation

Description of Figure 7-5 follows
Description of "Figure 7-5 Semi-Automated Implementation"

A semi-automated system works as follows:

  1. You design the extraction mapping and generate the ABAP report for this mapping.

    You can then test this report in the development environment.

  2. You then send the ABAP report to the SAP administrator, who tests the report, and loads it to the SAP repository in the production environment.

  3. The SAP administrator can create a new report or use the same report that you send.

  4. If the SAP administrator creates a new report, then obtain the name of the new report and use it in your mapping to extract data from the production environment.

  5. Before you run the mapping in the production environment, ensure that you have set the following configuration properties for the mapping:

    • ABAP Report Name: The SAP administrator provides the name of the ABAP report after verifying the code. You must then use this report name to extract data.

    • SAP Location: The location on the SAP system from where data is extracted.

    • Data File Name: Name of the data file to store the data generated during execution of ABAP report.

    Also ensure that you have provided the following additional connection details for the SAP location:

    • Execution Function Module: Provide the name of the custom function module created by the SAP administrator. On execution, this function module takes the ABAP report name as the parameter, and executes the ABAP code. You must obtain the function module name from the SAP administrator.

    • FTP Directory: A directory on the SAP system. The data file generated by the execution of the ABAP report is saved to this directory. Oracle Warehouse Builder imports the data file using FTP. The FTP server resides on the SAP system.

    • Also provide a user name with Read permissions on the FTP directory.

  6. In the production environment, when you run the mapping, Oracle Warehouse Builder sends the ABAP report name and the custom function module to the SAP system using a "Remote Function Call (RFC)".

  7. In the SAP system, the ABAP report gets executed and a data file is generated. The ABAP report gets executed only if the ABAP report name and the function module are available.

    This data file is stored in the location specified by the Staging File Directory.

  8. Oracle Warehouse Builder imports the data file using FTP. An FTP server must be available on the SAP server.

  9. Oracle Warehouse Builder uses SQL*Loader to load the target table with data from the data file.

Manual System

In a manual system, your role as an Oracle Warehouse Builder user is restricted to generating the ABAP report for the mapping, and sending the ABAP report to the SAP administrator. The tasks involved in this system are:

  1. You create an extraction mapping, and generate the ABAP report for the mapping.

  2. While designing the mapping, ensure that you specify the Data File Name to store the data file.

  3. You send the ABAP report to the SAP administrator.

  4. The SAP administrator executes the ABAP report in the SAP system.

  5. On execution of the code, a data file is generated.

You can then create a process flow to import the data file. The process flow may typically consist of the activities shown in Figure 7-6. For more details on implementing process flows, see Oracle Warehouse Builder Data Modeling, ETL, and Data Quality Guide.

Figure 7-6 Process Flow to Import SAP Data

Description of Figure 7-6 follows
Description of "Figure 7-6 Process Flow to Import SAP Data"

  1. A File Exists activity checks for the availability of the data file.

  2. If the file exists, then an FTP activity transfers the file to the Oracle Warehouse Builder system.

  3. If the file does not exist, then it must wait till the file is made available, and then perform an FTP.

  4. Using SQL*Loader, the target table is loaded with data from the data file.

In most production environments, the SAP administrator may not allow any other user to access the SAP system. In such cases, implementing the manual system may be the only viable option.

Using SQL*Loader in the Process Flow

To use SQL*Loader in the process flow, insert a SQL*Plus activity as shown in Figure 7-6. To use the SQL*Loader, use the HOST command. Once you insert the SQL*Plus activity, insert the following value for SCRIPT:

HOST sqlldr ${Target.User}/${Target.Password} CONTROL=${Working.RootPath}\C.CTL
quit

Insert the relevant value for the control (.ctl) file name.

Then configure the path settings for the SQL*Plus activity. Right-click the process flow and select Configure.

Under SQL*Plus Activities, expand the SQLPLUS node and provide the required values under Path Settings as shown in Figure 7-7.

Deployed Location refers to the location of the target table. Working location refers to the location of the control file.

Figure 7-7 Path Setting for the Process Flow

The path settings for the SQL PLUS activity.