(v23) Source Creation Tab

Source Creation Tab

Use the Source Creation tab to define new source definitions, edit source definitions, and define the data entity or entities that make up your data source.

Note: You must be logged in as user with the Administer Sources or Create New Data Sources privilege to see the Source Creation tab, or have Read and Write permissions on the source.

Note: When you upload a flat file, Select Schema and Select Entity fields are not present, and the option to create Custom SQL is not available. You can instead Select File, Edit File, and configure API Endpoints.

The general structure of this tab includes:

• In the upper portion of the tab, you can edit and update the Name and Description of this source in the Source Definition work area.
• In the lower portion of the tab, you can add and manage data entities in the Source Definition work area.
• In the bottom portion of the tab, you can manage Joins and Join Settings. The Join Definition work area is visible after you've added multiple data entities to create a Fusion data source.

Source Creation Tab

Define basic information about this data source.

Note: Replaces the General tab in the data source configuration wizard deprecated in a previous release of Composer.

Source Definition

Add and edit the unique Name and optional Description of your source.

Data Entity Definition

Note: Replaces the Tables/Indices tab in the data source configuration wizard deprecated in Composer v7.10.

Add and edit data entities for this source. After adding a data entity or making changes, select Save Source to save your changes, or Preview Source to preview your data.

Data Entities

Select Add to add a new data entity from a connection or an uploaded file.

• Select From Connection to add a data entity from an existing data source.

• Select From File to add a data entity from a file.

Data Entity Details - From Connection

Define a unique Data Entity Name, then define the details for how the information is accessed and presented.

Note: Depending on the your data source, different options are available to define your source.

When you select From Connection, details you can define can include:

• Data Entity Name - The unique name for this data entity.

• Select Connection: Select a connection for this data entity. You will only see connections you have access to.

• Select Schema: If a Schema is available for your connection, you can select a schema to filter a list of entities for this connection. See Connector Feature Support.

• Select Indices: ElasticSearch connections support Indices: select one or more to include.

  • Indice Selection: Select Manually to create a merged list of fields from selected indices, or Automatically to select a pattern that automatically selects indices.

    

• Existing Entity and Custom SQL:

  • Select Existing Entity to use an available entity from your source.

  • Custom SQL: Select to define a custom SQL to retrieve the data you want from your source. Select Entity is not available if you select Custom SQL.

• Select Entity: Select an available entity. A list of all native fields from this entity populates Available Fields and the Preview table.

  • Available Fields: All fields available from this data source are included by default. Disable (uncheck) specific fields to exclude them from the data source configuration. The fields, when disabled, are not included in any work areas of the data source configuration. If you attempt to remove a field in use by a visual or other object, Composer will prevent you from saving your changes to the source configuration.

  • Data Entity Preview table: A preview of the data from the source for all fields, even if deselected in Available Fields.

  • Source Preview table: A preview of data from the source as defined in Available Fields. Select Preview Source to preview your selected data.

    In the example shown here, the date field is deselected in Available Fields. It is visible in the Data Entity Preview table, but not the Source Preview table.

    

Select Apply to apply your changes, or add another data entity to create joins for a Fusion source.

Custom SQL

When you select this option, a Custom SQL editing pane opens that you can use to write and run your SQL query. After you have run a successful query, the results populates Available Fields and the Preview table, and you can Apply your changes. You can't save invalid SQL.

Table visuals and Details dialogs display fields in the order they are retrieved from the source. When you create a source using custom SQL, your fields are shown in the order you specify.

Important: Custom SQL queries are a powerful tool for performing complex data queries. However, be careful when creating custom SQL queries because it is easy to define a heavy query or a query that may overwhelm your database. Use this feature carefully.

Variables (specified as custom user attributes) can be inserted in custom SQL. See Specify Custom User Attributes. In addition, you can use a vertical bar (|) in the SQL to separate the custom attribute name from a default value used for user definitions that do not have the custom user attribute defined. For example, the following custom SQL uses the value of the state customer user attribute to filter data source data for records from whatever state the user's state custom user attribute is set to. If a state custom user attribute is not defined for a user, a default of Alabama is used.

SELECT * FROM Orders WHERE state = '${User.state|Alabama}'

Data Entity Details - From File

Define a unique Data Entity Name, then define the details for how the information is accessed and presented.

When you select From File, details you can define can include:

• Data Entity Name - The unique name for this data entity.

• Select File: Select an available uploaded file for this data entity. A list of all native fields from this entity populates Available Fields and the Preview table. Use Upload New File to add files to this source. See Manage File Uploads in Composer.

• API Endpoints: Select for more information about appending data, replacing data, or deleting data from your file upload.

• Edit File: Select to edit the File Details of your uploaded file.

Select Apply to apply your changes, or add another data entity to create joins for a Fusion source.

Join Definition

Create joins between pairs of data entities to create a Fusion data source. You must have at least two data entities in your data source to create a join. If you have more than one data entity in your source, all entities must be used in a join.

To create a new join, select Add, then define the entities, join type, and fields. Select Apply to finish creating the join.

When you create a join, settings you can define can include:

• Entity Left: Select an available entity from the data entities you defined.

• Join Type: Select Left, Inner, or Full Outer.

• Entity Right: Select an available entity from the data entities you defined.

• Enable Dimension Entity: Select the gear () icon to enable dimension for one or both entities. This improves the performance of queries execution by removing unused data entities from the join.

• Field Left: Select at least one field from this entity. You can add multiple fields by selecting the add field () button.

• Field Right: Select at least one field from this entity. You can add multiple fields by selecting the add field () button.