SFTP Import Integration

Last Updated: 19/11/2024     Tags: SFTP, Import, integrations
  • Switch Version
  • V5
  • V4

Integrations is not available for Taguchi V4 and is only available in V5.
Please contact Support if you don't know which version of Taguchi you are using.

Taguchi offers a self-service SFTP integration that allows you to easily import your data into Taguchi from an SFTP location.

This integration allows secure and efficient file transfers, enabling the import of large volumes of data, such as subscriber and purchase data, directly into the Taguchi platform. With the SFTP import integration, the process is fully automated, ensuring that your data is regularly updated without manual intervention, while maintaining the highest level of security.

Credentials

You will need to setup up one compulsary SFTP credential and one optional GPG credential before you are able to create an SFTP import integration.

Here is more information on SFTP credentials. In order to set up credentials, you will need to be an administrator of your organization.

SFTP Import Integration Setup

To create a new integration, ensure that you have the required User Type integration role privileges assigned to you. The 'integration' page is under Settings > Integrations.

Picture_2

Select โ€˜New Integrationโ€™ and a new screen will popup.

Create New List

Picture_4

There are 4 areas within this integration set-up screen:

  1. Details
  2. Configuration
  3. Map data; and
  4. Notes

Details

Create New List

  1. Set the name of the integration.

  2. Under 'Type', select 'Import: SFTP'. When selecting the integration type, do take note that the integrations are arranged in five different groups which are API, Import, Export, Messaging, Scheduled and that some integrations would appear in multiple groups.

  3. The Schedule start date/time determines the integration start point. When used in tandem with repeat (daily), this allows the integration to repeat daily at 13:00 commencing from the 12th of October 2024 (based on the example above).

  4. Repeats allow you to conduct the integration multiple times without having to create a new integration. You could choose either Yearly, Monthly, Weekly, Daily and Hourly. Some platforms have processing limitation whereby it could take some time to process the data. Hence hourly repeats might be affected and daily repeat might be a preferred option instead.

Configuration

Create New List

  1. Credential to use: Choose the desired credential to use that you have set up in the credential page for this SFTP import integration.

  2. Directory: The Directory is a specific folder on the SFTP server where files are stored.

  3. Search file paths: The Search file paths is the location that you are uploading the data to and will search for files in the directory and any of its subdirectories. (e.g. /to_taguchi/*)

  4. File Type: You can choose the file type that you would like to import. The file type options are - JSON, CSV, TSV. You can also choose PGP and GPG version of the three available types.

  5. Filter data by foreign column: You can filter data by using a foreign column within the import. (e.g. Country)

  6. Filter data by value: You can filter the data of the foreign column that you have stated above using a specific value. Do take note that you would need to add an input into the foreign column fo it to work. (e.g. AU)

  7. Ignore null text values: When this option is enabled, the integration will treat specific representations of null values (like \N or NULL) as empty or missing values rather than inserting them into the database as actual text.

  8. GPG Encrypted: You can choose to encrypt the file with GPG encryption by clicking on this checkbox.

  9. Select Credential for the GPG Decryption: You can choose the GPG credential from this dropdown menu. The GPG credential can be created within the credential section via the setting.

Map Data

This SFTP import integration has four Taguchi objects that you can use for data mapping

Taguchi Object - Subscriber Normal, Subscriber Custom Field, Subscriber List, Partition Custom Field.

Subscriber Normal -

  1. Add to subscriber list: Add a subscriber list ID that you would like to map/add data into.

  2. Performs Subscription Management: This checkbox will allow you to manage your subscription. By clicking on this checkbox, you will reveal two additional fields.

Subscription Unsub Type: You can choose to unsubscribe via a list or global. If you choose list, you will unsubscribe the subscriber based on the list that you have indicated above.

Subscriber Custom Field, Operator and Unsub Value: You can add a subscriber custom field which will determine whether the subscriber will be unsubscribed based on the value of their subscriber custom field.

There are two operators which you can choose from which are = (Equal) and != (Not Equal).

The "Unsub Value" is the value which will determine if the subscriber will be unsubscribed depending on the operator. So if the option that you have choosen is this (unsub = Y), this will mean that the subscriber, that has "Y" in their "unsub" custom field, will be equal (=) to the Unsub Value of "Y" and be unsubscribed. If you were to use the != (Not Equal) operator instead, you can unsubscribe subscribers that do value equal to the "Unsub Value"

  1. Subscriber Partition Custom Field (Optional): You can choose a partition custom field which will determine where the partition information comes from.

  2. Data Mapping: The data mapping section allows you to map the custom fields from the import file to the custom field within.

Subscriber Custom Field - This allows you to create custom fields based on data within the file.

  1. Custom Field Subscriber Reference Taguchi Field: This field allows you to map to either email or ref (Reference).

  2. Custom Field Subscriber Reference Foreign Field: This field will map to email or ref (depending on the option choosen above)

  3. Custom Field Field Name Foriegn Field: This will map to the the column name of the "email" field within the file.

  4. Custom Field Field Value Foreign Field: This will map to the the column name of the "field name" field within the file

  5. Custom Field Group Foreign Field: This will map to the the column name of the "field group foreign" field within the file

  6. Custom Field Group Constant Field: This will map to the the column name of the "field group constant" field within the file

  7. Custom Field Create Subscribers If They Do Not Exist: This checkbox will allow you to create subscribers if they do not exist.

Subscriber list -

  1. List Subscriber Reference Taguchi Field: This field allows you to map to either email or ref (Reference).

  2. List Subscriber Reference Foreign Field: This field will map to email or ref (depending on the option choosen above)

  3. List Foreign Reference Field: List Foreign Reference Field refers to a specific field in the incoming dataset that establishes a relationship between the current dataset and another related dataset.

  4. List Subscribed Boolean Field (Optional) [Comma Separated]: List Subscribed Boolean Field refers to a specific field in the incoming dataset that indicates whether a particular entity (such as a user, customer, or service) is currently subscribed or not.

  5. List Unsubscribed Boolean Fields (Optional) [Comma Separated]: List Unsubscribed Boolean Field refers to a specific field in the incoming dataset that indicates whether a particular entity (such as a user, customer, or service) has opted out of a subscription or service.

  6. List Start Datetime Field (Optional): List Start Datetime Field refers to a specific field in the incoming dataset that indicates the start date and time for an event, subscription, or activity.

  7. List End Datetime Field (Optional): List End Datetime Field refers to a specific field in the incoming dataset that indicates the end date and time for an event, subscription, or activity.

  8. List Template String: List Template String is a defined format or template that specifies how data from incoming files should be structured or transformed during the import process.

  9. List Resub Days: List Resub Days refers to a field that specifies the number of days after which a subscription or an item should be re-evaluated, re-subscribed, or subject to renewal processes.

  10. List Resub Timestamp Field: a List Resub Timestamp Field refers to a specific field in the incoming data that records the timestamp indicating when a subscription or related entity was last re-evaluated or re-subscribed.

  11. List Resub Timestamp Format: List Resub Timestamp Format refers to the specific format in which timestamps are represented in the incoming data.

Partition Custom Field -

  1. Partition Reference Foreign Field: The name of the foreign field column that identifies what the partition reference is.

Save Changes and Activate

Create New List

Once you have added your Details, Configuration and Target Expression, you will be able to save changes (1) and activate (2) the integration. You can only activate once you have save changes.

Deactivate

Create New List

To stop the integration, click on Deactivate. Update to : This will stop the integration between your SFTP server and Taguchi.

Active Integration

Create New List

Once the integration has been saved and activated, the on/off icon of the integration card will change from yellow (Off) to green (On) which indicates that it is now activated.

Update Integration

Create New List Create New List

If you want to update your Integration, you will be able to make the changes while it is activated or deactivated. Once the changes have been made, you will be able to update the integration by clicking on 'Save changes'. The Integration will be updated immediately.

Integration Status

Users can determine the status of an integration by looking for the status symbol (On/Off icon) colour within the integration tile.

Integration_Tile

  • Green โ€“ This colour indicates that integration is active and has either been deployed or awaiting deployment.

  • Yellow โ€“ This colour indicates that the integration has been deactivated.

  • Red โ€“ This colour indicates that the integration has failed to deploy the extract.

Integration Deployment Status

Picture_10

Users will be able to view the integration status message by hovering their cursor over the status symbol (On/Off) icon.

The messages the users will see will vary based on the deployment of the integration.

  • Green โ€“ "Integration has not yet run (Start time/date have not occurred yet) / Integration is deployed. Last successful run onMon May 06 2024 06:00:44 GMT+0800 (Singapore Standard Time)"

  • Yellow โ€“ "Integration is not deployed (Deactivated)"

  • Red โ€“ "Integration Failed to run on Mon May 06 2024 06:00:44 GMT+0800 (Singapore Standard Time)"

Troubleshooting Measures

  • When choosing the interval (Hourly, Daily, Weekly and Monthly) in the integration setup, users would need to take into consideration that the platform (Formstack) that will receive the data from us will take 24 - 48 hours to ingest which would mean that an hourly repeat for the integration would result in the data not being displayed on your ads platform and could also result in your connection being potentially cut. Hence, we recommend using Daily, Weekly or Monthly instead of Hourly.

If you are having trouble with this integration, please contact Taguchi Support for assistance.