This feature allows Datasets to be automatically refreshed at the point of Workflow execution. The latest data will be fetched from the origin.
Auto-refresh sources will work with External systems where the source table is still available. File uploads and dropzone files are excluded because once a Dataset is created, the origin file is not retained.
Valid data source types for Auto-refresh sources are JDBC, Azure BLOB, HDFS, Amazon S3, and SFTP.
Views will inherit the refresh setting from their base Dataset, so a Workflow that uses a View as source can also be configured to refresh that View's underlying Dataset on execution.
There are three settings required in order to enable Auto-refresh sources:
These settings have a cumulative effect and you need all three to be enabled in order for data to be refreshed.
When all of these options are enabled, and the Workflow is executed, it will run with the latest data from the origin.
Provided your dataset is from a supported data source type:
On the Source step, tick Allow auto-refresh.
Via the Run Workflow dialog or Schedule for execution:
Having this level of control allows for scenarios such as: