This feature allows Datasets to be automatically refreshed at the point of Workflow execution. The latest data will be fetched from the origin.
Auto-refresh sources will work with External systems where the source table is still available. File uploads and dropzone files are excluded because once a Dataset is created, the origin file is not retained.
Valid data source types for Auto-refresh sources are JDBC, Azure BLOB, HDFS, Amazon S3, and SFTP.
Views will inherit the refresh setting from their base Dataset, so a Workflow that uses a View as source can also be configured to refresh that View's underlying Dataset on execution, even if the view is shared to another Space.
There are three settings required in order to enable Auto-refresh sources:
These settings have a cumulative effect and you need all three to be enabled in order for data to be refreshed.
When all of these options are enabled, and the Workflow is executed, it will run with the latest data from the origin.
Provided your dataset is from a supported data source type:
On the Source step, tick Allow auto-refresh. On this step you can also control what the behaviour should be if the source refresh fails during workflow execution. Checking Stop execution when refresh failed will cause a workflow execution to fail if the Dataset could not be refreshed from the External System.
In the Run Workflow dialog or Schedule:
Having this level of control allows for scenarios such as: