Engine Web.Config Settings: Difference between revisions
Jump to navigation
Jump to search
Jkuehlthau (talk | contribs) No edit summary |
Jkuehlthau (talk | contribs) No edit summary |
||
(9 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
This file is generally located at: ..\inetpub\wwwroot\StarfishEngine\ | This file is generally located at: ..\inetpub\wwwroot\StarfishEngine\ | ||
{| class="wikitable" | {| class="wikitable sortable" | ||
! style="font-weight: bold;" | Setting | ! style="font-weight: bold;" | Setting | ||
! style="font-weight: bold;" | Default | |||
! style="font-weight: bold;" | Description | ! style="font-weight: bold;" | Description | ||
|- | |- | ||
| LogPath | | LogPath | ||
| C:\Inetpub\WWWRoot\StarfishEngine\ | |||
| Path to save log files in when LogToFile is enabled. | | Path to save log files in when LogToFile is enabled. | ||
|- | |- | ||
| LogToFile | | LogToFile | ||
| False | |||
| Boolean, Default False. If enabled, a log file (at the “High” level) with be created within the specified LogPath location. | | Boolean, Default False. If enabled, a log file (at the “High” level) with be created within the specified LogPath location. | ||
|- | |- | ||
| CommandTimeout | | CommandTimeout | ||
| 3000 | |||
| Length in seconds for SQL commands to execute | | Length in seconds for SQL commands to execute | ||
|- | |- | ||
| ProxyEnabled | | ProxyEnabled | ||
| False | |||
| Boolean, Default False. If enabled, Starfish ETL™ will use the specified Proxy server when validating it’s license. | | Boolean, Default False. If enabled, Starfish ETL™ will use the specified Proxy server when validating it’s license. | ||
|- | |- | ||
| ProxyURL | | ProxyURL | ||
| | |||
| URL for proxy server | | URL for proxy server | ||
|- | |- | ||
| ProxyPort | | ProxyPort | ||
| 80 | |||
| Port proxy server is running on | | Port proxy server is running on | ||
|- | |- | ||
| ProxyUseDefaultCredentials | | ProxyUseDefaultCredentials | ||
| False | |||
| Boolean, Default False. If enabled, default network credentials will be used. | | Boolean, Default False. If enabled, default network credentials will be used. | ||
|- | |- | ||
| ProxyUseSuppliedCredentials | | ProxyUseSuppliedCredentials | ||
| False | |||
| Boolean, Default False. If enabled, the credentials specified below will be used. | | Boolean, Default False. If enabled, the credentials specified below will be used. | ||
|- | |- | ||
| ProxyCredentialsUsername | | ProxyCredentialsUsername | ||
| | |||
| Proxy username | | Proxy username | ||
|- | |- | ||
| ProxyCredentialsPassword | | ProxyCredentialsPassword | ||
| | |||
| Proxy password | | Proxy password | ||
|- | |- | ||
| ProxyCredentialsDomain | | ProxyCredentialsDomain | ||
| | |||
| Proxy user’s domain | | Proxy user’s domain | ||
|- | |- | ||
| RowThreadCount | | RowThreadCount | ||
| | | 1 | ||
| The default number of threads to use when executing a Starfish ETL™ job. | |||
|- | |- | ||
| UseParamQueries | | UseParamQueries | ||
| False | |||
| Boolean, Default False. If enabled (default), Starfish ETL™ will build parameterized SQL queries. If disabled, queries will instead be sent as normal text. Useful for performance tweaking. | | Boolean, Default False. If enabled (default), Starfish ETL™ will build parameterized SQL queries. If disabled, queries will instead be sent as normal text. Useful for performance tweaking. | ||
|- | |- | ||
| UseNoLock | | UseNoLock | ||
| False | |||
| Boolean, Default False. If enabled, Starfish will append the SQL “WITH(NOLOCK)” command to generated lookup statements. Useful for performance tweaking. | | Boolean, Default False. If enabled, Starfish will append the SQL “WITH(NOLOCK)” command to generated lookup statements. Useful for performance tweaking. | ||
|- | |- | ||
| UseDatabaseXref | | UseDatabaseXref | ||
| False | |||
| Setting this value to true will use a SQLite database to store XREF files instead of text documents. This has the added benefit of automatically updating value pairs instead of appending duplicates. | | Setting this value to true will use a SQLite database to store XREF files instead of text documents. This has the added benefit of automatically updating value pairs instead of appending duplicates. | ||
|- | |- | ||
| CountStatement | | CountStatement | ||
| COUNT(*) | |||
| Default “COUNT(*)”. When doing lookups, Starfish ETL™ will perform a statement such as “SELECT COUNT(*) FROM TABLE WHERE …” to determine if a record exists or not. Changing this value to something like “COUNT(1)” may help performance for some database systems. | | Default “COUNT(*)”. When doing lookups, Starfish ETL™ will perform a statement such as “SELECT COUNT(*) FROM TABLE WHERE …” to determine if a record exists or not. Changing this value to something like “COUNT(1)” may help performance for some database systems. | ||
|- | |||
| HaultOnError | |||
| False | |||
| If any kind of error occurs, the entire job will be stopped. | |||
|- | |||
| LogErrorRowsToCSV | |||
| True | |||
| Creates the Exception CSV files when any errors occur, logging the entire row's data, the row # and error description. | |||
|- | |||
| DefaultLastRunDate | |||
| 2000-01-01 | |||
| When using the LastRunDate system variable, this is the initial date that will be used when the SetLastRunDate function has never been called for the particular job. | |||
|- | |||
| DateFormat | |||
| yyyy-MM-dd HH:mm:ss | |||
| The date formatting used for the LastRunDate variable. | |||
|- | |||
| SaveLogDatabase | |||
| False | |||
| If True, the entire high logging level output will be saved to the engine.db SQLite database in the "log" table. | |||
|- | |||
| SQLColumnPreceding | |||
| | |||
| If your database driver needs special column name handling, you may supply the value to surround the field names with. Typically [] or "". | |||
|- | |||
| SQLColumnTrailing | |||
| | |||
| See above. | |||
|- | |||
|PrefetchRowCount | |||
|True | |||
| Specifies whether Starfish should figure out how many rows it needs to process before starting a job. If set to false, the progress indicator will not be know. However, for some systems it may take a long time to determine this record count so you may choose to disable it. | |||
|- | |||
|RetryRowOnErrorAttempts | |||
|0 | |||
|Options: 0 or greater. Retry Rows when an error is encountered. (January 18, 2023 - I have used up to 10 but more should work.) | |||
|- | |||
|RetryRowOnErrorAttemptWaitMS | |||
|5000 | |||
|Options: 1 or greater. The number of milliseconds to wait between retry attempts. | |||
|- | |||
|RetryRowReconnectAfter | |||
|0 | |||
|Options: 0 or greater. Attempt reconnect after x number of failures. This will tell Starfish to attempt to completely reconnect. Internally this calls the OpenConnection method of the connector, which for some connectors will cause it to get a new Session. (January 18, 2023 - I am not aware of anyone ever using this.) | |||
|} | |} |
Latest revision as of 21:57, 18 January 2023
This file is generally located at: ..\inetpub\wwwroot\StarfishEngine\
Setting | Default | Description |
---|---|---|
LogPath | C:\Inetpub\WWWRoot\StarfishEngine\ | Path to save log files in when LogToFile is enabled. |
LogToFile | False | Boolean, Default False. If enabled, a log file (at the “High” level) with be created within the specified LogPath location. |
CommandTimeout | 3000 | Length in seconds for SQL commands to execute |
ProxyEnabled | False | Boolean, Default False. If enabled, Starfish ETL™ will use the specified Proxy server when validating it’s license. |
ProxyURL | URL for proxy server | |
ProxyPort | 80 | Port proxy server is running on |
ProxyUseDefaultCredentials | False | Boolean, Default False. If enabled, default network credentials will be used. |
ProxyUseSuppliedCredentials | False | Boolean, Default False. If enabled, the credentials specified below will be used. |
ProxyCredentialsUsername | Proxy username | |
ProxyCredentialsPassword | Proxy password | |
ProxyCredentialsDomain | Proxy user’s domain | |
RowThreadCount | 1 | The default number of threads to use when executing a Starfish ETL™ job. |
UseParamQueries | False | Boolean, Default False. If enabled (default), Starfish ETL™ will build parameterized SQL queries. If disabled, queries will instead be sent as normal text. Useful for performance tweaking. |
UseNoLock | False | Boolean, Default False. If enabled, Starfish will append the SQL “WITH(NOLOCK)” command to generated lookup statements. Useful for performance tweaking. |
UseDatabaseXref | False | Setting this value to true will use a SQLite database to store XREF files instead of text documents. This has the added benefit of automatically updating value pairs instead of appending duplicates. |
CountStatement | COUNT(*) | Default “COUNT(*)”. When doing lookups, Starfish ETL™ will perform a statement such as “SELECT COUNT(*) FROM TABLE WHERE …” to determine if a record exists or not. Changing this value to something like “COUNT(1)” may help performance for some database systems. |
HaultOnError | False | If any kind of error occurs, the entire job will be stopped. |
LogErrorRowsToCSV | True | Creates the Exception CSV files when any errors occur, logging the entire row's data, the row # and error description. |
DefaultLastRunDate | 2000-01-01 | When using the LastRunDate system variable, this is the initial date that will be used when the SetLastRunDate function has never been called for the particular job. |
DateFormat | yyyy-MM-dd HH:mm:ss | The date formatting used for the LastRunDate variable. |
SaveLogDatabase | False | If True, the entire high logging level output will be saved to the engine.db SQLite database in the "log" table. |
SQLColumnPreceding | If your database driver needs special column name handling, you may supply the value to surround the field names with. Typically [] or "". | |
SQLColumnTrailing | See above. | |
PrefetchRowCount | True | Specifies whether Starfish should figure out how many rows it needs to process before starting a job. If set to false, the progress indicator will not be know. However, for some systems it may take a long time to determine this record count so you may choose to disable it. |
RetryRowOnErrorAttempts | 0 | Options: 0 or greater. Retry Rows when an error is encountered. (January 18, 2023 - I have used up to 10 but more should work.) |
RetryRowOnErrorAttemptWaitMS | 5000 | Options: 1 or greater. The number of milliseconds to wait between retry attempts. |
RetryRowReconnectAfter | 0 | Options: 0 or greater. Attempt reconnect after x number of failures. This will tell Starfish to attempt to completely reconnect. Internally this calls the OpenConnection method of the connector, which for some connectors will cause it to get a new Session. (January 18, 2023 - I am not aware of anyone ever using this.) |