Check for duplicates using xref

From Starfish ETL
Revision as of 19:09, 10 September 2015 by Jkuehlthau (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

I needed to check for duplicate attachments in a system and remove the duplicates. I created a Job that used the same origin and destination and pulled all records from the Attachments table. I had a before operation that would read and write to the same xref list, thanks to the use of the newer SQLite xref list functionality, http://wiki.starfishetl.com/index.php/Cross-reference_(Xref)#SQLite_Xref. In this code, I check to see if the record I'm reading already exists in the XREF. If it exists, then I know it is a duplicate and can proceed to the Delete Stage. If it does not exist, then it is not a duplicate and I write the data to the XREF and skip the Delete Stage.

Sub VBScriptProcedure
	dim strExists
	strExists = xrefread("sf-attachments","@@ORG:ParentId@@@@ORG:NAME@@")
	If strExists = "Exists" Then
		LogMessage "Duplicate of: @@ORG:NAME@@"
	Else
		XrefWrite "sf-attachments","@@ORG:ParentId@@@@ORG:NAME@@","Exists"
		GoToNextRow
	End if
End Sub