Use the same job to loop through multiple origins

From Starfish ETL
Revision as of 21:21, 12 February 2018 by Jkuehlthau (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

This example looks at reading from multiple mailboxes for email. Use a variable in the origin connection string. If the job needs to run for another mailbox, it calls the GotoJob function in the “Once After Conn” and pass in the Job's ID. Similar code can be used to write to multiple destinations, for example: Write events to an Exchange calendar for different users.

Once Before Conn Before Operation that returns the current Mailbox to process.

object ScriptedVariable()
{
        string mb = Starfish.GetSetting("Mailboxes");
        Starfish.LogMessage(runCnt.ToString());
        mb = mb.Split(';')[runCnt].Split(',')[0];
        mailBox = mb;
        Starfish.LogMessage("Mailbox: " + mb);
        return mb;
}

Connection String:

@@VAR:vMailbox@@

Once Before Conn Before Operation that retrieves the last Run Date Time value for the User we're processing.

object ScriptedVariable()
{
	//Variable as used in the Origin: @@VAR:userLastRun@@
	string sDt = Starfish.GetSetting("Events-LRD-"+gUserID);
	DateTime dt = Convert.ToDateTime(sDt);
	//Sometimes, the Starfish server and the Source Server clocks are not quite in sync, so I subtract 1 minute from my last run datetime to guarantee I don't miss any records.
	dt = dt.AddMinutes(-1);
	//Add 6 hours to convert to GMT.
	//Starfish.LogMessage(dt.ToString("yyyy-MM-ddTHH:mm:ss+00:00"));
	dt = dt.AddHours(6);
	//Starfish.LogMessage(dt.ToString("yyyy-MM-ddTHH:mm:ss+00:00"));
	
	return dt;
}

Once After Conn After Operation that sets the last Run Date Time value for the User we're processing AND checks to see if we have any more mailboxes to process.

void CSharpProcedure()
{              
	if (!Starfish.PreviewMode)
	{
		Starfish.SaveSetting("Mail-LRD-"+mailBox, DateTime.Now.ToString());
		string mb = Starfish.GetSetting("Mailboxes");
		if (runCnt < mb.Split(';').Length-1)
		{
			runCnt++;
			Starfish.GotoJob("f54baebc-d128-4f9d-9d1a-ac7c88e027fe");
		}
	}
}