Use the same job to loop through multiple origins: Difference between revisions

From Starfish ETL
Jump to navigation Jump to search
No edit summary
No edit summary
 
(5 intermediate revisions by the same user not shown)
Line 1: Line 1:
This example looks at reading from multiple mailboxes for email.  Use a variable in the origin connection string.  If the job needs to run for another mailbox, it calls the GotoJob function in the “Once After Conn” and pass in the Job's ID.
This example looks at reading from multiple mailboxes for email.  Use a variable in the origin connection string.  If the job needs to run for another mailbox, it calls the GotoJob function in the “Once After Conn” and pass in the Job's ID.  Similar code can be used to write to multiple destinations, for example: Write events to an Exchange calendar for different users.


Once Before Conn Before Operation that returns the current Mailbox to process.
Once Before Conn Before Operation that returns the current Mailbox to process.
<source lang="vb">
<source lang="csharp">
object ScriptedVariable()
object ScriptedVariable()
{
{
                //Sample UserIDs setting: ADMIN      ;UAAAAA000069;UAAAAA00006B;UAAAAA00006A;UAAAAA000068
        string mb = Starfish.GetSetting("Mailboxes");
                string mb = Starfish.GetSetting("Mailboxes");
        Starfish.LogMessage(runCnt.ToString());
                Starfish.LogMessage(runCnt.ToString());
        mb = mb.Split(';')[runCnt].Split(',')[0];
                mb = mb.Split(';')[runCnt].Split(',')[0];
        mailBox = mb;
                mailBox = mb;
        Starfish.LogMessage("Mailbox: " + mb);
                Starfish.LogMessage("Mailbox: " + mb);
        return mb;
                return mb;
}
}
</source>
</source>


Connection String:
Connection String:
<source lang="vb">
<source lang="csharp">
@@VAR:vMailbox@@
@@VAR:vMailbox@@
</source>
</source>


Once After Conn
Once Before Conn Before Operation that retrieves the last Run Date Time value for the User we're processing.
<source lang="csharp">
object ScriptedVariable()
{
//Variable as used in the Origin: @@VAR:userLastRun@@
string sDt = Starfish.GetSetting("Events-LRD-"+gUserID);
DateTime dt = Convert.ToDateTime(sDt);
//Sometimes, the Starfish server and the Source Server clocks are not quite in sync, so I subtract 1 minute from my last run datetime to guarantee I don't miss any records.
dt = dt.AddMinutes(-1);
//Add 6 hours to convert to GMT.
//Starfish.LogMessage(dt.ToString("yyyy-MM-ddTHH:mm:ss+00:00"));
dt = dt.AddHours(6);
//Starfish.LogMessage(dt.ToString("yyyy-MM-ddTHH:mm:ss+00:00"));
return dt;
}
</source>
 
Once After Conn After Operation that sets the last Run Date Time value for the User we're processing AND checks to see if we have any more mailboxes to process.
<source lang="csharp">
<source lang="csharp">
void CSharpProcedure()
void CSharpProcedure()
{               
{               
                if (!Starfish.PreviewMode)
if (!Starfish.PreviewMode)
                                Starfish.SaveSetting("Mail-LRD-"+mailBox, DateTime.Now.ToString());
{
                string mb = Starfish.GetSetting("Mailboxes");
Starfish.SaveSetting("Mail-LRD-"+mailBox, DateTime.Now.ToString());
                if (runCnt < mb.Split(';').Length-1)
string mb = Starfish.GetSetting("Mailboxes");
                {
if (runCnt < mb.Split(';').Length-1)
                                runCnt++;
{
                                Starfish.GotoJob("f54baebc-d128-4f9d-9d1a-ac7c88e027fe");
runCnt++;
                }
Starfish.GotoJob("f54baebc-d128-4f9d-9d1a-ac7c88e027fe");
}
}
}
}
</source>
</source>

Latest revision as of 21:21, 12 February 2018

This example looks at reading from multiple mailboxes for email. Use a variable in the origin connection string. If the job needs to run for another mailbox, it calls the GotoJob function in the “Once After Conn” and pass in the Job's ID. Similar code can be used to write to multiple destinations, for example: Write events to an Exchange calendar for different users.

Once Before Conn Before Operation that returns the current Mailbox to process.

object ScriptedVariable()
{
        string mb = Starfish.GetSetting("Mailboxes");
        Starfish.LogMessage(runCnt.ToString());
        mb = mb.Split(';')[runCnt].Split(',')[0];
        mailBox = mb;
        Starfish.LogMessage("Mailbox: " + mb);
        return mb;
}

Connection String:

@@VAR:vMailbox@@

Once Before Conn Before Operation that retrieves the last Run Date Time value for the User we're processing.

object ScriptedVariable()
{
	//Variable as used in the Origin: @@VAR:userLastRun@@
	string sDt = Starfish.GetSetting("Events-LRD-"+gUserID);
	DateTime dt = Convert.ToDateTime(sDt);
	//Sometimes, the Starfish server and the Source Server clocks are not quite in sync, so I subtract 1 minute from my last run datetime to guarantee I don't miss any records.
	dt = dt.AddMinutes(-1);
	//Add 6 hours to convert to GMT.
	//Starfish.LogMessage(dt.ToString("yyyy-MM-ddTHH:mm:ss+00:00"));
	dt = dt.AddHours(6);
	//Starfish.LogMessage(dt.ToString("yyyy-MM-ddTHH:mm:ss+00:00"));
	
	return dt;
}

Once After Conn After Operation that sets the last Run Date Time value for the User we're processing AND checks to see if we have any more mailboxes to process.

void CSharpProcedure()
{              
	if (!Starfish.PreviewMode)
	{
		Starfish.SaveSetting("Mail-LRD-"+mailBox, DateTime.Now.ToString());
		string mb = Starfish.GetSetting("Mailboxes");
		if (runCnt < mb.Split(';').Length-1)
		{
			runCnt++;
			Starfish.GotoJob("f54baebc-d128-4f9d-9d1a-ac7c88e027fe");
		}
	}
}