Allow me to reintroduce myself

Alright, alright, I haven’t posted in a long time (7 months or so).  In my defense I have had some things going on.  But someone has been bugging encouraging me to get back to it. I argued that I have better ROI spending time with them, when I was politely informed that I don’t really need the full 6 hours of sleep I attempt every night.  So I’m typing this up at 11:30 pm while the rest of my family is asleep so please read quietly so as to not disturb anyone.

Now that I’m typing and you’re reading quietly, on to the reintroduction: here I am, over here to, also mentioned here and here.

As to why no activity lately, to be honest I have had a lot to say and I have a whole list of ideas for blog posts however as soon as I start to write one, my perfectionism comes into play and I never have time to create what I think would be a perfect post, so I don’t.  I look up at what I don’t know and decide I don’t know enough, instead of looking down or around at what I do know to talk about.  This is the beginning of the effort to change and just put some thoughts out there (for better of worse :)

For those keeping score at home I’m still at Magenic and while a couple good friends have left, we’re hiring.  In fact in the Chicago office we have had 3 people start in the last 2 weeks (including here)! So if you are looking, hit me up and we’ll see what can be done.

Other than that, I’m still doing all things SQL Server from SSIS to Service Broker to CDC to Replication to CLR to encryption to SSRS to data modeling and of course VSDBPro.  Basically the entire stack and oh by the way, still loving it!

For review this post had 3 purposes:

  1. Begin Momentum
  2. Remind everyone I’m still around
  3. Fulfill my promise to Ted

Mission Accomplished!

SSIS CDC LSN Debugging nightmare second problem

Continuing the CDC debugging posts:

Now that I have the StartLSN and EndLSN I was ready to go.  Hit my breakpoint and the variables were set appropriately.  Woo-hoo!

Then…wait a minute…what?

vSQLQuery.Value =
    "SELECT " +
    "    c.__$operation " +
    "    , c.Column1 " +
    "    , c.Column2 " +
    "    , ColumnN " +
    "FROM " +
    "    cdc.fn_cdc_get_net_changes_<instance_name>( " + sStartLSN + ",  " + sEndLSN + ", 'all') c";

That statement wasn’t changing the value of vSQLQuery.  huh?

Ok, I don’t by any stretch of the imagination claim to be an expert at C# but that looked pretty straight forward to me, however it wasn’t changing the value.

So now I start looking in earnest, google, bing, Stackoverflow, Serverfault, all to no avail.  This just wasn’t making sense.  I was about to call over an expert C# person (Magenic is full of them :) ) when I remembered that my previous problem was not a problem with the code but a problem with the configuration of the task.  So I start looking into the configuration of the script task and the variables that I was using and that’s where I saw it. 

vSQLValue was set to evaluate as an expression.  I had done this so I could copy and paste the SQL query I had developed in SSMS as opposed to making the query 1 line and pasting it into the value of the variable (the variable box in SSIS can’t interpret multi-line values, it only pastes the first line).

Turned off the fact that it was an expression and since it had already been evaluated it set the variable correctly (score!) and now I could manipulate the vSQLQuery.Value via the script task.

Now as to why I couldn’t modify a variable that is set to evaluate as an expression, I don’t know I’m sure there’s some sort of order of operations or locking catch here, but I still haven’t been able to find anything official about this.  Best I saw was this which would indicate to me that it would work.

Onto the next problem.

SSIS CDC LSN Debugging nightmare first problem

So now that we have the idea let’s look as the four main problems I encountered:

The first problem I noticed was that the SQL query that was running still had nulls in on the StartLSN and EndLSN

so I was querying:

cdc.fn_cdc_get_net_changes_<instance name> (null, null, ‘all’)

Which as I said last post, returns an error. So throw a break point into the Script task and figure out why it’s not setting properly.  This took me awhile, here’s the code:

var vStartLSN = Dts.Variables["User::sStartLSN"];
var vEndLSN = Dts.Variables["User::sEndLSN"];
var vSQLQuery = Dts.Variables["User::sSQLQuery"];

string sStartLSN;
string sEndLSN;

Dts.VariableDispenser.LockForRead("User::sStartLSN");
Dts.VariableDispenser.LockForRead("User::sEndLSN");
Dts.VariableDispenser.LockForWrite("User::sSQLQuery");

sStartLSN = vStartLSN.Value.ToString();
sEndLSN = vEndLSN.Value.ToString();

vSQLQuery.Value =
    "SELECT " +
    "    c.__$operation " +
    "    , c.Column1 " +
    "    , c.Column2 " +
    "    , ColumnN " +
    "FROM " +
    "    cdc.fn_cdc_get_net_changes_<instance_name>( " + sStartLSN + ",  " + sEndLSN + ", 'all') c";
Dts.Variables.Unlock();

So hitting my first breakpoint, I first realized that the StartLSN variable was receiving a null.  That wasn’t supposed to happen, so I looked at the SQL code I used to set the variables in the SQL Task:

SELECT sys.fn_cdc_get_min_lsn(<instance_name>) as @StartLSN

A-ha, that’s it. The function returns a record set and I had the task to just get a return value, not a record set.

So changed to:

SELECT @StartLSN = sys.fn_cdc_get_min_lsn(<instance_name>);

SELECT @StartLSN as StartLSN

That got me past the first problem, now onto the next problem.