Professional Development No comments yet

Do you want to be the best?

By on October 25th, 2011

If you’re into sports or any type of serious competition, you’ve probably heard the phrase:

If you want to be the best, you need to beat the best

The reason this is important is that we often don’t know what we are truly capable of until pushed to our limits.  To get pushed to our limits we need to be challenged by something or someone that will cause us to stretch further and get outside of our comfort zone.  By definition this isn’t a relaxed, comfortable situation.  That’s how we get better.  On the flip side it is said of teams that are up and down:

they play to the level of their competition

As SQL Server professionals it’s easy to play to the level of our competition (i.e., our current data environment).  I don’t know how many times I’ve interviewed someone with 7+ years experience with SQL Server that considers themselves a senior person, when in fact they have just had the same one year experience, 7 times.

So while attending Are you a Linchpin? Career Management Lessons to Help You Become Indispensible, I posed the question to the panel: during the ebbs and flows of their careers, how do they keep challenging and pushing themselves?

Kevin (twitter | blog) restated the question succinctly this way:

  • Year 1 – Focus on Reliability/Standardization
  • Year 2 – Focus on Optimization
  • Year 3 – Focus on Automation
  • Year 4 – Focus on lolcats (ok so Kevin didn’t specifically say lolcats)

And the question is what to do year 4.  First was a motivation discussion surrounding the concept of what you do today is a down payment on tomorrow

Then came a couple suggestions:

  • Ask for additional problems within your company that need solving
  • Review things you worked on before, the idea being that your better now than you were 3 years ago (referencing the example above), go back and improve those first scripts you created

So let’s change 1 word of the quote to make it applicable:

If you want to be the best, you need to beat the your best

While this is more applicable, this is a tough nut to crack, and I’m always interested in how people deal with this situation.  Some people create extensive labs to be able to test a multitude scenarios, some people create projects on codeplex, while others might attempt to attain some uber-certification.  The point is to continually challenge yourself and improve.

As part of this discussion, it’s important to understand how you learn.  Personally, I learn best through discussion and just doing it.  I don’t learn well from a class nor from a book.  For me, classes are when I don’t even know where to begin and I use books mainly for reference.  However I have known people that learn best by books, in fact I had a developer once that was able to sit by himself in the corner with a book and learn a new language in a week!

For me, this is why we have community.  The discussion.  Events like SQLPASS, SQLSaturday and local user groups along with tools like twitter and blogging can produce great discussions and out of those discussions can come creativity and ideas that challenge assumptions or opens your eyes to additional possibilities.

So how do you beat your best?

Database, Professional Development, SQL Server 1 comment

SQLPASS Conference Day 3, the Last Day

By on October 18th, 2011

The Final day of the conference began with Rob Farley (twitter | blog) and Buck Woody (twitter | blog) singing a Rob Farley original “I Should have Looked the Other Way” which you can see here (audio compression issues), for the performance or here for the lyrics version (start at 2:20).

Next up was board announcements such as Wayne Snyder and Rick Heiges both rolling off the board.  While they were bringing Wayne up on stage they were showing quotes from people within the community about Wayne, all of which were very moving, however the one that I thought really stood out was about (and I apologize that I didn’t catch who said it) how Wayne transformed SQLPASS from a technical conference into a family reunion.

Keynote

Then onto the final keynote done by Dr. David DeWitt (site), you can download his presentation here (this is a summary of that presentation, if you are really interested, go download it or better yet, watch it).  Dr. DeWitt was a professor for years and has mastered the ability of explaining complex concepts.  During the presentation we were able to email questions which were answered after the presentation.  Today he decided to tackle Big Data.

In 2009 there was 0.8 Zettabyte (ZB) (1 million petabytes/1 trillion terabytes/1 quadrillion GB) and by the years 2020 the expectation is to have 35 ZBs (growth factor of 44).  That’s a lot of data, much of it coming from sources such as sensors, Web2.0, Web clicks, etc.  However much of this data while valuable to store, we don’t necessarily care about ACID properties or relational integrity or other things that would utilize a traditional RDBMS. Point being as always, right tool for the right job.

So in these cases, what is the right tool?  Well with Wednesday’s announcement, looks like Microsoft is putting their weight behind Hadoop and MapReduce, which offers:

  • Scalability and a high degree of fault tolerance
  • Ability to quickly analyze massive collections of records without forcing data to first be modeled, cleansed and loaded
  • Easy to use programming paradigm for writing and executing analysis programs that scale to 1000s of nodes and PBs of data
  • Low up front software and hardware costs

So what’s the system look like:

image

  1. Hadoop Distributed File System (HDFS) –  objectives are load balancing, fact access and fault tolerance, designed with the expectations that hardware/software failures
  2. MapReduce – framework for writing/executing distributed, fault tolerant algorithms – 2 functions map which divided a large problem into smaller problems and then performs the same function on all smaller problems and reduce which then combines the results.
  3. Hive & Pig – Hive was created by Facebook as a and is SQL-like, while Pig was created by Yahoo and is more procedural; both target MapReduce jobs.  However due to the complexity of MapReduce, HiveQL was created to combine the best features of SQL with MapReduce
  4. Sqoop – package for moving data between HDFS and relational DB systems via command line load and unload utilities

He then showed some performance metrics of SQL PDW and stated

I assert that it is MUCH easier to add support to SQL Server PDW for unstructured data (w/o having to load it), improved scalability, and fault tolerance than it is to ever get competitive performance from a Hadoop-based system

But again the point being that both of these type of systems (RDBMS and Hadoop)  are going to be working together, it will not be a case of choosing one or the other.

Sessions

After the keynote I sat in Adam Machanic’s (twitter | blog) Query Tuning Mastery for a bit before going downstairs to host the “SSIS for all, DBAs developers, etc.” table at the Birds of a Feather luncheon with Matt Masson (twitter | blog).  Ted Krueger (twitter | blog) had to leave leave early and Mike Walsh (twitter | blog) was looking for volunteers.  Had some good conversation ranging from “What is SSIS” to “How do I do meta-driven SSIS”

After that I jumped into Rewrite Your T-SQL for Great Good by Jeremiah Peschka (twitter | blog).  Jeremiah’s slides as always excellent, and I have a similar presentation entitled Writing Professional Database Code so I figured I’d go and borrow check it out.  I was glad I did as it had a different focus and perspective.  This just underscored to me again that each person brings their own experiences and perspective which is valuable.  In all honesty it is a common excuse that I used (as well as others) as to why I couldn’t present or blog, because the content was already out there.   So lesson learned again, stop making excuses. SmileAnyway he talked about consistency and gave a link to open source Unit Testing tools.  As a user of TFS, I haven’t had much experience with these and look forward to investigating them.  He gave a lot of query performance options and even threw down the gauntlet for everyone who uses “Distinct” in a query to explain the necessity of it as most of the times it’s used due to either a bad data model or bad joins.  Great stuff!

Last I went to Are you a Linchpin? Career Management lessons to help you become indispensable.  This was a panel discussion with Q&A at the end Linchpin: Are You Indispensable by Seth Godin as the starting point of the discussion.  The panel was made up of:

One of the interesting things that Kevin said (attributing credit to Brent) was that

People are going to remember you for 1, 2, maybe 3 adjectives.  What adjectives do you want to be known for?

My mother had always told me something similar:

People won’t remember what you did, but they will remember how you made them feel

Anyway it was a great conversation with some interesting Q&A, would love to see more of these type of things.

Whew, that’s it for the summit, had a great time and looking forward to next years!

SQL Server, VSDBPro No comments yet

SQLPASS Conference Day 2

By on October 14th, 2011

Today started out with a keynote from Quentin Clark and he talked about what he termed the “Fantastic 12 of 2012” reasons :

  1. Required 9′s & Protection – Integration Services is now Integration Server; HA for StreamInsight; AlwaysOn
  2. Blazing-Fast Performance – Column Store Indexes; improvements across the board
  3. Rapid Data Exploration – Power View + PowerPivot; Admin from SharePoint; Reporting Alerts
  4. Managed Self-Service BI – same as #3
  5. Credible Consistent Data – BI Semantic Model (BISM); Data Quality Services (DQS);Master Data Services (MDS)
  6. Organize Compliance – Expanded Audit (user defined and filtering); User Defined Server Roles
  7. Peace of Mind – Production-simulated distributed testing; System Center Advisor & Management Packs; Expanded Support – Premier Mission Critical
  8. Scalable Data Warehouse – SQL Server Appliances; HW + SW + Support – Just add Power; choice of hardware
  9. Fast Time to Solution – Same as #8
  10. Extend Any Data, Anywhere – Greater Interoperability, New Drivers for PHP, Java and Hadoop; ODBC Drivers for Linux & CDC for SSIS and Oracle; Beyond Relational: FileTable, 2D Spatial, Semantic Search
  11. Optimized Productivity – SQL Server Data Tools (SSDT); Unified Across Database & BI; Database & Targeting Freedom
  12. Scale on Demand – AlwaysOn; Deployment across Public & Private; Elastic Scale

We saw demos of AlwaysOn, Column Store Index, Data Quality, and Semantic Search, plus they showed off the SQL Server Appliances.  The thing that really stood out to me in the keynote was the Semantic Search Demo.  The presenter took two pdf files and utilizing TSQL was able to relate them conceptually.  Looked very cool, you can go here for more info.

After that I attended Matt Masson’s (twitter | blog) session on Top 10 Things You Didn’t Know About SSIS in SQL Server Code Name “Denali”.  These were items that are post CTP3.  To kill some time while waiting for the session to start Matt went through a mock scenario of utilizing SSIS to create virtual friends to party with him and his cat (Mr. Wiggles) on Facebook using Facebook Services and StreamInsight Transform – it was absolutely hilarious!

  1. Change Data Capture – as someone who has done CDC in SSIS this was very cool
  2. ODBC Support
  3. Connections Managers – Shared In-Memory Cache Across Packages using the shared project level connection manager
  4. Flat File Source Improvements – supports varying number of columns and can parse embedded qualifiers
  5. Package Format Changes – doesn’t persist default values; doesn’t use lineage id’s, instead uses ref id’s; sorted by name
  6. Visual Studio Configurations
  7. Scripting Improvements – can now set a break point in a script task/component and see data
  8. Troubleshooting & Logging – Built-in reporting; Component Timing & Row Counts; Server level logging
  9. Data Taps – can write out data to a csv location without code change (doesn’t log binary)
  10. Server Management using PowerShell
  11. Honorable Mention – Project Barcelona Metadata Discovery – crawler that crawls entire MS stack to determine Impact Analysis and Data Lineage

These were on top of all the other things we new about for SSIS in SQL 2012, and I have to say that I’m really looking forward to it!

Next I went to the Lighting Talks which is a cool idea.  The premise is to have a number of speakers, this one has seven.  Each speaker however is limited to 5 minutes and there is someone utilizing a timer and the crowd is encouraged to clap people off stage that go to long.  It was a lot of fun.  The two that really stood above the rest (in my opinion anyway) were Grant Fritchey’s (twitter | blog) “Backup Testing, The Rant ” and Niko Neugebauer’s (twitter | blog) “Build Up”.

Last was Database Development with SQL Server Data Tools Code-Named “Project Juneau” by Gert Drapers (blog).  As someone who has used Database Professional extensively, I was pretty skeptical going in since in that Unit Tests aren’t in SSDT.  As you would expect one of the big features is that it has support for Azure.  However some of the stuff that I did see was really cool:

  • A True Debugger with break points and a Call Stack, with the ability to start in the .NET app
  • Refactor can now reach into the application
  • True Language Services – can right click to go to reference, etc.
  • Drift Detection – if working in a connected stated it will poll database to see if any changes happened behind you and you can just drag and drop changes into project
  • Easier way to save specific deployed versions
  • Table designer where you see both code and the designer window
  • SQL CLR is within the same project, no longer have to create a separate CLR project

Some things I still have to think if I like or not:

  • Renamed Deploy to publish so that Projects are now Published
  • Multiple objects are now in a single file, for example the indexes, keys are now defined in the create table script file

And the things I don’t like, until vNext Visual Studio + 3 months we’re missing:

  • Data comparison
  • Data Generation
  • Unit Testing

Now for the great news.  It’s completely FREE!!  It will be delivered as part of the SQL Server install as well as part of the Visual Studio install or you can get it through  the Web Platform Installer to get it!  We’ve come a long way from requiring TFS 2005 Team System.  Looks like we are getting closer to a standard SQL Server development environment.  That was it for day 2, next up is the 3rd and final day.