SSIS – Global replace for all columns using a script component

This saved me a bunch of time—thanks, Waheed!

Waheed Rous

Sometimes while in the data flow you need to apply the same function on all the columns in the data flow or all columns have a specific data type, for example:

  1. Remove all commas and new lines from text columns before exporting them to a CSV file.
  2. Replace text “Null” with Null value when loading data from an excel file that has nulls as text in it.
  3. Check for each column if has a null value and replace it with a default value based on the data type.
  4. Convert all strings to upper or lower case. etc…

The easiest way to do this is to add a script component then use the approach below, here i’m converting all strings to upper case.

few notes about the code above:

  • i’m skipping all columns with name ends with “_isnull” as these columns just to indicate if the original column has a null…

View original post 46 more words

Markdown Test

I just had a terrible thought: what if WordPress lets you post in Markdown but converts it immediately to HTML and doesn’t preserve the original Markdown source. That would be a terrible shame. I’ll find out in a minute…

Oh, good—it does preserve the Markdown source. What a relief!

Write (More) Effortlessly With Markdown

I’m so happy they finally added Markdown to wordpress.com!!! Now we need it (natively) in the self-hosted (.org) version…

The WordPress.com Blog

Markdown has arrived on WordPress.com! Some of you may respond with “Finally!” Others might be asking, “what’s that?” Markdown is a quick way to add formatted text without writing out any HTML.

Let’s take a closer look. Here is an example of how Markdown looks while editing a post:

Markdown Example in the Text Editor

This is how that same example looks in the Reddle theme after it’s converted to HTML:

Markdown Example shown in the Reddle theme

Writing with Markdown

Markdown lets you compose links, lists, and other styles using regular characters and punctuation marks. If you want a quick, easy way to write and edit rich text without having to take your hands off the keyboard or learn a lot of complicated codes and shortcuts, then Markdown might be right for you.

For example, to emphasize a word, you just wrap it with an asterisk on both ends, like this: *emphasized*. When your writing is published, it will instead look like this:

View original post 285 more words

Time Limits on Browser Plugins?

When Steve Gibson talked on Security Now 398 about how few users’ Java plugins are actually up-to-date, this question hit me:

Should browser plug-ins have built-in expiration dates?

The problem with having all of these old Java versions running around is that attacks always get better. How much more sophisticated are the attacks of today than the attacks of just one year ago? Why, then, should anyone think a free browser plugin released today—even if it’s secure by today’s standards—will stand up to the attacks of one year from now?

Fix the ecosystem…

Of course, vendors need to continue to do their best to write secure code in the first place, and release timely updates to fix errors that do make it into the wild. We also need to work on the ecosystem to make it easy for users to stay current—figure out what Apple is doing right, what Android is doing wrong, and how to apply those lessons to the browser plugin market. (I’m not just picking on Java—I’m thinking of Adobe Flash and Reader, too.) I’m not sure how to get end users to care about keeping these plugins up-to-date, but the problem deserves attention. Obviously, the major plugins now auto-update, which will help, but it’s not foolproof (I’m envisioning malware that intercepts update checks to keep vulnerable plugins in-the-wild longer).

…and build in a time limit

What I’m proposing is that vendors build in an expiration date as a safety net, so if a user tries to run a 12-month-old plugin (which won’t happen if auto-update is working and the vendor is still maintaining the product), it displays an expiration message and instructions for how to get a new version. Obviously this doesn’t solve our current problems, but it should be part of a strategy to make sure we’re not still in the same boat a few years from now.

New Project: Backbone Reference App

Today I released a JavaScript reference application, built on Backbone, Marionette, and RequireJS.

I’ve learned a lot over the past several weeks, and at times the learning curve was steep, partly because I couldn’t find a good reference application that I could learn from. To-Do apps are the classic example, but they’re too trivial to demonstrate how to architect a larger application. I’m hoping this resource will help fill that gap.

I’m still learning this stuff and am certainly no expert, but I’m happy to share what I’ve learned.

Lessons in Bug Hunting

Yesterday’s lesson in bug hunting: don’t assume you’re an idiot. I spent a few days trying to figure out why my success callback wasn’t being called. It had been working before I updated to jQuery 1.9.0, and I didn’t think I had changed anything. After much head scratching I found out that jQuery 1.9.0 introduced an Ajax() bug where HTTP status 204 is considered an error. A fix is in the jQuery master branch and will be in jQuery 1.9.1.

Today’s lesson in bug hunting: don’t assume you’re not an idiot. I spent hours yesterday and this morning trying to get Mousetrap.js working. I triple-checked my code against Craig’s documentation, verified the library was loading in the browser, etc. It should have worked. But I set a breakpoint on the line that was throwing the error and there was simply no Mousetrap in the global namespace. Having ruled out an error on my part, in desparation I opened mousetrap.js, hoping to find the bug in there. Instead I found… nothing. Yep, something had gone wrong when I downloaded it, and the file was completely empty.

Bottom line: keep in mind that everyone makes mistakes.

Shipped!

I launched a new intranet application today. Nothing fancy, just a simple app to address a real need in my organization. Initial feedback has been very positive. Feels good to ship!

Washingsoft UAnix

Washingsoft UAnix

Here’s another product parody Lars and I did in college: Washingsoft UAnix. This was born out of our frustration with the way the University of Washington had—shall we say—embraced and extended standard Unix functionality.

Microsoft Orifice

Microsoft Orifice

Aaahhh, the joys of Photoshop and free time. I just stumbled on this little trip down memory lane—a parody of a Microsoft Office box that my roommate (Lars Blacken) and I did in college.

How to Override IE’s Compatibility View Behavior On Intranet Sites

To force IE to edge mode (even on intranet sites, where IE would otherwise use compatibility mode), the server needs to send X-UA-Compatible as an HTTP header. Using a meta element in the document head doesn’t work reliably.

I found the answer buried in this Stack Overflow thread.

In my case, I was working on a WordPress theme, so I added

header("X-UA-Compatible: IE=Edge");

at the top of header.php.

You can also do it at the web server level with Apache or IIS (I haven’t tested either of those methods).

UPDATE 3/13/2014: I have now verified the IIS method linked above.

Single Sign-On Epiphany

When I wrote about my experience setting up AD Single Sign-On for Linux, I said the next step was to extend the transparent SSO experience into WordPress. The biggest reason for that—I thought—was so that the WordPress server could then impersonate the logged-in user to pull resources from our SharePoint server (using SharePoint Web Services) and include them on WP pages. Basically a WordPress front-end with SharePoint doing some Digital Asset Management duties on the back-end.

The epiphany I just had is that it wouldn’t be WordPress connecting to SharePoint, it would be PHP, which already knows who the user is, thanks to the Kerberos authentication I already have set up. I don’t need to tackle the WordPress part before I can build the SharePoint part.

Transparent SSO to WordPress is a benefit mainly for content creators, editors, and admins—those are a small percentage of my total user base, and managing their accounts is relatively easy.

Heads-Up for LinkedIn Users

If you have a LinkedIn account, stop what you’re doing and change your LinkedIn password immediately. I’m not kidding–just do it. Once you’re logged in, click on your name near the upper-right corner, click Settings from the menu, click the Account tab near the lower-left corner, and click Change password.

Now that you’ve changed your LinkedIn password, think about all of the other web sites where you have accounts–did you use the same (now-probably-hacked) password on any of those? If so, go change those, too (and don’t use the same password this time). If you use the same credentials across multiple sites, all an attacker needs is to crack one of them, and then (in principle) they own any other account with the same username and password.

Done? Great! So here’s what’s going on:

The social networking website LinkedIn is investigating claims that more than 6 million passwords were stolen and uploaded to a Russian-language web forum today.

That was yesterday, June 6. To be clear, it was actually cryptographic hashes of the passwords that were stolen–not the plain-text passwords themselves–but LinkedIn was using an insecure technique to generate the hashes (unsalted SHA-1). I won’t write here about why that’s so easy to crack–Steve Gibson had a good discussion about this in his Security Now! podcast, episode 356 (the transcript is not up on that page yet as of this writing, but he should have it posted soon). For some good guidance on choosing passwords that are resistant to the kind of attacks (“rainbow tables“) that are effective against unsalted hashing schemes, see Steve’s Password Haystacks page.

Converting to Project Connection Across Multiple Packages in SSIS 2012

I’m migrating a Business Intelligence project from SQL Server 2005 to SQL Server 2012. Microsoft has, overall, done a great job with their development and migration tools, and some of the new features of SQL 2012 are great and will save me a lot of time going forward. One neat new feature in SQL Server Integration Services (SSIS) is Project Connections: you can define a connection at the project level, and all packages in the project automatically inherit a reference to that connection.

So this project I’m migrating has maybe 40 packages, many of which had the same two connections (primary source application and the DW database). In SQL Server Data Tools, you can open a package, right-click on a connection, and “Convert to Project Connection.” So far, so good. Problem is, all those other packages that have a connection of the same name will not inherit the project connection because the local one overrides it (by design). And if you open another package and delete the local connection, every task and data flow component that used that connection gets the dreaded red “X” icon–they don’t automatically revert to the project-level connection with the same name. Best I can tell, the only way to fix it in SSDT is to reconfigure every one of those broken tasks and components. The Internet is full of articles showing how to convert a connection in one package, but nothing gave me any clue what to do with the other 39 packages. I couldn’t accept that I would have to do all that–there must be a better way. Continue reading “Converting to Project Connection Across Multiple Packages in SSIS 2012”