Category Archives: Programming

Importing thing to remember: This tech stuff is secretly fun

There’s a lie we tech folks tell ourselves–and anybody else who will listen–that what we’re doing is tough, arduous, and demanding work. It involves impossible demands, long hours, and nothing we do is ever completely perfect, or completely finished. In short, programming is hard work.

We start out telling folks this because it is difficult, demanding stuff, and we do put in crazy amounts of work at the keyboard, and stress ourselves out thinking about it all the time. We would like your admiration for all we go through, but since what we go through is largely incomprehensible and makes terrible stories at dinnertime, we’ll settle for your pity.

We tech types work marathon shifts, drink coffee by the gallon, and run ourselves ragged, all the while telling ourselves how bad we’ve got it. But here’s the truth–a truth we rarely admit to even ourselves: we’re secretly enjoying the whole thing, because it’s just about the most interesting game we’ve ever played.

Solving technical problems on computers is fascinating stuff, involving incredibly complex mental models, and elaborate systems built out of nothing but thoughts organized into code. It’s an amazingly creative pursuit, where you can think of an idea for a feature, and merely by thinking through all the angles of it in enough detail, can convert that idea into code, and thus into existence.

Even debugging is a challenge better than most mystery books, where your job is to solve the mystery of some strange occurrence, see it for what it really is, and slowly trace it back to the malformed code which perpetrated it. From there, you have to either tear up the bad code outright (risking the creation of new bugs in the process), or craft a suitable fix to restore the proper working of the system.

This stuff is downright fascinating. And on many days, we’d do it even if we didn’t get paid.

But we do get paid–and most of us fairly well at that–because it’s indeed difficult stuff, which requires ridiculous amounts of patience and knowledge, as well as a constantly replenished skill set to keep up with a technical landscape which is ever changing.

As deadlines loom and project problems pile up, it can also have an element of terror built in, as problems which would be fun to solve if there was more time to muck about with them suddenly become the narrowing jaws of a vice that threatens to crush you if they aren’t worked out by the time the ship date arrives.

At times like this, I often think back to a notable economics lecture in college where the professor demonstrated the law of diminishing returns by proposing “the Beer Game”. The  game goes as follows: He agrees to buy a string of beers for you, but whenever he buys you a beer,  you have to drink it.

“Wow!” we all thought, and on the hot autumn day he told the story, I was thinking that the first beer would taste awfully good indeed. The second would also be good–if not quite as refreshing as the first. But as the game goes on, each beer brings you less and less pleasure, and you’d eventually reach the point where you’d be willing to pay the instructor to be let out of the game.

Getting a software release out reminds me a lot of that game. As you dream up features and do the initial exploration, it’s a dream job, full of hope and creativity. As things ramp up and you have to work the technical details out, it’s still engaging, but seems more work and less fun. And then there’s always the terrible crush of deadlines at the end, where the non-stop work and pressure make you feel like the college kid trying to get out of the old econ professor’s beer-buying game. Only in this case, the only way out is through.

But then, if you don’t crumble under the pressure, you ship the product, the pressure evaporates, and in a while, a part of you starts looking forward to doing it all over again.

As I write this, I’m still catching my breath a bit and doing a little tidying up after a very major software release indeed. And yes, I’m happy to tell anyone who will listen about all the pain we went through to bring our new creation to life. We’re all incredibly proud of the new release, but there’s definitely a part of us that knows that there are far easier ways to make a living, and wonders why we keep putting ourselves through all this.

And here’s the truth: it isn’t just a way to earn a living, and it isn’t entirely about the pride we have in bringing something cool to life (although there’s a lot of that too). It’s because despite it all, it’s the most fun we’ve ever had while”working.” And despite our grousing–even to ourselves–we secretly know we’re having a blast.

One for the Programmers: Daily WTF

If you’re not a VB.Net or C# geek, probably best to move on to the next post. But if you are, perhaps someone can explain to me the what the !@#%! is going on here.

While hacking away on a new code project which involved porting a routine from VB6 to VB.Net (4.5 framework), I discovered that my performance had gone completely to hell on a function which assembles a big text file from database values. In it, I tracked the trouble to a function checks to see if certain values from the database were null, and returns “safe” values in case they are. It also looks for telltale “null” date fields, with values like 12/31/1899 which–although not technically null, are acting as blank for these purposes.

Here’s code sample #1:

  Public Shared Function ConvertNulls( _ 
     ByVal theString As Object) As String
      If IsDBNull(theString) Then
        ConvertNulls = ""
      ElseIf theString = "12:00:00 AM"  OrElse theString = _
           "12/31/1899" OrElse theString= "12/30/1899" Then
        ConvertNulls = "" ' This is a null date string
        ConvertNulls = theString
      End If
  End Function

And code sample #2:

  Public Shared Function ConvertNulls( _ 
     ByVal theString As Object) As String
      If IsDBNull(theString) Then
        ConvertNulls = ""
      ElseIf theString.Equals("12:00:00 AM") _
         OrElse theString.Equals("12/31/1899") _
         OrElse theString.Equals("12/30/1899") Then
        ConvertNulls = "" ' This is a null date string
        ConvertNulls = theString
      End If
  End Function

Here’s the thing:

Code sample #1 was on track to take a hour or so to run through 100,000 iterations; code sample #2 did the same 100,000 iterations in about 20 seconds. And the only difference? Whether I used theString = “x” or theString.equals “x”

Luckily, I thought to try .equals pretty early on, or else I’d be pulling my hair out trying to figure out why there should be even a tiny difference between the two approaches. Can anyone out there give me a coherent explanation as to why using the “=” operator vs. the .equals method should yield such wildly different performance results?

Tip: Solving “Restart Required” problem installing SQL Server Express x64 on Windows 7

There’s a bug in the current build of SQL Server Express which prevents it from being installed on a Windows 7 machine (the pre-installation checks will fail on a “restart required” link. Restarting, obviously, doesn’t fix this.

The solution:
Go to Start > Cmd > RegEdit and navigate to the following key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager

In the right-side pane, double-click on the key “PendingFileRenameOperations” and clear any values in there. Exit RegEdit.

Then, just re-run the SQL Express intaller and all should be fine.

Internet Explorer 8 Ships, Fixes Dropdown List Problem

Microsoft finally made Internet Explorer 8 a “recommended download” with the latest set of patches on Microsoft Update. At first glance, there’s not a ton new with IE 8, but it does fix one incredibly annoying problem with all previous versions, including IE 7: Dropdowns containing many items no longer take a huge amount of time to be built, freezing the page (and computer!) during the process. Firefox and other browsers haven’t had this problem, but it’s nice to see IE finally getting this one fixed.

Why is this one important? My big point of pain was the “List of Titles” under the seller inventories on Atomic Avenue. After complaints from users, we tracked down what appeared for all the world to be a hard crash in IE 6 and 7 to the user’s going to my personal titles on Atomic Avenue, where my list of over 10,000 titles in stock resulted in a drop-down list which could take several minutes to fill on a fast computer.

Cutting down the number of titles resulted in an exponential decline in the time required, leading me to guess it was a case of the programmer equivalent of the blonde joke about painting lines on the highway–basically, the programmer was adding onto the end of the list in a way which involved constantly going back to the start, counting to the end, then tacking on the item there, rather than setting an index point at the end and tacking new data on from there directly.

On modern machines, the lazy, brute force way of adding to lists by counting from the beginning is normally not a problem, but when thousands of items are involved, you can quickly set up conditions so that the computer must walk up a number of items in the list equivalent to  [the number of items in the list] raised to second or third  power—each and every time they want to add a new item. Repeat that ten thousand times, and you can see how little programming inefficiencies, repeated with very large numbers, can become killers.

Once we discovered the original problem in Internet Explorer, we’d been forced to cap IE clients to seeing the first 2000 titles worth of content from a given seller when viewing individual inventories. This only affected a couple of sellers on the system, and was largely a temporary measure until we could address the matter in a more satisfactory way.

Unfortunately, one of those sellers with wide-ranging titles for sale was me! (By nature, we tend to grab one copy of everything in order to throw it in ComicBase).  Happily with IE 8 now available, we’re safely able to remove that limit (and warn users of older versions that they could do better if they upgrade to the latest version, or use another browser like Firefox which never had that particular problem). We’ll track browse usage in the months ahead and see whether we still need to engineer a workaround for older IE user.

(I’m hoping not, frankly, since any fix would not only be reasonably complicated, but would also involve a fair amount of overhead to load up the list, realize that there’s too many to be safely be shown by old versions of IE, then display smaller batches in a safer way. Having Microsoft simply fix IE seems much preferable, although if nobody ever updates their browser, we may have to rig up the workaround anyway, I guess…). In any case, kudos (and thanks!) to whichever person on the IE 8 team fixed this one!

Stopping “Invalid Callback or Postback” errors when using Ajax with dynamically filled dropdowns

This is one of those posts I’m just putting out there in the hopes it saves some other poor programmer the hours and hours worth of headaches that we just went through.

When we were developing the new search control panel for Atomic Avenue, we set it up so that several of the dropdownlists fill up with data based on other selections. For instance, if you choose to search by artist, a list of artist names will appear to choose from. As it turns out, such data-driven fields—particularly if they’re not always made visible until a later action (such as indicating you want to search by Artist) unhides them—is a great way to generate “invalid postback” errors under Ajax. This is especially true if you combine them with the AddHistoryPoint method in Asp.Net Futures’ July release to enable Back Button support.

The problem is that the new security controls in ASP.NET 2.0 can’t validate that the postbacks on the page were generated by those controls in the first place since (a) the contents can change as a result of the postback, and (b) the event validation may not have been set up in the first place if the control in question was hidden to start.

For the best rundown on the issue, here’s  the critical blog post by K.Scott Allen (check out part 1 as well)

Three options for getting around the situation presented themselves:

1. We could just disable the event validation checking, but that would open us up to new and fun injection attacks from folks faking postback data and using it to trick our web page into doing various inconvenient things.  In particular, we didn’t want to do that with our search since it was meant as a system-wide facility, meaning it would be used on virtually every page in the system.

2. We could avoid Ajax altogether and use traditional, full-page HTML rendering. This wasn’t a good option for us, since the Ajax-enabled version is *much* faster (and better looking) than the traditional HTML architecture which would require full page redraws after every control selection.

3. We could try calling Clientscript.RegisterForEventValidation and supply all the possible values of each dropdown prior to validation, but since there were often thousands of possibilities, this was impractical in the extreme.

So what was there to do? The answer was to roll our own version of the Dropdownlist control, leaving out the (non-inheritable) SupportsEventValidation attribute. By replacing the three problematic dropdownlists with the new control (“DynamicDropdownList”), we were able to get things working (including back button support) without tripping all over this rather thorny problem.


Code Sample