If I had to pick one flaw in the geek culture, it’s the lack of respect for the effort it takes to get people to use your software. Take any geek, point them at a successful product with lots of users like Facebook and they will talk about its limitations and how they could code up a better product over the weekend. It is not the technology that makes products like Facebook a success but rather the endless effort to get people to use them. Even if they could code up a better Facebook over the weekend, who would use it?

When business people talk about distribution they are referring to not only how many stores the product is in but, as in the case of software, how many users are actually using the product. The greater the distribution the more valuable the business. Make no mistake, try to sell a business that has no distribution and there will be surprisingly little interest.

And much to the frustration of the software purest, most products with distribution tend to be technically inferior. A widely cited example among the tech community is Microsoft Windows—which is generally seen as technically inferior to Linux. Although this can be debated, what everyone can agree on is Windows has distribution and Linux doesn‘t.

Ever stand behind some rental car place and peer over the counter to take a peak at the software only to be greeted by a blue screen running some DOS app and while the poor employee struggles to enter your information into line 64, your mind starts thinking about how you could create an awesome WPF application and make millions selling it to these poor technology illiterate people. Surely they wouldn’t say no.

But then they say no.

Recently, I tried a little experiment. I noticed the forum software my Home Owners Association was using was lacking. I explored other HOA forums and noticed that not only did my HOA forum need help but other neighborhoods as well. They were using software that looked years outdated like FUDForum and the notorious phpBB and like the rental car DOS app experience above, I thought I could do better and over a couple months in my spare time I coded one up in ASP.NET MVC. It included features like an HOA corner where HOA members could post events, topic and response voting, a hot section that ranked active topics, and a snappy design.

I had confidence my neighborhood would adopt it so I fired off an email to the HOA members with a link to the demo site. Not known for their promptness, I wasn’t initially worried by the long delayed response but when the response finally came I became concerned. There in front of my eyes was a response I was not anticipating. It simply stated that the HOA had a forum already—as if I didn’t know—and they hadn’t received any complaints about it. They closed by thanking me for the thought and as quickly as the email began it was over.

Not completely dejected—and still thinking someone out there must want to use it—I tried other local neighborhoods. If my own HOA response was unexpected the others proved to be also. No interest even though it was free.

What I had failed to anticipate was not the failure of adoption due to lack of features but rather the total lack of interest. No one would even try the thing out and as I began to think about why that was, I began to see that my software failed to solve any real problems the HOA was having. What I thought were problems turned out to be non issues. It was as if I was trying to sell pay-off-your-debt software to multi billionaires.

And the more I thought about the lack of adoption the more I came to the conclusion that I didn’t even know what HOA organizations did. I just saw some issues I thought were problems and coded some software that solved those problems and then expected everyone to jump on board.

It turns out, the real secret to getting adoption and hence distribution is to find real business problems that can be solved by software. As I have demonstrated above and the number of failed startups can attest to, finding a problem and solving it well enough that someone desperate enough for a solution will take a change on your unknown software and try it out is extremely hard.

It is better to be a person that knows of a problem that can be solved by software than a software engineer with skills to create a program that solves no problems.

Few things frustrate more than driving that shiny new sports car home from the car lot with a sinking feeling of knowing you didn’t get the best deal. If there is one mistake we make, it is not respecting the salesman ability to influence our decisions.

To the trained, these techniques are well known and used not just by shady car salesman but by business leaders as well. Unlike harsh techniques like yelling, these play on our natural human instincts. Instincts as basic as to fit in and be liked. Used effectively, they can cause us to make decisions that we might not otherwise make.

The book Influence: The Psychology of Persuasion outlines these techniques. It is important to learn not only how to recognize when we are being put under the magic spell but also to cast it.

Contrast Principle

In a world of constant competition we learn to compare. We compare our houses, academic grades, athletic ability, and professional success to others. These comparisons—although sometimes unfair—allow us to measure where we stand relative to others.

But while these comparisons help gauge our progress they can also work against us. What salesman and others know that we don’t is when comparing two different things presented one after another and the difference between them is large we tend to perceive them as more different than they really are.

This weakness becomes more apparent with some examples. A study was done where college students were asked to rank average looking members of the opposite sex after looking through ads in some popular magazines. Their attractiveness was ranked lower by the ones who first looked through the magazine of models than by others who didn’t.

Placing a subjects two hands in buckets of water, one of which is hot and the other which is cold and then putting both hands into lukewarm water makes the lukewarm water feel cold by the warm hand and warm by the cold hand.

What both these examples make clear is the second comparison can be drastically affected by the first. But what is more frightening than these harmless studies is what can be accomplished in the hands of those with more sinister intentions. Retail stores, for example, are taught to show the most expensive item first and then move to less expensive items. This makes the later items seem cheaper than they may be. Some realtors keep overpriced run down homes—that they have no intention to sell—to show first thus making later average looking homes seem like glorious mansions.

Car dealerships understand that once you are committed to buying a car, little add on accessories that might otherwise seem very expensive on their own are now less so. After spending $15 grand on a car, what is $500 more for window tinting or that flashy racing spoiler.

Reciprocation

Who doesn’t like a nice gift. But that innocent little gift comes with a steep price. Those who want the law of reciprocity to work for them understand that humans who receive something for nothing have a desire to repay the gift sometimes by giving more in return.

A simple experiment shows the power of reciprocity. In the experiment two individuals were part of a study about rating paintings, however, the study was not about paintings but the effects of reciprocity. One of the participates after a short break bought himself and the other participant a Coke and when the session was over asked a favor of the other. He was selling tickets for a raffle at 25 cents a piece and if he sold the most he would win a 50 dollar prize. The participants who received a Coke bought twice as many tickets as those that didn’t with some buying as many as seven. With Coke costing just 10 cents, the deal was well worth it.

While pushing Coke and raffle tickets shows that reciprocation works, a little known variant called rejection-than-retreat can be just as effective. It combines reciprocation with the contrast principle to produce a surprisingly potent rule.

Rejection-than-retreat starts by proposing an offer that will most likely be rejected and then immediately offering a smaller offer. The smaller offer is a concession that in some weird psychotic way we want to reciprocate. If the reciprocate rule doesn’t do us in, the contrast rule quickly will because the second offer is smaller than the first thus making it look much smaller that it really is.

Of all the ways the rejection-then-retreat rule is deployed none is more effective than the selling of extended warranties. If the salesman is any good the rule is played by offering you some overly expensive warranty plan that most reject off hand. After the rejection, a second more reasonable offer is made. The reciprocation and contrast rule are both in play making the offer hard to resist. Very high success rates of 70% or more have been achieved by using this tactic alone. 

Commitment

It is human nature that once we commit to a plan of action we are very likely to follow through. The principle multiplies in effectiveness if the commitment is made in writing or publicly.

During the Korean war many Americans found themselves prisoners or war in the communist China camps. The Chinese knew about the power of commitment and got the Americans to first admit little things like: America isn’t perfect. Then they were pushed to elaborate reasons why and asked to make a list and sign their name. The commitment starts little and expands until the Americans start to defend their commitment to others.

Tactics used by the Chinese are also used by companies to gain better customer loyalty. Many companies like Proctor and Gamble have product praising essay writing contests—which to the untrained eye looked cheesy—that have proven effective in increasing customer loyalty.

Yet of all the ways to get a commitment, none are more effective than the well known low ball technique. Auto sales are masters of this technique which starts with a low price. The customer than becomes committed to buying the car as time passes. When the deal is ready to close, however, some error is discovered that raises the price a couple hundred dollars. You have already made the decision to buy the car and it takes Herculean effort to walk away from the deal.

Social Proof

The universal law of human behavior is if lots of people are doing something it tends to be the right thing. Salesman often recount endless tails of happy customers who have purchased the same product you are considering buying. Restaurants have been known to limit seating so long lines form outside giving passer-byers the proof of its popularity. Online forums and communities are often seeded with topics to give the impression of importance.

Scarcity

Of all the laws presented above, the law of scarcity is perhaps the most effective. There is no greater draw then to be told you can’t have something.

The game is typically played by retailers by first noticing that a customer is interested in a particular item. Then the salesman—to increase interest in the item—informs them the item is a great item but unfortunately the last one was just sold. The scarcity of the item increases the desire to buy. Miraculously, upon seeing intensified interested from the customer further investigation reveals one is located in stock after all.

The best technique, however, I have heard for utilizing the law of scarcity effectively is from a guy trying to sell his used car. To make the item appear scarce, he double books appointments to show off the car. When the second person arrives on scene to find another looking over the car he is informed to wait his turn while the first person looks it over. The first person seeing the second, becomes more interested in the car. The car becomes a scarce resource and competition increases. Between the two of them they have no hope and quickly cave to the law of scarcity.

It’s not that centralized version control is bad but rather it has a habit of getting in your way. Take creating branches. In Perforce, for example, creating a branch makes you feel more like being interrogated by the FBI than a developer wanting to set up a sandbox. Integrating and reverse integrating branches takes more work than necessary. It got so bad we abandoned branches.

A strategy of not branching—with one big trunk—may make merging a non issue but brings limitations. Complicated bug fixes that should be fixed in isolation are not an option. Instead, checkins like “bug fix part 1” and “bug fix part 2” are common leaving the trunk in a precarious state. Even worse, experiments are impossible to do in isolation discouraging innovative thinking.

Distributed version control got its start in the open source community where experimentation is essential. In distributed version control (DVCS) branching is a first class citizen. In fact, when the code is brought down locally from the server—called cloning in Mercurial—you have a copy of the entire repository. It is a self contained unit that doesn’t need to contact the server ever again. Checkins, diffs, reverts, and updating to earlier revisions all happen locally. If the server goes down, who cares?

While visions of developers being productive on airplanes, parks, or the local coffee shop brings a smile to every software manager’s face it isn't the real reason DVCS are becoming popular. Instead, it is the ease with which every developer gets their own sandbox. If the experiment goes bad simply delete the repository from the file system and start over. The server doesn’t have to know about your copy and no one is the wiser.

Not only can sandboxes be cleaned up easily but they can also be shared. Mercurial has a built in web server that allows changes to be pulled from your repository by others. This can be very handy when two or more developers are working on a feature and want to share unstable changes. No one outside this circle has to be bothered by changes going into the trunk just so they can reach a targeted subset.

If sharing repositories is easy then merging them must be a nightmare. But it turns out the most impressive feature of DVCS is the speed at which merges are done. Mercurial, for example, has a rename feature that automatically moves changes to the correct renamed files.

I started to experiment with Mercurial for my home projects and was impressed with its easy to learn command set and light weight feel. I was so impressed, I recommended we move to it at my new job. Other like minded developers I work with agree.

Few companies could compete with Bucyrus Erie in the early half of the twentieth century. Their cable pulley excavators powered by gas engines—capable of holding up to 5 cubic yards in their massive steal jaws—where vital to the mining, canal digging, sewer and piping industries. Ironically, their biggest competition threat would not come from other ambitions pulley excavators but from a little company called Sherman which was pushing a small hydraulic shovel that moved a puny ¼ cubic yards and attached to a farm tracker. They named it the backhoe.

The backhoe began innocently enough with the invention of hydraulic technology in the late 1940’s. Lacking the earth moving capacity and distribution of Bucyrus, companies like Sherman designed their backhoes to be compatible with popular farm tractors of the day like John Deere. Unexpectedly, the backhoes found a devoted following with residential contractors who appreciated their maneuverability allowing them to squeeze between houses to dig water and sewage lines. Without the machines, the trenches had to be dug by hand which made even the most devoted ditch digger want to beat themselves senseless with their own shovel.

If Bucyrus was afraid of the hydraulic threat, it didn’t show it. Bucyrus viewed the cute backhoes as too wimpy for their customers needs. Their customers, after all, were demanding bigger bucket sizes that hydraulics couldn’t match so Bucyrus continued investing in cable technology to meet existing customer needs.

And while Bucyrus was putting their best engineers minds to work squeezing more tension out of those steal cables, other smaller companies like Caterpillar took notice. With Caterpillar and others pushing hydraulics they improved at an astonishing rate. In 1955 bucket size was 3/8 cubic yards, ½ cubic yards by 1960, and 2 cubic yards by 1965. Only 9 years later in 1974 bucket size reached 10 cubic yards and continued to climb skyward. It was as if Caterpillar showed up one day in front of Bucyrus headquarters with a 10 cubic yard hydraulic machine and offered to dig their sewage lines.

In reality, Bucyrus knew all about hydraulics they just didn’t care. Only after hydraulics capacity reached 2 cubic yards did things began to go south. At that size, sewer and pipe contractors became interested for the first time and with sewer and pipe contractors representing a good chunk of their business they began to lose revenue. Even worse, at 5 cubic yards hydraulics started to dig into their prime business of general excavators. How did this happen?

To outside observers the Bucyrus dilemma seems easily preventable. For starters, they could have invested in hydraulics technology earlier allowing them to build up their expertise. With this expertise they would be better able to compete in the new hydraulic market when hydraulics reached sizes their customers cared about.

This criticism is justified but the answers are not clear cut. When hydraulics first came out their usefulness was unclear. At ¼ cubic yards their functionality was limited to small markets. After years of doing business, Bucyrus had built up a cost structure that needed certain revenues to sustain. The hydraulic market initially was not big enough to justify significant investment from them. The revenue generated would be small at best when compared to their cost structure so Bucyrus felt their resources would be better spent improving their existing products. After all, no one knew if hydraulics would continue to improve.

The uncertainly around how disruptive technology will improve and whether established businesses should invest is a hard decision to make. In the book Innovator’s Dilemma, Clayton Christensen describes the struggles business leaders face when confronting disruptive technology like Bucyrus faced with hydraulics. In one part, Clayton describes that during the 80’s established computer disk drive companies where being attacked by disruptive upstarts sporting sexy smaller drives. One of those established companies, Seagate, was struggling in 1985 with whether to make 3.5-inch drives when they were already the leader in 5.25-inch drives.

Seagate was confronted with this decision when overly ambitious engineers developed working 3.5-inch prototype drives. Seagate marketing showed the drives to IBM PC division even though the capacity was clearly lacking compared to the existing 5.25-inch drives. IBM was not interested as their computers were designed for 5.25-inch drives and they were looking for greater capacity not less. As a former Seagate manager put it: “We needed a new model which could become the next ST412—a very successful product generating $300 million sales annually in the desktop market that was near the end of its life cycle. Our forecasts for the 3.5-inch drives were under $50 million because the laptop market was just emerging, and the 3.5-inch product just didn’t fit the bill.”

Seagate scrapped the 3.5-inch drive and paid dearly for it later as the drives got more popular. But here lies the crux of the dilemma. As firms get bigger their cost structures get bigger thus making it harder to innovate. Disruptive technologies have uncertain markets with low margins which established firms don’t like because they are liable to their share holders to grow and make more money. It is hard to justify dabbling in low margin uncertain markets when existing products with known markets and large margins can be improved.

So ironically, as companies get bigger they become more vulnerable to being attacked from below by small innovative firms. Lower cost structures allow startups to experiment with new technology. This is why innovation typically comes from small firms. If the technology improves, it finds new markets and can—as we saw with Bucyrus—began to steal away customers from established firms. If enough customers are stolen, the established firm will go out of business. This often happens and why we see large companies fail.

Even today, a glance at Bucyrus website reveals it still hasn’t penetrated the hydraulic market. Caterpillar has that covered.

In my family—in laws included—I am the engineer weirdo performing voodoo magic to make computers do things. The majority work in business related fields and tout MBA degrees with years of experience to back them up but somehow still think that no matter where I work the company must be creating some kind of operating system. In this type of family environment discussions consist not only of how are the kids or pass the mashed potatoes but things like calculating percents—like say 35% of 60—or calculating how much money will appreciate at 8% over 30 years. If there is one thing I have learned it’s that nothing will bring quicker looks of disdain then to trip up calculating percents to the exact number. Close enough doesn’t cut it.

At first, I didn’t know whether to feel disturbed or envious at the speed at which these calculations came spewing out of their mouths like hungry bats emerging for a night feed. The disturbed part of me felt they had spent way too much time practicing the skill; after all, anyone could grab a calculator to do the same thing. The envious part of me couldn’t help but admire it for a computer program would be hard pressed to beat them. Still, I couldn’t pinpoint why this was so important.

Years later, while reading business books, it dawned on me that in the world of understanding how businesses work calculating percentages is essential. Not knowing them would be like coming across a program without for loops causing the intelligence of the author to be questioned.

And like for loops in programs, percents are everywhere in business. What percent of revenue is spent on R&D vs. Sales vs. Marketing? How do those percents compare to our competitors? What percent is our revenue going to increase? Every thought and comparison is expressed as a percent. Spend any time studying financial statements and there is no way to escape them.

Despite this, it took me—who took many courses in calculus and physics—years before I understood how to calculate a 15% tip in my head. Shamefully, I have found that many engineers also struggle with this skill. It is time we stop being shown up by our business associates and learn how to calculate percents and value appreciation in our heads without the help of our favorite programming language.

Percent calculation

Remember Chunk from the 80’s movie "The Goonies". His name not his character—the paranoid husky kid who would rather be eating rocky road ice cream than contributing to the goal of finding the treasure of one-eyed Willie—holds the secret to calculating percents in your head. The technique that resembles his name, called chunking, is commonly applied to memorizing numbers by breaking the problem into groups. Like numbers, percent calculation can be broken down into groups to simplify things.

First rule of percent chunking is 10%. 10% is easy to calculate because you move the decimal one place to the left. To reach 1% you simple do the 10% calculation twice in your head resulting in the decimal place moving two times to the left.  Calculating 5% is easy once you have the 10% number because you just divide it in half.

In our example above of taking 35% of 60 we first take 10% which is 6—simply move the decimal place of 60.0 one to the left. Now, we multiply that number by 3 to get to the 30% number of 18. We know that 5% is just half of 10% which is 3 in our case. Add that to 18 and we get the final answer of 21. So mathematically this can be chunked like: .35x = .30x + .05x = (.10*3*x) + (.10*(½)*x) = [(.10*x)*3] + [(.10*x)*(½)]. Substituting 60 in for x using the last equation gives us the same thought process previously worked through. The key is to master this chunking technique in your mind.

You can use subtraction as well. Say we want to calculate 18% of 95—which I just made up and figured to be 17.10. Here is the chunking equation: .18x = .20x - .02x = [[(.10*2)*x] - [(.10*.10)*2*x] = [(.10*x)*2] - [(.10*x)*.10*2] = (9.5*2) - [(9.5)*.10*2] = 19 – (.95*2) = 19 – 1.9 = 17.10.

Practice doing random problems in the shower every morning and at the very least you will amaze the helpless Gap employee who can’t seem to calculate 20% of anything without the register.

Value Appreciation

Calculating how your principle will grow after x years at y percent is easy if you know the secret Rule of 72. After discovering it on my own I would be lying if I didn’t admit I was angry at my family for not informing me of it.

The rule, which is surprisingly accurate and powerful, is simply: take 72 and divide it by the interested rate to get the number of years it will take to double. So back to our example of 30 years at 8% which is a common retirement account calculation. The rule of 72 says that at 8% your retirement money will double every 9 years—72 / 8 = 9. So if you have 20k in your account, in 27 years you should have 160k—3 doubles: 40k, 80k, and finally 160k. If the calculation is done manually—20k*1.08^27—it gives 159,761.

Thinking more about the rule will reveal some interesting facts beyond retirement appreciation. For example if inflation is 4% a year, how long will it take before prices double? The rule of 72 says 18 years; so in 18 years if your money just sits in an account somewhere earning meagerly interest its purchasing power will be cut in half. Even more depressing, that sweet little baby you hold in your arms now will grow up and when they are 18 years old and ready to go to college everything will be twice as expensive.

Bonus - Multiplication

Cost per month is a common price scheme; like a car payment of $250 per month. To calculate the price per year effectively in your head, it is important to move in chunks of 10. To do this, factor 250 into tens like 100*2.5 and then multiply by 12 to get final chunks of 12*100*2.5. Taking 12*100 gives 1200. We need 2 and a half of those. A half is 600 and 2x is 2400. Add them together to get 3000.

A better approach to the same problem is to rearrange the numbers like 100*(12*2.5). This allows the more complicated math to be done on small numbers which becomes more important as the numbers get larger. Thinking 2.5 times 12 is 30—[2*12]+[(½)*12]—then multiplying by 100 (or equivalently 10 two times) to get 3000 is easier than multiplying 12 by 100 and taking 2.5 of that.

A third option is to use elementary algebra and the distributive law like 12*250 = (10 + 2)*250 = (10*250) + (2*250). This gives us the nice number 10 in one factor and the low number of 2 in the other. The math from here is easy to do in your head. 2500+500 = 3000.

The three methods above were given not to show which one is better but to give three different methods for attacking multiplication. For certain numbers, a particular method may work better which makes knowing as many methods as possible essential for doing fast multiplication in your head.

One more example. Employees get paid $1500 a month and there are 550 of them. What is the expense per month? 100*15*100*5.5 = 100*10*10(15*5.5) = 1,000*10[(15*5) + (15*(½)] = 1000*10*82.5 = 10*82,500 = 825,000.

For years I made the same dull walk from my college apartment to campus and the only part of the walk that perked my interest was the daily charade that occurred outside the business building. Business students with their laptops, mochas, and Wall Street Journals hurried around as though they were late for an important board meeting obviously too busy to look up from their paper to make eye contact or smile. So pervasive was the attitude that even humble freshman couldn’t prevent their egos from tripling after spending an hour inside the business building with its huge foyer complete with fancy water fountains, nice leather lounge chairs, and offices engrained with titles on the doors.

I couldn’t help but feel slighted. The computer science building was a homely looking square building no doubt designed by an engineer trying to maximize space. The computer labs were located in the basement down a lonely stair case and no matter how hard I looked I was never able to find the leather chairs.

I spent many hours down in that basement banging on the keyboard coding quicksorts, binary trees, huffman codes, and traveling salesman. Sometimes when it got quiet and the hum of the many CPUs seemed to be playing the prelude from Bach’s Cello Suite No.1, I would think: What do those business students study all day anyways?

It was a few years into my career before I became curious enough to investigate. During the process, I became enamored with Joel Spolsky and found his recommended business books page which I quickly devoured. Of all those books none explained marketing concepts more concretely than 22 Immutable Laws of Marketing. Reading this book gave me more perspective than all those years walking by the business building. I will not go through all the laws just the ones I feel are most important.

The Law of Perception

Software engineers think the best code will win. This law says the real battle is not between code but how people perceive your product. Once someone develops an opinion of your product it’s impossible to change it. Take Honda for instance. In Japan, Honda is viewed as a motorcycle company and their cars do not sell well; however, in the US they do fine. Forget the product, the battle begins in the minds of customers.

The Law of Focus

The most important thing a product can do is own a single word in the mind of the customer. When they think of your product what word comes to mind? Prego spaghetti sauce got the upper hand on Ragu when Prego became synonymous with the word thicker which implies quality ingredients. Once thicker was owned by Prego, no other brand could use it because everyone knew thicker meant Prego. You cannot change the mind of the customer.

The Law of the Ladder

There is a natural ranking of products that all customers know. Knowing where your brand stands on the ladder should dictate the marketing strategy. Don’t claim to be #1 when everyone knows your not. This can only backfire. Interestingly, admitting your rank on the ladder can help as was the case with Avis. Their slogan—Avis is #2 and we try harder—greatly increases sales.

The Law of the Opposite

In the long run, the race becomes a battle between two brands. Think Coke/Pepsi, Scope/Listerine, and McDonalds/Burger King. The second place brand must not mimic the first but be opposite in some way. Some people don’t want to buy from the market leader so having a value proposition that is different attracts those types.

The Law of the Division

Every category over time splits into different vertical segments. Computers, for example, have developed many divisions like: mainframes, personal computers, laptops, tablets, netbooks, and now smart phones. Each separate computer division has its own market leader. This is a very powerful concept to understand because if a brand cannot compete with the market leader simply invent a new division of that category and plow ahead.

No industry understands this concept better than the beer industry. There is light beer, best imported beer, best domestic beer, and more recently low calorie beer. They even combine divisions to make best light imported beer or best light domestic beer. If there is one thing the beer industry has proven it's there is no limit to how a category can be sliced and diced.

The Law of Line Extension

Line extension is taking a successful product and introducing a new product around that same brand. Take A-1 steak sauce. The chicken market was growing and so A-1 poultry sauce was introduced with the thought that A-1 is so well known in the steak market people will want to use it for chicken. It was a complete disaster. Everyone knows A-1 is for steaks. 7-up also tried this same approach with Cherry 7-up. Same disastrous result. Line extensions don’t work. Instead, a new identity and brand must be created from scratch.

The Law of the Sacrifice

This law is similar to the previous one except it doesn’t apply to creating a new product but rather extending a brand beyond its identity. Federal Express was once known for overnight delivery but expanded their focus when they tried to capture more of the international market. The result was a company that lost its once strong identity of overnight delivery to become a company without one and lost a lot of money in the process.

Interestingly, Philip Morris did the opposite with Marlboro. It narrowed the focus from a general campaign to target both men and women to one that targets men only with a cowboy as the center piece no less. The result was a strong identity that helped Marlboro became the best selling cigarette for both men and women. A brand that stands for everything doesn’t work. In this law, less really is more.

It is often said that Microsoft doesn’t innovate but simply copies existing ideas. To argue in opposition would be no small feat considering it doesn’t take long for anyone familiar with the tech industry to rattle off examples from Windows to the browser. However, LINQ is not one of those. LINQ was wrestled forth from the keen mind of Anders Hejlsberg—the author of Turbo Pascal and chief architect of Delphi—who joined Microsoft in 1996 after being lured out of Borland.

What bothered Anders the most when he joined Microsoft was not his reported 3 million dollar signing bonus but how disconnected the existing programming models were for different domains. To communicate with databases, for example, you must know SQL and how to write stored procedures. The more they thought about the problem the more they realized different programming models were most prevalent around data query. The solution was LINQ—which stands for Language INtegrated Query. For an organization better known for its late game heroics rather than innovative ideas, LINQ sheds the stereotype by delivering a solid piece of software engineering which can query everything from arrays of object to SQL databases all from C#.

I have been using LINQ to SQL for some home projects and love it. Before you think it must be a slow library abstraction consider that Joel Spolsky and the gang at StackOverflow, which gets 6 million unique visitors per month, have been running it from the beginning with no performance issues to date.

To get started with LINQ goodness let’s look at a recent project I have been working on that screen scrapes grocery ads from the web and stores them in a database. Below is my simple database schema shown from within SQL Server Management Studio 2005.

From the schema each store can have many ads and each ad can have many items. Now that the database is defined, we can use the LINQ to SQL designer in Visual Studio 2008 to generate our data access classes that LINQ runs against. To do so, right click the C# project in Visual Studio and select Add>>New Item. Select the Data Category and then the LINQ to SQL classes template in the right pane as shown below.

A new .dbml file will be add to the project. Double clicking on it will bring up the a designer surface. The next step is to connect to the database using Server Explorer and then drag the database tables onto the LINQ to SQL design surface. To bring up Server Explorer, click View>>Server Explorer in Visual Studio. When the Server Explorer window appears right click Data Connections and select Add Connection. Follow the wizard to connect to a database. Once this is done the Server Explorer window should look something like the screen shot below.

Now multi select all the tables and drag them onto the .dbml designer surface. You will get a warning about the connection string being stored in clear text. Click yes because the generated classes have connection string overloads that can be used later. Now the designer should look something like:

The generated code lives in a partial class underneath the .dbml file in Solution Explorer ending in designer.cs. On closer inspection of the generated code, notice there is a class for each table. LINQ to SQL doesn’t have the ability to generate model classes that are different from the actual tables. This functionality is reserved for the ADO.NET Entity Framework which Microsoft has decided is their future data access technology. As such, LINQ to SQL’s priority has been changed to maintenance only. Even so, it’s as powerful as ADO.NET Entity framework just less flexible.

Now with the SQL model created I can use LINQ to query and add values to the database. To add values, I implemented the visitor pattern to insert scraped data. The HebDataContext class, below, is the data context that was code generated by Visual Studio and is the key class which LINQ queries are run on. I created an instance of it and then do a LINQ query to see if the store exists in the database. If not, it is added and then the children of the store—the ads—are visited. The code looks like:

void IElementVisitor.VisitStore(ScrapeEngine.Modeling.Store storeModel)
{
    _currentStoreID = storeModel.HebID;
    using (HebDataContext context = new HebDataContext())
    {
        DataAccess.Store currentStore = context.Stores.Where(x => x.ID == storeModel.HebID).SingleOrDefault();
        if (currentStore == null)
        {
            DataAccess.Store store = new DataAccess.Store()
            {
                ID = storeModel.HebID,
                Name = storeModel.Name,
                Address = storeModel.Address,
                City = storeModel.City,
                State = storeModel.State,
                Zipcode = storeModel.Zipcode,
                Phone = storeModel.Phone
            };

            context.Stores.InsertOnSubmit(store);
            context.SubmitChanges();
        }
    }

    foreach (ScrapeEngine.Modeling.Ad ad in storeModel.Children)
        ad.AcceptVisitor(this);
}

Where LINQ to SQL really excels is the queries. Below is an example that gets all the items in a particular store's ads that are missing a price—this happens when the item’s price is dependent on buying another item—all from C#.

using (HebDataContext context = new HebDataContext())
{
    string[] items = context.Stores
        .Where(x => x.ID == 580)
        .SelectMany(x => x.Ads)
        .SelectMany(x => x.Items)
        .Where(x => x.Price == String.Empty)
        .OrderBy(x => x.Name)
        .Select(x => x.Name)
        .ToArray();

}

This query above gives the following when I poked in the debugger.

[0] "any four (4) Lean Cuisine Entrées"
[1] "Beneful Dry Dog Food"
[2] "Bumble Bee Solid White Tuna"
[3] "Cheetos"
[4] "Deviled Eggs Party Trays"
[5] "Eckrich Meat Smoked Sausage Links or Rope"
[6] "Eckrich Premium Beef Franks"
[7] "Fritos Corn Chips"
[8] "H-E-B Cafe Olé® Coffee"
[9] "H-E-B Chef Prepared Rotisserie Chicken"
[10] "H-E-B Chocolate Milk"
[11] "H-E-B Fresher Lasting® Chunky Guacamole Kit"
[12] "H-E-B Fully Cooked® Burgers"
[13] "H-E-B Fully Cooked® Shredded Chicken or Beef"
[14] "H-E-B inControl(TM) No Coding Test Strips"
[15] "H-E-B Ridged Potato Chips"
[16] "Hebrew National Beef Franks or Knockwurst"
[17] "Kiolbassa Value Pack Smoked Sausage"
[18] "Live Louisiana Crawfish"
[19] "Maxwell House Coffee"
[20] "Oscar Mayer Lunchmeat"
[21] "Planters Peanuts"
[22] "Sunshine Krispy Saltine Crackers"

Behind the sounds of keyboards clanging out lines of code and second hand Aeron chairs brushing against the floor lies the passion of a dedicated few bringing to market a product they believe customers will actually give them money for. Like a bird working feverishly to build its nest, they dig for ideas and scavenge for employees while carefully constructing their business plan. If they execute well and build the product they envision the market will reward them because the path is laid before them and all it takes is enough determination to navigate it.

Unfortunately, the path can fork and the very skills that entrepreneurs must have like self confidence and egotism can make it hard to recognize forks. Many get so emotionally invested in their idea that when sales fail to materialize they rationalize it away by thinking the next feature or sales pitch will make the difference. Their mind can become so clouded that they come to expect the market to care about their shiny new product as much as they do. In reality, the market is not interested in shiny objects but rather in how those shiny things solve their problems. Much to the dismay of the entrepreneur, the market’s needs could be down the dodgy path, the one that nobody likes to walk down because it is covered with mud, weeds, and crabgrass.

Of all the ingredients that make a product a success, none are more vital than matching a product to what the market wants. Few have observed this process with more first hand experience than Eric Ries who was involved with a business which failed badly—blowing through 40 million in five years—and a later one which he started that was very successful. The failed first experience encouraged him to try something new with the second. A process he calls the Lean Startup.

The unintuitive first step of the lean process is to start with the assumption that you don’t know what the market wants but you are willing to learn. In order to learn what the market wants, you start with a small hypothesis of what you think it may want and then create what he calls the minimal viable product. The minimal viable product should be created with as little effort as possible and he goes so far as to suggest that it is commonly overridden with bugs. If that weren’t enough, he mandates you charge for it as well.

When I first heard the concept of a minimal viable product, I felt violated as a software engineer because a software engineer creates well designed low bug products not quick and dirty buggy ones. The thought of releasing something to the public that is buggy and worse expected to be buggy is nothing short of shameful. How could any self respecting engineer even consider shipping a buggy thing that will bring embarrassment to themselves?

What most engineers fail to understand, however, is the more time you put into a product without showing it to the market—what is commonly called stealth mode—the harder it is to change direction. Code that has lots of time, money, and reputation invested in it is hard to throw away and is often the source of emotional attachment which can ultimately drive a product into the depths of failure. This is precisely what happened in Eric’s first startup experience.

The real strength then of the minimal viable product is not just low emotional attachment but the ability to rapidly try different products ideas by releasing, getting feedback, and repeating until the market clearly signals what they want. The ability to change directions quickly and keep the burn rate low before a product/market fit is found is essential to the process of founding a successful business.

But how can one know when a product/market fit is found? No one has a better acid test for this problem than Sean Ellis. His method is shockingly simple. Survey the customers with one simple important question: if the product was discontinued would you be severely disappointed?

According to Sean’s experience with various startups, if 40% or more of the users say they would be severely disappointed if the product went away, the product has a fit and thus a market. So vital is the number that even Sean himself thinks twice about working with a company with a number less than 40% because the fate of the startup is still in question. Marketing a product is hard enough but marketing a product that no one wants is a losing situation anyone with smarts will thankfully leave to others.

I first realized I was a Visual Studio weenie when I showed up to a Mono conference in Spain with a Windows laptop in a sea of Linux ones. I was so outnumbered I entertained the idea of not booting the Windows laptop at all because having the familiar Windows boot tone echoing off the conference walls would undoubtedly gather unwanted attention. Eventually, I calmed down and even managed to boot my Windows machine without too many stares.

One of the talks at the Mono conference—Mono is an open source implementation of .NET for Linux based machines—was about the progress on the Mono .NET debugger. Mono had released without a debugger initially and the debugger quickly become the #1 requested feature, no doubt mostly from .NET Windows users looking to port their .NET apps to Linux. The debugger being worked on, the presenter informed us, would not just be a command line debugger like GDB but would be fully integrated into MonoDevelop—the Linux equivalent of a Visual Studio—so that breakpoints could be set next to the source code. He even had a cute demo that worked. This is nice, I thought. After all, who needs a complicated command line debugger that takes years to learn. I needed to get things done.

During the many breaks we had between presentations, I was able to sit next to a few of the Mono developers and watch them code. They were very efficient with GDB and VI—the command line debugger and editor of choice for any self respecting Linux hacker. Still I figured, I was just as productive in Visual Studio.

And for the most part I am, except for nasty programming realities like memory leaks. For those, Visual Studio was at a loss and I was too until I discovered Windbg, the command line debugger for Windows, and its extension SOS. After learning a handful of commands, I was impressed in how quickly I was able to track down the source of leaks that had eluded fellow Visual Studio zealots. Maybe those Mono guys were on to something. After all, nothing says I am a debugger extraordinaire quicker than opening up Windbg and pounding out a few commands in front of your fellow programmers.

Below is a few commands and a tutorial on tracking down a leak in an example program.

!dumpheap -stat            Displays every managed type on the heap

!dumpheap -type typename   Displays every managed type that matches the specified type

!dumpheap -mt methodtable  Displays every instance of the specified method table

!gcroot address            Shows reference chain

!dumpobj address           Shows fields and addresses of references

!dumparray address         Shows objects in an array

!dumpmd                    Dumps the method description

!u                         Show assembly code

g                          Continue, let the program run

Ctrl+Break                 Break into the Windbg debugger

!help [command]            Show help

The .NET Windows Forms example program I wrote draws circles on a form. The circles are placed in random locations and move from top to bottom until they vanish. There are 500 circles on the form at any time. The program works well except it leaks memory like the Titanic. It went from 10MB to 20MB in 30 seconds and climbs into the hundreds of megs if left to run.

The code is below. See if you can spot the issue before we run Windbg.

public partial class Form1 : Form
{
    private const int NumberOfCircles = 500;
    private Timer _timer;

    public event EventHandler<DrawCircleEventArgs> Draw;

    public Form1()
    {
        InitializeComponent();

        _timer = new Timer();
        _timer.Interval = 100;
        _timer.Start();
        _timer.Tick += new EventHandler(OnTick);
        InitCircles();
    }

    private void InitCircles()
    {
        for (int x = 0; x < NumberOfCircles; x++)
            AddCircle();
    }

    private void AddCircle()
    {
        Circle circle = new Circle(new Rectangle(0, 0, Width, Height));
        circle.Done += new EventHandler(OnDone);
        Draw += new EventHandler<DrawCircleEventArgs>(circle.Draw);
    }

    private void OnTick(object sender, EventArgs e)
    {
        Invalidate();
    }

    protected override void OnPaint(PaintEventArgs e)
    {
        if (Draw != null)
            Draw(this, new DrawCircleEventArgs(e.Graphics));
    }

    private void OnDone(object sender, EventArgs e)
    {
        AddCircle();
    }
}

public class Circle
{
    private static Random Rand;

    private Rectangle _parentBounds;
    private Point _position;
    private Size _circleSize;
    private byte[] _bytes;
    private bool _done;
    public event EventHandler Done;

    static Circle()
    {
        Rand = new Random((int)DateTime.Now.Ticks);
    }

    public Circle(Rectangle parentBounds)
    {
        _bytes = new byte[10240]; //10K for no reason but to make object bigger
        _done = false;
        _parentBounds = parentBounds;
        int xPosition = Rand.Next(parentBounds.Left, parentBounds.Right);
        int yPosition = Rand.Next(parentBounds.Top, parentBounds.Bottom);
        _position = new Point(xPosition, yPosition);
        _circleSize = new Size(10, 10);
    }

    private void OnDone()
    {
        _done = true;
        if (Done != null)
            Done(this, EventArgs.Empty);
    }

    public void Draw(object sender, DrawCircleEventArgs args)
    {
        _position = new Point(_position.X, _position.Y + 1);
        Rectangle rect = new Rectangle(_position, _circleSize);

        if (!_parentBounds.Contains(rect) && !_done)
            OnDone();
        else
            args.Graphics.DrawEllipse(Pens.Green, rect);
    }
}

To find the leak, I will open Windbg and attach to the program using File>>Attach to Process. If you don’t have Windbg, download the debugging tools for windows here. Next, load SOS like:

.loadby sos mscorwks

With SOS loaded, I can run commands from above like:

!dumpheap -stat

658551a8      163         3912 System.Windows.Forms.InvalidateEventArgs
641b8c4c       95         5320 System.Configuration.FactoryRecord
6585394c       78         5616 System.Windows.Forms.Internal.DeviceContext
6a72a930      190         6840 System.Collections.Hashtable+HashtableEnumerator
6a73303c       28         7608 System.Collections.Hashtable+bucket[]
6a7308ec      812        61060 System.String
6a7040bc      153        63636 System.Object[]
6a729b58     4804       153728 System.EventHandler
00126ba8     4847       155104 System.EventHandler`1[[Leaky.DrawCircleEventArgs, Leaky]]
00126ac4     4725       245700 Leaky.Circle
0032c6b0     1754       279984      Free
6a73335c     4729     48451204 System.Byte[]

This shows all types on the heap, number of instances, and the sizes with the largest on the bottom. I can see the byte arrays are taking 48 Megs and Leaky.Circle is 245k. The large byte arrays are a field of the Circle class designed to show the leak easier. What is more interesting is there are 4725 instances of the circle class. I was expecting only around 500 because as the circles go outside the form they should be garbage collected. So what is causing these Circle instances to stay in memory? I dump the method table listed for the Circle like:

!dumpheap -mt 00126ac4

071deb54 00126ac4       52    
071e13f4 00126ac4       52    
071e40fc 00126ac4       52    
071e699c 00126ac4       52    
071e923c 00126ac4       52    
071ebadc 00126ac4       52    
071ee37c 00126ac4       52

This shows all the instances of that method table on the heap. I can now use !gcroot on one of the addresses like:

!gcroot 071e13f4

ebx:Root:01c842b4(System.Windows.Forms.Application+ThreadContext)->
01c83878(Leaky.Form1)->
071f0bfc(System.EventHandler`1[[Leaky.DrawCircleEventArgs, Leaky]])->
064c5b38(System.Object[])->
071e3c54(System.EventHandler`1[[Leaky.DrawCircleEventArgs, Leaky]])->
071e13f4(Leaky.Circle)

This Circle instance is being held by an event handler which is being tracked back to the form itself. Let’s examine why the event handler above is holding onto the Circle instance by using the dump object command.

!dumpobj 071e3c54

Name: System.EventHandler`1[[Leaky.DrawCircleEventArgs, Leaky]]
MethodTable: 00126ba8
EEClass: 6a4c3518
Size: 32(0x20) bytes
 (C:\Windows\assembly\GAC_32\mscorlib\2.0.0.0__b77a5c561934e089\mscorlib.dll)
Fields:
      MT    Field   Offset                 Type VT     Attr    Value Name
6a730508  40000ff        4        System.Object  0 instance 071e13f4 _target
6a72fd60  4000100        8 ...ection.MethodBase  0 instance 00000000 _methodBase
6a7331b4  4000101        c        System.IntPtr  1 instance   12c118 _methodPtr
6a7331b4  4000102       10        System.IntPtr  1 instance        0 _methodPtrAux
6a730508  400010c       14        System.Object  0 instance 00000000 _invocationList
6a7331b4  400010d       18        System.IntPtr  1 instance        0 _invocationCount

Finding the method this event handler is pointing to is a bit tricky. First, we dump the raw memory for the _methodPtr field

dd 12c118

0012c118  1c4653e9 00005f00 00126aac 00000000
0012c128  00000000 00000000 00000000 00000000
0012c138  00000000 00000000 00000000 00000000
0012c148  00000000 00000000 00000000 00000000
0012c158  00000000 00000000 00000000 00000000
0012c168  00000000 00000000 00000000 00000000
0012c178  00000000 00000000 00000000 00000000
0012c188  00000000 00000000 00000000 00000000

Then we dump the method description who’s address is located on the first line fourth column

!dumpmd 00126aac

Method Name: Leaky.Circle.Draw(System.Object, Leaky.DrawCircleEventArgs)
Class: 008f0b28
MethodTable: 00126ac4
mdToken: 06000014
Module: 00122c5c
IsJitted: yes
CodeAddr: 002f0770

The event handler is pointing to the Draw method on the Circle and keeping it in memory. I must not be unhooking the event. Looking at the source code, it becomes apparent that I am not unhooking that method anywhere in the code. The proper place to do it would be in the OnDone method of the Form1 class.

private void OnDone(object sender, EventArgs e)
{
    Circle circle = sender as Circle;
    Draw -= new EventHandler<DrawCircleEventArgs>(circle.Draw);
    AddCircle();
}

Now if I run the same test as before and dump the heap I get:

!dumpheap -stat

6a72b16c      245         5880 System.Collections.Stack
6a73303c       28         7608 System.Collections.Hashtable+bucket[]
6585394c      245        17640 System.Windows.Forms.Internal.DeviceContext
6a729b58     1193        38176 System.EventHandler
00146ba8     1230        39360 System.EventHandler`1[[Leaky.DrawCircleEventArgs, Leaky]]
6a7308ec      755        59232 System.String
00146ac4     1176        61152 Leaky.Circle
6a7040bc      315        95188 System.Object[]
0035c6b0      918      1738620      Free
6a73335c     1180     12066856 System.Byte[]

Notice there is a smaller number of circles and byte arrays and when I run for a long time the memory doesn’t go over 30 megs. There are 1176 circles but most of those are waiting to be collected. This may seem like a contrived example but from my experience it is all too real. Most memory leaks I have found in code bases are because of event handlers not being unhooked.

One of the quickest ways to bring a delightful code review to a halt is to slip in a big switch statement somewhere in the code. Upon encountering it, some will feel uncomfortable and not know why while others will give you a long soul piercing stare.

Switch statements get a bad rap for a number of reasons. The chief among them being when a value is added to an enum you have to track down every switch statement that is using that enum and add your new code. This may seem simple but most of the time there is more than one switch statement to contend with. Often the changes needed are in unfamiliar code which leads to a long and error prone process.

There are alternatives thanks to object oriented programming and polymorphism. To understand such an approach, the C# code below shows a scenario with a switch statement. The class Rect takes an enum FillStyle as a property. In the Draw method it creates different brushes—depending on the FillStyle property—using a switch statement. Code like this is very common.

public enum FillStyle
{
    Solid,
    Gradient
}

public class Rect
{
    private Point _location;
    private Size _size;

    public Rect(Point location, Size size)
    {
        _location = location;
        _size = size;
        FillStyle = FillStyle.Solid;
        FillColor = Color.Green;
    }

    public FillStyle FillStyle { get; set; }
    public Color FillColor { get; set; }

    public void Draw(Graphics graphics)
    {
        Brush brush = null;
        Rectangle bounds = new Rectangle(_location, _size);
        switch (FillStyle)
        {
            case FillStyle.Solid:
                brush = new SolidBrush(FillColor);
                break;
            case FillStyle.Gradient:
                brush = new LinearGradientBrush(bounds, FillColor, Color.Red, LinearGradientMode.Horizontal);
                break;
            default:
                Debug.Fail("Not a valid FillStyle");
                break;
        }

        using (brush)
        {
            graphics.FillRectangle(brush, bounds);
        }
    }
}

An alternate and better approach is to use an abstract class with public static concrete instances. The primary motivation for this is the abstract class can masquerade as an easy to use enum. This way the code the API user has to write is exactly the same as before and most importantly just as simple. In fact, most users, will never notice they are working with an abstract class with singleton instances; to them it just looks like an enum.

In our case, an abstract FillStyle class is created with two public singleton instances that have the same names as our original enum values above. The FillStyle class has one abstract method called CreateBrush. Reworking the above example looks like the code below.

public abstract class FillStyle
{
    public static readonly FillStyle Solid = new SolidImpl();
    public static readonly FillStyle Gradient = new GradientImpl();

    protected FillStyle()
    {
    }

    public abstract Brush CreateBrush(Color color, Rectangle bounds);

    private class SolidImpl : FillStyle
    {
        public override Brush CreateBrush(Color color, Rectangle bounds)
        {
            return new SolidBrush(color);
        }
    }

    private class GradientImpl : FillStyle
    {
        public override Brush CreateBrush(Color color, Rectangle bounds)
        {
            return new LinearGradientBrush(bounds, color, Color.Red, LinearGradientMode.Horizontal);
        }
    }
}

public class Rect
{
    private Point _location;
    private Size _size;

    public Rect(Point location, Size size)
    {
        _location = location;
        _size = size;
        FillStyle = FillStyle.Solid;
        FillColor = Color.Green;
    }

    public FillStyle FillStyle { get; set; }
    public Color FillColor { get; set; }

    public void Draw(Graphics graphics)
    {
        Rectangle bounds = new Rectangle(_location, _size);

        using (Brush brush = FillStyle.CreateBrush(FillColor, bounds))
        {
            graphics.FillRectangle(brush, bounds);
        }
    }
}

The first thing you will notice is the Draw method on the Rect class is simple and adding new values to the FillStyle class has become substantially easier than the former traditional enum example because the code can be added in one central place. Gone is the frustrating experience of updating every switch statement in the entire code base. The introduced abstraction adds to the extensibility of the design. If a user wants to add a custom FillStyle—not originally anticipated by the API authors—they can by inheriting from FillStyle and overriding CreateBrush in the same way.