This is the first in a series of posts coming this week on the Top Web Bug Trackers we saw at Ghostery last month.

What surprised me:

1. Google’s utter domination of the Top 10 list (Google Analytics, Google Adsense, Doubleclick and Google Custom Search).
2. Google Analytics coverage is massive and growing quickly. I instinctively knew this but its breadth and velocity even surprised me.
3. Two sleepers: Statcounter and AddThis. Their penetration rates eclipsing many well-known services, including some very large ad networks.
4. Free: 8 of the trackers are “free” or have “free” versions. With only Doubleclick and Omniture being paid “enterprise” services.
5. Open-source inside: OpenAds and Wordpress Stats are based on, or are, open source projects.

Read the rest at Ghostery – Top 10 Web Bug Trackers on the Web

The common theme that runs through every startup that I’ve started, or been a part of, is the need for “big data” analysis. “Big data” analysis is the area where off-the-shelf software tools breakdown, where statisticians, analysts and developers meet and where normal number-crunching turns into “supercrunching“.

At my last two startups, Compete and Lookery, this “big data” analysis has transcended its usual internal audience and become a fundamental part of, if not the entire product.

In the time before Lookery (B.L.) we needed to create our “big data” infrastructure from scratch. This usually took the form of large clusters of computers running proprietary, created from scratch, software. The most formal of these being the software we created at Compete which included the Compete Filesystem (CFS) and CompeteSQL (CSQL).

Today we have open-source software like Hadoop to provide the framework for our data analysis software at Lookery. The Hadoop project has grown fast with companies like Yahoo, Facebook, Last.FM, and The New York Times using it. There are even venture-backed startups focused solely on building services and products on top of the framework.

This weekend Elias Torres, our VP of Engineering at Lookery, released a project he calls Hadoop Timelines. Hadoop Timelines is a great example of what I’m calling “Hadoop Performance Optimization” (HPO).

While the barriers to use something like Hadoop have fundamentally dropped there are only a handful of experts that can make your Hadoop cluster perform well. What’s needed is a new suite of services and tools that can analyze your cluster and automatically optimize your performance. Hadoop Timelines, while rudimentary, is the beginning of an exciting new business niche, Hadoop Performance Optimization (HPO).

If you’re a Hadoop user please comment on how you’re optimizing your performance today.

This graph shows what the greybeard VCs and angels have known for a while. If your company has VC investors, they will reduce the probabilities of an exit that would produce a 1-5x return for the angels. That exit might have produced a 100x return for the entrepreneurs because they paid much less than the angels for their shares.

Having VC investors does increase the probabilities of exits above a 5x return.

But there is no free lunch. This data shows that after a VC invests your chances of failing completely also increase significantly.

via Exits with VC and Angel Investors – The Wiltbank Data.

David says: The news is that this isn’t news. Venture investors are almost always transparent on what it means to take their money. They have no choice anymore as all this information is freely available to anyone who wants it although I believe they are transparent because it makes good business sense first and foremost.

The reason I’m reblogging this post, from the excellent AngelBlog site, is that “we” startup people are either blind eye optimists, blinded by our ideas, ignorant, or all three. “We” commit the same (fatal?) mistakes over and over despite, usually, knowing better.

My gift to you (and me) today is to hit you over the head with this one more time. Hopefully it’ll only take a few more loving strikes with this 2×4 before “we” all finally get it.

(Note the emphasis on the above quote is mine)

“In business about 90% of your ideas suck and about 10% of them actually work. And we never know which is which. So you have to survive your bad ideas and when you borrow money to do them you magnify the size of the mistake.

Version 1.0 seldom even makes it to market and version 2.0 doesn’t make you money it loses money. 3.0 starts to work. About 7.0 is the sweet spot by the time you polish this rock a little bit it will shine but the first time your throw it out there it’s a piece of coal.

…Business is a process, you cannot analyze it. – Dave Ramsey

At Compete and now Lookery, Product Development was the key difference between good products and mediocre ones. The greater the distance between clients and developers the worse the product became. At Lookery, Elias, our head of engineering, is putting the rest of us to shame, juggling sales calls, support, and coding simultaneously.

I’ve never taken Product Development as far as Jeff has (elimnating “engineering”) but the next time I start a company I plan to do so. Jeff’s definition of Product Development below (emphasis mine):

In my companies, we go so far as to merge two functions that are often separated in other organizations: engineering and product marketing. It is my opinion that these should be one in the same. The folks that are building the product are the folks that need to be out in front of customers, finding out what that product should be. They are also the same folks that should be telling those customers how they can use the products they’ve built better or in different ways. We call the combined entity “product development” and it is their job to build products people will buy. They get full responsibility, so there is no blame game between product marketing (”the engineers built a product that sucks!”) and engineering (”marketing got the requirements all wrong!”). This one department is responsible for the whole enchilada, no questions asked.

A lot of technology companies delegate the responsibility for coming up with product requirements to “marketing” who then talks to customers (maybe) and analysts (probably) and copies what the competition does (unfortunately and almost certainly), and then hands a list of requirements to engineering, who inevitably further misinterprets the requirements on their way to creating a product that at best is marginally passable and at worst is so far of the mark that no one will buy it. What an unnecessary chain of misinformation and complexity. The people who design the product should be out there talking to people who want to buy it, and should build what they will buy.

via Market research for startups – McStartup Blog – McStartup – tasty advice for startup companies.

My partner Scott Rafer dropping some startup science during an interview at e27 in Singapore, worth reading the entire post.

Lookery looks really great, maintaining steadfastly on permission marketing and not taking users for granted, very similar to Google’s Don’t be Evil mantra. I like that! Would you like to share how’s the culture like at Lookery?

Company cultures reflect the founders. David Cancel and I have each been working on startups for a long time and have learned some painful lessons that we try not to repeat. Generally, our mistakes have come from excess complexity and unrealistic expectations. Oren Michels of Mashery is the same way. We all think alike. Oren and I met David because he was Mashery’s first customer.

1. We’re big fans of simplicity and volume. Do one small thing — and do it a huge number of times. This leads directly to… If it can’t be done with cloud computing, don’t do it.
2. We don’t have secrets or tell lies, because both too expensive to keep track of. We can’t justify the time, effort, or capital.
3. Small companies should be virtual. We’re six people in three cities who have only all been in a room together once. When the economy improves, we’ll have twice-yearly get togethers. Otherwise, each of us tends to see one or two other employees one day a week or less.
4. Ideas are free; execution provides the only value.
5. Automate only after you know the automation is justified. Manual processes are fine until then.
6. Only raise money if you know EXACTLY how it will improve the business to the benefit of the current shareholders.
7. Etc.

Also, for smart, productive people, family life is critically important and stress is a terrible motivator. Half the company are work-at-home parents. We make schedule accommodations for childcare all the time and are very used to screaming three year olds “participating” in our staff calls.

via Sniffing out Great Ideas, Company Culture and Demographics! – Interview with Lookery CEO Scott Rafer » e27 – Discovering Web Innovation in Asia.

Here are some of the replies I received:

If you are interested in starting a company I recommend following nealrichter, changds, daynagrayson, robgo, kirilnyc, and payne92 on Twitter, all six are successful startup entrepreneurs.

Acquiring MySQL was Sun’s hedge bet against Oracle. Oracle’s investment in Linux was their hedge against Sun. In the end Oracle and Sun’s core customers, their most lucrative, didn’t really care and kept on buying Oracle software and Sun hardware together.

Just because you care about a feature or a product doesn’t mean your best customers will. Listen to how they’re using your product today, assume nothing, sleep on it and repeat. If you’re lucky you’ll learn something in the process.

by Miller Info Commons

Are you thinking of raising venture capital to fund your startup?

I did in order to start Compete and at my previous companies, so I know the process  intimately. With Lookery I’m doing things differently this time.

Raising any capital is going to be more challenging today than a year ago but still quite doable.  With Compete we raised our first round 6 months after the Internet Bubble had burst as did many others, it can be done.

Although I chose to do things differently this time I have plenty of VC friends that I’d recommend to anyone interested in raising money.  Contact me for any recommendations. Before you do that though I recommend you read “Why VCs Will Block Good Exits” carefully. Very simple math just make sure you do it before not after raising money.

Ghostery originally started out as GreaseMonkey script I created last year called Invisible Web. Turning it into a Firefox extension made it a bit more user-friendly and made updating the definitions database easier.

It’s a fun side project that I’ve been playing with for a little over a month now. In that time its been installed over 4000 times, mostly thanks to Hacker News & Reddit, been mocked by Fred Wilson, is currently featured on the homepage of CrazyEgg, and was written about by Greg Yardley.

I’ve also gotten tons of great feedback and contributions from Scott Switzer, Greg Yardley, Neal Richter, Mitch Fournier, Scott Rafer, Jay Meattle, Rex Dixon and Hiten Shah. Thanks to everyone for taking time to check it out.

Get Ghostery now. I’d love your feedback.

Have a great and prosperous 2009,
David