A Failed Experiment – And why It's Important to Measure Everything
For some time now I’ve been considering having a weekly post where all I’d do is list interesting blog posts, articles, news, etc. that I found online over the week. In late April I decided to go for it, and so was born the Lazy Friday Reading Assignments. These posts consisted each of a list of links with information around each link explaining why that link was interesting.
In terms of results, I mostly expected that these posts would have much higher clickthroughs than normal (because it’s a list of links to check out). I also thought that the overall traffic, and the subscriptions (Google Feedburner count), to the blog would continue it’s normal growth. And if I was lucky, other sites would learn about this blog and hopefully send additional new traffic.
The results were not what I expected! And this is why it’s important to measure everything.
In the graph above, the blue line represents “Reach”, or as Google defines it, the number of people who have taken action on your content. I had 4 Lazy Friday Reading Assignments on the graph, and all 4 resulted in higher clickthrough days. There are actually about 8 posts in the timeframe of the graph, with the last post being on May 21st (I’ve been too busy over the last while to post). In any case, the end of the graph is the baseline with no posts (about 2+ weeks since my last post), the lower levels of the blue line on the left side are the baseline for normal posts, and the higher numbers are the Lazy Friday Reading Assignment post days. Exactly as I would expect them to be!
The traffic, as measured by unique visitors, did increase over this time, but not much more than my normal growth levels (it’s not displayed on the above graph, I measure unique visitors through other sources). This is more or less what I expected.
But, and this is a big but, the subscription count as measured by Google Feedburner (the green line), was not at all what I expected! Looking again at the graph above, you’ll see that while I was publishing the Lazy Friday Reading Assignments, the green line has dips and overall didn’t really increase. Although the dips don’t exactly correlate to the Lazy Friday Reading Assignment days, I can assure you it’s the first time I’ve ever seen this behavior (there’s also some delays with when Google Feedburner sends out the newsletter by email for those that subscribed by email). Normally when I write a post I’ll see an increase in subscription count on those days (the reverse). I would’ve included an example, but I wasn’t able to find a way to generate a graph from Feedburner for a specific date other than the last 30 days (the full length is too massive).
Although those dips may not look too big, they do represent a decrease of several hundred subscribers. This is significant enough! And more importantly, it’s consistent. During the experiment the subscription rate had absolutely no growth. There’s been more subscription growth with me doing nothing for 2 weeks after the experiment than during the whole month of the experiment!
In other words the experiment was a failure. Therefore the Lazy Friday Reading Assignment is no longer. Although I thought it was a good idea, this didn’t turn out to be the case. Which is why it’s a good idea to measure what you’re doing. Had I not had these metrics, I may have continued for a long time before realizing my error.
Which is why it’s important to measure, measure, and measure again.
PS: Looking at the graph also reminds me I should be posting more consistently.
Permalink to this article Discussions (2)
Lazy Friday Reading Assignments
- There’s a lot of discussion going on about how patent’s don’t really make sense in the IT world like they do in other industries (such as medicine). The post Microsoft Patent Trolls Salesforce from the guys at 37Signals really does show the absurdity of the world of patents in the IT world.
- In the post No Plan Survives First Contact With Customers, Steve Blank explains why a business plan isn’t the end all be all. He explains why it’s important to initiate the contact with your customers early on, even before you start building your product if you can. It’s even more important than creating your business plan, which I agree.
- If you think it’s not possible to release quickly and release often in software development, then you really need to check out WordPress.com’s continuous deployment strategy. They average 16 product releases a day!
- In the last few years Sun Microsystems has been struggling, losing market share and pretty much collapsing financially. Oracle, ie. Larry Ellison, recently acquired Sun and is in the process of building it back up. Interestingly, Larry was pretty blunt, and he didn’t really hold back, in this interview about his views on what Sun did wrong.
- I’ve always wondered about Life Lock’s ads, where Todd Davis their CEO, publishes his Social Security Number. Well it turns out his identity has been compromised at least 13 times! Not exactly the showcase example for his product. He’s now in trouble with false advertising, amongst other issues.
- I’ve been a fan of Balsamiq for some time for quick prototyping, however Google just released a new Rapid Wireframe Sketching feature in Google Docs that looks pretty powerful. And as far as I can tell it appears to be free. Should be interesting to see how that market develops.
- Ever wonder how sites like Reddit.com can grow and support large user bases? The post 7 Lessons Learned While Building Reddit to 270 Million Page Views a Month gives a good explanation.
- The world of artificial life has had a major breakthrough recently. Without being too technical, they’ve been able to create cells according to their specifications, that is program cells to do what they want. If the report is accurate, which it appears to be based on the source, then this will be one of the bigger scientific discoveries of all time!
- The world of VC is very froth with peril, and often the founders are taken for granted. Steve Blank tells an interesting story of how he was treated poorly because, as he puts it, You’re Just the Founder. As you can tell I’m not a big fan of VC, and this is just another reason why.
- Alwin Hoogerdijk published his results of “Upgrade Week” where his company gave a significant upgrade discount on their software product for one week to all their customers that hadn’t yet upgraded. I have to admit, I was quite surprised by the success he got.
- Anthony Feint has written How to Suck at Branding, and this is exactly how you can suck. Very simple and very straightforward. My favorite his is last item: Outsource it. I especially liked the last lines, “Did I mention you’re busy. Let some PR company handle your branding. They’ll get a nice generic logo designed for you, and they’ll take some nice profile shots of your uber cool founder. Then they’ll launch a “social viral interactive marketing experience” (youtube video), to help spread the word. You don’t have to do anything. In fact, why not retire to an island and be a recluse.”
- There are many ways to calculate the most efficient routes, but I never would have guessed using slime mold.
- Sometimes when you sign up for a new credit card or a new bank account you’re offered a signup bonus, which can be in the form of a cash, an item of value, and so on. The problem is that you need to be careful because signup bonus from banks are usually taxable whereas those from credit cards aren’t. Not only that, but the taxes are calculated on the retail value and not the actual market price, which means that in some cases the bonus signup item may cost you as much, if not more, than if you bought it yourself!
- And the last reading assignment for this week comes from The Oatmeal, 8 Websites You Need to Stop Building. My personal favorite is #6, The Next Facebook: “All I need is someone who can design, code, host, launch, manage, and market it”. “Okay what’s your budget?” “Well I don’t actually have any money right now. But I’ll pay you in equity, which according to my projections will be worth eight hundred million dollars within 6 months.”
Permalink to this article Discussions (1)
Lazy Friday Reading Assignments
- The iPad is not a larger iPhone, it’s much more than that. It’s a computer for the non-technically inclined. Chuck Hollis brought one home, and wrote What iPads Did to My Family. This is exactly what I anticipate will happen all over the world. It’s not a computer as much for geeks and techies like myself, it’s for the rest of the world.
- The opposite of that, for the geeks and technically inclined, Will Urbina created his own storage appliance capable of holding 16TB of data! He calls his device the Black Dwarf, and in this post he explains how to build it.
- Like Chess, computer programming is easy to learn but hard to master. This is especially true when it comes to building systems that will be used and maintained over time. Anyone can slap together something that will work, but very few people can build systems that can thrive over many versions. Because of this, Donnie Garvich wrote 10 Ways to Suck at Programming to help you learn how not to suck at programming.
- Recently Greece has been having some financial problems, to put it mildly. The following piece Greek Debt Crisis Offers Preview of What Awaits US is really eye opening. Although the US isn’t as yet as financially troubled as Greece, it’s still not a giant leap.
- If you’re a Facebook user, apparently its possible to have apps being installed on your account without your knowledge. The following post explains it: New Facebook Social Features Secretly Add Apps to Your Profile. And if that wasn’t enough with all the heat Facebook has been receiving this week, the following article by Ryan Singel of Wired magazine just adds to the fire: Facebook’s Gone Rogue; It’s Time for an Open Alternative.
- If you’re having problems coming up with a company or product name, look no further than Andy Brice’s recent post called How to Find a Great Software Product Name.
- On the StackOverFlow site the question New programming jargon you coined was asked which lead to some very interesting replies. This in turn eventually lead to the post New Programming Jargon by Joey Devilla which summarizes the answers and adds some additional humour.
- There was a vulnerability exposed on Twitter this week which let anyone add anyone as their follower. Of course within an hour they had resolved it and were in the process of rolling back all abuses.
- Sales conversions are a tricky thing. It’s not always obvious. In the post Increase Conversion Rate by Making Your Site Ugly, Zack shows that the best looking site is, as the title suggests, not always the best selling site.
- As if Apple, er Steve Jobs, didn’t have enough excitement with a prototype next generation iPhone turning up on Gizmodo last week, this week the second missing iPhone turned up in Vietnam.
- If you really want to appreciate the scale of the oil spill mess going on right now, Paul Rademacher created a good mashup website where you can overlay the oil spill over any location in the world. For example, you can see how far it would be spread if it was centered on your home city.
- Ever wonder what your best profile picture is? Instead of trying to guess, you may want to check out MyBestFace. For more details on the how it works and the story behind it, check out the post by Christian called What is Your Best Profile Picture.
- If you think the housing market is starting to get better, you may want to check out this post from Dr. Housing Bubble. His argument is that we’re just entering the second wave of problems, and based on his data, I tend to agree. It’s gonna get worse before it gets better.
Permalink to this article Discussions (0)
Lazy Friday Reading Assignments
- To quote Brad Feld: “The great companies that I’ve been an investor in share a common trait – the founder/CEO is obsessed with the product. Not interested, not aware of, not familiar with, but obsessed. Every discussion trends back toward the product. All of the conversations about customer are really about how the customer uses the product and the value the product brings the customer. The majority of the early teams are focused entirely on the product, including the non-engineering people. Product, product, product.”.
- I’ve always wondered if creating a separate virtual machine for each development environment would be worthwhile, especially for consultants working in a lot of different environments. Fortunately for me Alex tried it out and wrote his findings at: Using a Virtual Machine For Software Development
- I had never heard of “fauxting” (fake texting) to avoid a personal interaction until I read When You’re Only Text Friends by Charlotte Steinway
- I remember studying speech recognition in university a long time ago. It was ok, but it really wasn’t ready anywhere near primetime ready yet. Since then I’ve seen some improvements, but I haven’t really seen a commercial offering. This article helps to explain why speech recognition hasn’t really been moving forward much.
- John Chow has had an interesting and, well let’s say different, had interesting way of building up his blog community. He tried several traffic building techniques that were definitely on the gray side, which eventually got him banned from Digg, de-listed from Google for a while, and so on. However through all that he’s managed to come out ahead. This week, he’s trying a new technique where you can buy real Twitter followers (and not bots). He even explains how he will recoup the cost within 2-3 months.
- If you’re a software developer, this is one of those crazy stories that you just can make up.
- I just learned this week about the Memristor. If this YouTube video presentation is right, this could really revolutionize computers. I would compare this jump in computing power to the discovery of transistors. Here’s to hoping it’s real and will come to fruition sooner than later. 1 Petabyte in the size of a sugar cube! Whole databases in fast ram. The way we program would drastically change.
- I’m a big fan of history, not dates and places, but understanding how things were, which in turn explains how and why we are where we are today. Jeremy Keith wrote A Brief History of Markup which does a good job explaining the history of markup (more or less html).
- It’s been said that a picture is worth a thousand words. Well this picture about The [Real] Current State of Web Design says it all.
- With everything going on with Apple and Flash recently, it’s almost impossible not to have a related story. Hank Williams writes a good article on How Apple is Breaking the Law with the App Store.
- And if you don’t understand why there’s so much discussion about mobile computing these days (iPhone, BlackBerry, etc.), then you should check out this information graph about how teens are using their cell phones. Mobile computing is huge.
Permalink to this article Discussions (0)
Lazy Friday Reading Assignments
Starting this week, every Friday I’m going to post a list of some of the best articles I discovered over the week. That and some of the more exciting news in all things related to LandlordMax (technology, real estate, entrepreneur, and so on). So without any further ado, let’s start the first Lazy Friday Reading Assignment!
- Apple, er I mean Steve Jobs, er I mean both since they are basically one and the same, just released this statement about why will not support Flash on the iPhone. To that, Shantanu Narayen (the CEO of Adobe) responded with this interview with the Wall Street Journal. Looks like things are only going to heat up.
- If you have, or are thinking of building, an Saas (Software as a Service), this is a great post on pricing it.
- It’s very easy to fall prey to the Sunk Cost Fallacy, so this article is a good reminder of why it’s always important to look at your ROI starting from now, and not to look always look at how much you’ve already invested.
- If you don’t believe the power of a good referral, then you need to check out this post about how a video post from a 16 year old blogger generated over 450,000 views and 90,000 comments from a single video.
- Software estimate is very difficult (almost impossible I would say to do accurately). Here’s a list of 12 reasons why software estimation is so hard.
- It’s not just your imagination, buying a home is getting more and more expensive. It’s not just inflation, but the cost of a home relative to our salaries has climbed.
- How many of us are on Facebook? Probably most of us. How private is that information? What happens if Facebook starts to go the way of MySpace and starts running out of money? Will they sell your data to try and save the company? If you don’t think it can happen, just look at MySpace as it appears they may be selling some of their data.
- The database/software world may be in for a world of hurt if Massachusetts has it’s way. They want every company that stores any information in a database about a Massachusetts resident to be encrypted (which is fine), but more importantly they want you to file a Written Information Security Plan (WISP) and file it with the state. Failure to do so could results in large fines. Knowing how most government entities are with IT (lack of understanding), not to mention behind the times, I can’t see how this will help in any way other than kill innovation by causing normal companies to jump through hoops to support any Massachusetts residents. If it was me, I don’t know that I’d be interested in offering any support for Massachusetts. Who wants to file all that paperwork? Never mind very likely having to use older systems because that’s all they have had the time to certify (many government entities around the world are still on IE6 which is why Microsoft is still supporting it). Sounds like a disaster waiting to happen.
- Not all employees create an equal amount of revenue for their companies. The guys at 37Signals created a quick graph to show an example just how much more efficient some companies are at generating revenues from their employees.
- And as we all know, an iPhone 4 prototype was “lost” in the wild and found it’s way on Gizmodo. Whatever the truth is about how Gizmodo got their hands on the iPhone, John Chow wrote a post called “How to Get 20 Millions Views for Only $5000” about how it was worth it for them. Even if they end up having to pay a hefty fine.
And that’s this Lazy Friday’s first reading assignment. Hope you enjoy it. And if you have any suggestions for next week, please do feel free to email me or comment below.
Permalink to this article Discussions (0)
