Musings on books, technology, entrepreneurship, nonprofits and umm.. everything else …

Archive for July, 2012

What does a bandit have to do with my website?

There comes a time in life when we decide that we know what we know and what we don’t, well its to late to look it up anyway. That used to be the case with me and website optimization. Over the years I’ve heard words such as A/B testing and multi-armed bandit thrown around by more knowledgeable friends and always wondered what they meant. We had Hal Varian speak to us at Startuponomics last year and mention how the multi-armed bandit was standard operating procedure at Google in terms of how they optimized their sites.

In the recent past, a minor storm has been brewing in this teacup among practitioners of both these mechanisms in terms of what works well and under what circumstances. As the geeks in the reader list might be aware, The Obama campaign has been A/B testing their website and email campaigns too, to get better conversion rates. For a quick summary of references and arguments on both sides, you can read this blog post. If you like to watch and listen to talks rather than read websites, you can watch this talk around recommender systems and optimization methods such as MAB to make them better. Coincidentally enough, one of my wiser and smarter colleagues was telling me about the similarities of the epsilon-greedy MAB algorithms to the adaptive filter coefficient optimization and other steepest gradient algorithms in digital signal processing.

In short, the argument seems to be that if you are a startup that is optimizing over a small set of visits, then the multi-armed bandit approach may not give you statistically significant results and you might in fact be optimizing away from your best solution. But if your changes can be done incrementally, and your visit volume is large enough, the MAB approach allows you to track towards the more optimal options quicker while only running experiments on a much smaller subset of your audience. I won’t pretend to know more than this on the topic, but for all my friends that are starting companies where conversion rates are important, this is an issue that you need to look into and understand as you grow. With tools such as the Google Website Optimizer and companies such as Optimizely, it has gotten much easier to do some of this as well.

The key is to build testability into the product, especially one that interacts with a large chunk of its customer base through a website. It can be said with a fair amount of certainty that what we might assume to be the optimal layout, choice of colors, positions of buttons etc might turn out to be not as optimal as other options. For startups that do not have the money to conduct extensive market research surveys, A/B testing or multivariate testing provide a quick way to improve conversions and CTRs through trial and error.  If you know more about this, I would love to buy you lunch and pick your brain sometime, so let me know!


The “golden age” of venture capital

Last Thursday I went to Wharton to attend the screening of Something Ventured a 2011 documentary that shines some light on the golden days of venture capital in Silicon Valley. Conceived by Paul Holland of Foundation Capital, this documentary interviews some of the legends of the VC world in the valley today that are slowly retiring from the industry such as Tom Perkins, Arthur Rock, Don Valentine and others. It follows an interview format where it weaves conversations with the investors and conversations with the entrepreneurs that they backed to present a great inside look into what made those deals happen.

And what one comes out with is more respect for the venture industry in general, and these protagonists in particular. Despite their string of successes they seem humble enough or realistic enough to admit what a big role luck played in many of their successes, and how their existence or success is predicated on having entrepreneurs who risk everything to start companies based on their vision for a new product or a new world. It profiles the founders of Intel, Atari, Tandem Computers, Genentech and a few other firms from the 70s and the 80s and looks at how the venture industry in the Silicon Valley started and grew.

Another insight gained from the movie was about the influence Georges Doriot had in terms of creating the venture capital industry as we know today. In addition to coming up with equity based returns for investors and encouraging stock options as the way to incentivize people, through his tenure at Harvard he was also able to influence several young MBAs about entrepreneurship and the potential of growth in the venture capital world. It is many of these students that started off getting money from the East and bringing it to the West to fund entrepreneurs. The Traitorous Eight from Shockley Semiconductor Labs being one of the famous early ones that started Fairchild Semiconductors.

For people outside of the valley or outside of the startup world, some of the finer details of differences between PE and VC might not be clear and the documentary does not go into details of what differentiates VC from other forms of investing. But it offers unique insights into how these successful VCs went about raising their early rounds and grew to be successful, while almost being wiped out during the course of their careers. I recommend this as a must watch for most MBA students as part of their entrepreneurship or venture capital curriculum.

Ashmeet Sidana from Foundation Capital was the main speaker at the event that introduced the film to a packed audience and also responded to Q&A at the end. This was organized by the Wharton Entrepreneurial Programs (WEP) out here in San Francisco and was well attended. I think this was a great idea in terms of the unique learning opportunity that it offered and I look forward to more such events out here. Looks like WEP is getting strong roots out West and will soon be a name to reckon with in terms of supporting successful entrepreneurs that come out of the Wharton program.

How do you hire the “best” talent?

Starting from the days of written tests followed by technical, role-specific interviews to the world beyond the “Microsoft interview” and the “Google interview”, hiring for talent has come a long way. Given the dominance of platforms such as LinkedIn, instead of the Descartes version of “I think therefore I am” we are sadly in a world of “I can be searched on 250 attributes, so I am”. And this is one of the big issues that recruiters face today.

Human behavior gravitates towards optimizing along the variables that we are measured on or compared against. If it is for undergraduate or graduate school admissions, then people now have the  mandatory set of instruments they have played since childhood, the international humanitarian visit and lessons learnt from it, and the non-profits that they either started or played a key role in for some period of time. A fantastic article I read on this several years ago is one by David Brooks from April 2001 titled “The Organization Kid”, where he outlines among other things conformism to the majority view while playing the system to meet all of its checklists to get into the best of the best schools and be well placed to be the leaders of tomorrow. In the hiring world, this means that our resumes become a collection of technical keywords for engineering roles, partially meaningless revenue or performance improvement numbers for business roles and other standard “templates” that recruiters scan for when they evaluate resumes. There are even studies that purport to know where recruiters look the most, such as the one from The Ladders which has several real issues in terms of its validity as highlighted in articles such as this blog post.

Given this, how does one go about detecting a signal in the middle of all the noise? Some studies such as the one by Feltovich and Harbaugh looked at homogeneous classes of high-achievers and showed how “counter-signaling” might play an important role in signaling value. Other fields of research have focused on behavioral interviews and how they say more about a person than just a skills verification. Some companies also ask you to take Myer-Briggs Type Indicator tests to look at match between your personality and the job requirements. Very often, in addition to these verification mechanisms, companies fall back on anecdotal data – recommendations from current employees or known contacts in a candidate’s company.

But the problem is that all said and done, this is a one-shot optimization problem. This hardly guarantees that the talent you pick will be optimal over a period of time, even assuming that the tools you have at hand for the one-shot optimization are doing their job. In other words, how do you make sure that the talent you hire becomes the best talent for the job that you hired them for, and remains so over time? Many rigorous processes such as the Google and Microsoft interviews try to incorporate some elements to detect how open a person is to new ideas and how they seek to answer problems, as indicators of success at their roles. In “The Innovator’s DNA”, Jeff Dyer, Hal Gregersen and Clayton Christensen talk about how asking questions is an important indicator of an innovator, a quality that most jobs in today’s ever-changing world need. Another important aspect to this, in my mind, is how cross-pollinated an employee can get within the organization. One great example of this is the Associate Product Manager role within Google, something that Marissa Mayer championed and led during her time there. Giving a person rotations through different business units, and responsibilities above what their resume was hired for is a great way to identify the outliers in terms of talent and leadership and reward them quickly, rather than have them languish in traditional corporate growth ladders. It also helps foster non-traditional thinking to solve problems because you have atypical folks attending meetings where the problems are discussed.

In summary, in today’s world, most successful companies have already tweaked and optimized the one-shot optimization problem of hiring a resume and doing some basic “culture” match detection. What will distinguish the winners in this hiring and retention race of tomorrow, both at startups as well as large firms, will be how they handle the talent they have in-house and help them grow into their jobs and progress in ways that they feel satisfied. Time to dust off those books on Needs Theory and theories on motivation and job design and see where the next competitive advantage in hiring and retaining talent comes from!

Overindulgence and its weighty consequences

A couple of months ago, New York Times ran an interesting article about an MIT trained mathematician and his attempts at quantifying what causes obesity and how we can solve this issue that has weighed us down, literally. After a few years of research Carson Chow, the academic, came up with a conclusion that seems as obvious as it is profound – we’re a nation where 2 in 3 are overweight, and 1 in 3 are obese because we eat too much. 

Lets digest that a bit. There are several factors conflated into that one conclusion. We are a nation of increasingly “productive” people, or in other words, people who are slowly losing their work-life balance. This means that cooking healthy meals at home becomes less of an option and eating out or ordering in seem more convenient. This leaves us at the mercy of fast food manufacturers or the food industry in general where the goal is not to see what diet keeps us the most healthy, but the goal is to maximize their bottom lines. As a result, we order and consume more fatty foods as well as foods where there are much fewer “good calories”.

In addition to this, as shown by several studies, when we’re more stressed or “ego depleted”, we go for the junk food. And we eat more. Given the direction in which our lives are headed in today’s economy, this double whammy adds to the problems of higher caloric intake as well as lower quality caloric intake. Easy access to junk food at workplace cafeterias and school vending machines makes the ego depletion time much quicker as well, resulting in the habitual afternoon walks to the cafeteria in search for the sugar high.

As Dr. Chow shows, the human body is also a feedback loop that takes time to arrive at its equilibrium. So if we set ourselves a goal for weight reduction, it takes a long while of sticking to a reduced caloric intake for us to lose the weight, and to keep it off. This demands a lot of will power and attention from weight watchers, making it harder to accomplish. So give yourselves those infrequent binges to satisfy some  craving. But keep at your baseline daily caloric intake goal for a while if you intend to keep that weight off.

Another problem that the modern office worker faces is that of the transition from homo sapiens to homo cubiculus. Most of us spend much of our waking days in front of a computer in a cubicle. Many, such as myself, spend time out of work at home with kids and neglect to expend any significant amount of energy burning calories or working out. However, we forget this conveniently when we sit down to eat. As a result way more calories go in than what could be burnt off doing our daily activities. On top of these calories, we consume a lot of empty calories in the day in the form of soda, which makes things worse. As some studies are hinting at, soda also modifies the way in which we metabolize food, adding to the obesity problem.

In the mix of all these external stimuli, our minds play games on us too. As Brian Wansink showed in his “bottomless soup bowl” research, how much we eat is also decided by visual cues as much as by hunger triggers. By manipulating size of portions or automatically feeding more soup into a soup bowl, he was able to change how much people ate and felt sated at the end of those experiences. The fact that restaurant plate sizes have been growing over time is not a good sign in this regard – the visual cues ask that we have more food on those plates for us to feel satisfied.

So where do we go from here? The Produce for Better Health Foundation has published a really nice report on the behavioral economics and psychology of fruit and vegetable consumption. Dr. Chow, as an outcome of his research has also setup a “Body Weight Simulator” site under the aegis of the National Institute of Diabetes and Digestive and Kidney Diseases, which allows you to set goals based on where you are and where you want to be, and offers suggestions on changes to caloric  intake. There are countless weight loss programs, celebrity-endorsed diets and all the buzz around those. But the key here is to not forget the behavioral and psychological cues that are an important part of achieving the goal. And the social of course, as shown by groups such as Weight Watchers – when you have others who care about your well-being as well, it makes it easier for you to stick to your goals. I should however point out that the land of behavioral economics based results is also littered with questionable studies such as the Christiakis and Fowler study that claims that friends can cause one to become obese. In my opinion this confuses correlation and causation quite badly, as well as not accounting for several other confounding factors that might have impacted the significance of the findings. Nevertheless it seems to be catching on in terms of social currency – so in case you’re unsure, go find some lean friends!

In conclusion I agree with Dr. Chow’s final assertion that we should stop marketing food to children. Childhood obesity is a big problem and while marketing unhealthy junk food to this demographic might be legal, it does leave a bad taste in the mouth and leaves one wondering about the ethics behind those acts. Here is where we can all vote with our wallets – if only we had the time of day to care and act, and share!

What can we learn from the Galleon saga?

[A version of this post was originally submitted towards an assignment in the Legal and Ethical Environment of Business class by Prof. Kevin Werbach.]

Morality, like art, means drawing a line someplace.
– Oscar Wilde

“Three graduates from Wharton class of 1983 indicted on insider trading charges.”

If you missed the color behind the coverage, that is the message that you internalized. That somehow there were a few bad apples in this scandal who went astray, leading to unethical and illegal communication between them due to which Galleon Group profited. In the recent past, the mantle of the bad apple transferred to Rajat Gupta as well, as another mighty executive fell from the perch of the infallible.

But reality is quite removed from this. One big lesson from this saga was that we need to know clearly the difference between feeling indebted to someone or having a good personal relationship with someone and violating the law. Most acts of fraud or corruption in the world are not because of a few big scandals by a few bad apples, but because of complicit participation of several people in very small amounts (Ariely, 2012). As evinced by the details of the proceedings of the Rajaratnam trial, this case brought to light the various ways in which people were made to feel indebted to him, something that he milked to the fullest extent. Fresh graduates from the top MBA programs had dinner parties thrown in their name. Friends had loans extended to them in times of need and forgiven later on.

Such an enlightened self-interest view of the world leads to a contractual view of ethics, where one feels that “ethical behavior extends only to the boundaries of our extended community” (Green, 2010). Another behavioral economics bias to understand here, especially in the rarefied world of financial instruments is that the further removed we are from real dollars, the more likely it is that we convince ourselves that it is okay to cheat. (Ariely, 2012).

As MBAs graduate and continue along their careers to become business leaders, it is critical to understand the difference between what feels right from a transactional point of view between individuals in different roles in companies, and what is right by the rule of law and broader codes of ethics.

Unfortunately most of the courses during an MBA program speak of things that we can measure such as profit maximization for shareholders, but what differentiates great leaders from those that escaped being caught is how they conduct themselves and the affairs of their company as they work towards this objective. Be it greasing the wheels in a developing country to get a favorable tax-exemption in a Special Economic Zone, or dumping dangerous pollutants in a country with lax environmental laws to minimize waste disposal costs – the slippery slope of materially increasing ethical compromises always starts with small lapses of judgment.

In summary, if you think you would find an action wrong if someone else did it, then it is probably wrong.

Ariely, D. (2012). The (Honest) Truth About Dishonesty, How we Lie to Everyone – Especially Ourselves. New York: HarperCollins.
Green, S. (2010). Good Value, Reflections on Money, Morality, and an Uncertain World. New York: Atlantic Monthly Press.

Standing on the shoulders of giants

It was Newton who famously remarked that if he was able to see farther that was by standing on the shoulders of giants. The past session at Wharton was a good reminder of that facet of our lives on several fronts.

It was one of the lightest weeks in a long time, given that we had just two days of two classes each. The first evening ended with a partners event at Lucky Strike where we met several spouses and significant others and many children of our classmates as well. It was amazing to see folks driving up with two kids after a day at work, or flying in from another city with an infant to be there for the event. Needless to say, for the parents in the program such as myself, we are in this program because of the unstinting support from our spouses and support system in our homes, that allows us to keep our jobs, be in this program, and be a parent and be part of significant moments in our children’s lives. A big shout out to them, for letting us work towards our dreams and supporting us through it. Also big high fives to the understanding children in our lives that ask us “how come you have a call every night?” yet keep their peace and love us anyway. The success we get out of our careers would owe a lot to these giant shoulders of support and caring within our lives.

In International Corporate Finance, the session was a marriage of international finance topics along with corporate finance. Without questioning the shortcomings of CAPM here, the model itself gives one a great start in terms of asset pricing. Modern finance owes so much to several academics that defined this field and set the foundation to analyze wide-ranging issues with a basic toolset in finance and arrive at results accurate to the first-order. It remains to be seen how far theories based on Gaussian models remain in the mainstream given the past several years of shocks to the financial world that were “black swan” events which may not have fit neatly into these asymptotic or equilibrium models. In our lives, thanks to people like Bodnar, we can actually make more sense out of headlines as the world of finance seems to be shaking at its foundation all around us.

Which brings me to the other class we had this weekend on legal and ethics issues in business. We went over contract law and intellectual property protection.  The class provides great perspective to see the world of business through a different lens – in terms of relationships between the different parties and the implications of actions from a legal and ethical standpoint. As Prof. Werbach mentioned in class, law provides the floor in terms of acceptable behavior, and ethics sets the ceiling. Its pretty interesting to see how often the world stays at the floor, and who the rare leaders are that elevate us to the ceiling on issues that matter. A mentor that sets the stage for us by leading through example often leaves an indelible mark in our lives and makes it easier for us to assume that the floor is just not enough and it doesn’t hurt to aspire for the ceiling in terms of our behavior.

Outside of classes, I managed to get myself out of bed early to join some classmates for a short  three mile run in the morning as well. Its been three years since I put on my running shoes and I pretty much wimped out after 2.5 miles, but man did it feel good to run down Embarcadero on a chilly morning! A big shout out to Tony Fong and the Team Asha program to for bringing running into my life. Granted, I’m a bad example for the program given I don’t run often enough, but I learnt a lot watching so many of my peers scale new heights – be it the Western States 100, Ironman events or just the sheer magnitude of ultra marathons and crazy triathlon events.

Hmm..almost looks like a Thanksgiving post. But can we ever thank people in our lives often enough?

How dishonest are we?

In “The Honest Truth about Dishonesty” , Dan Ariely’s latest foray down the behavioral economics path that looks  into our irrationality, he recounts another fascinating series of experiments and lessons learnt from them. He deconstructs the simple model of rational crime as he calls it and shows how dishonest acts are not committed through a series of rational decisions. He then looks at various influencers that make one more dishonest – seeing others behave dishonestly, the “Robin Hood” factor of doing it to help others, being “ego-depleted” after a long series of tempation-resisting events, wearing fake brands, being more creative in finding excuses for it and a host of other insights. These came from experiments he did with his students and other university students, and also the studies that his collaborators did elsewhere.

Interestingly enough he observes through his studies that in those instances, the amount of money to be gained, or the probability of getting caught did not change the likelihood of people cheating in his experiment. He also talks about how one can provide timely reminders to make people more honest – making them take pledges, sign their honor codes before a test, moral reminders within their environment and being supervised being the big ones among them.

All in all, another engaging read from Dan that brings down complex human behavior and interaction issues into tractable experiments and offers  interesting insights. One of the key lessons from the book was regarding the impact of dishonest acts – a disproportionately large impact came not from large acts of dishonesty by a few individuals, but small acts from a large number of people. This makes “sense” given that most of us would try to be honest for the most part but when we’re not being observed, or if we could get away with it, might be inclined to cheat in small ways.

One concern I had with these studies in general was sampling bias, and how well the students sampled represented the world population. In an interesting paper published a couple of years ago titled “The Weirdest People in the World”, Joe Henrich and and his co-authors argue that most behavioral economics experiments happen in societies that are “Western, Educated, Industrialized, Rich and Democratic (WEIRD)” and in that sense, we might in fact be sampling outliers and not a representative sample. There is an interesting summary of this at the Freakonomics blog as well.

Concerns notwithstanding, I love Dan’s work and that of his collaborators that provide very interesting insights into behavior and open up interesting questions to ponder on, on the topic of human irrationality and honesty – I’d recommend the book as a must-read!

Speaking of Dan, it is time for Startuponomics 2.0. If you want to be part of this year’s cohort, better apply soon! It was a wonderful experience to be part of the inaugural cohort last year. You can read about it from a post I did soon after that. In addition to Dan there’s an amazing array of great speakers doing pretty interesting research. I believe that startups in the valley with limited budgets for user testing and market research are uniquely positioned to test out these hypotheses in their own products or services, so this is a unique opportunity that shouldn’t be missed! Ping me if you have more questions about my experience from last year.