Buy new:
-50% $12.98$12.98
FREE delivery February 28 - March 6
Ships from: Kuleli Books Sold by: Kuleli Books
Save with Used - Good
$8.26$8.26
Ships from: Amazon Sold by: RNA TRADE LLC

Download the free Kindle app and start reading Kindle books instantly on your smartphone, tablet, or computer - no Kindle device required.
Read instantly on your browser with Kindle for Web.
Using your mobile phone camera - scan the code below and download the Kindle app.
Super Crunchers: Why Thinking-by-Numbers Is the New Way to Be Smart Hardcover – August 28, 2007
Purchase options and add-ons
Today, number crunching affects your life in ways you might never imagine. In this lively and groundbreaking new book, economist Ian Ayres shows how today's best and brightest organizations are analyzing massive databases at lightening speed to provide greater insights into human behavior. They are the Super Crunchers. From internet sites like Google and Amazon that know your tastes better than you do, to a physician's diagnosis and your child's education, to boardrooms and government agencies, this new breed of decision makers are calling the shots. And they are delivering staggeringly accurate results. How can a football coach evaluate a player without ever seeing him play? Want to know whether the price of an airline ticket will go up or down before you buy? How can a formula outpredict wine experts in determining the best vintages? Super crunchers have the answers. In this brave new world of equation versus expertise, Ayres shows us the benefits and risks, who loses and who wins, and how super crunching can be used to help, not manipulate us.
Gone are the days of solely relying on intuition to make decisions. No businessperson, consumer, or student who wants to stay ahead of the curve should make another keystroke without reading Super Crunchers.
- Print length272 pages
- LanguageEnglish
- PublisherBantam
- Publication dateAugust 28, 2007
- Dimensions6.25 x 1 x 9.3 inches
- ISBN-100553805401
- ISBN-13978-0553805406
Book recommendations, author interviews, editors' picks, and more. Read it now.
Frequently bought together

Customers who viewed this item also viewed
Editorial Reviews
From Publishers Weekly
Copyright © Reed Business Information, a division of Reed Elsevier Inc. All rights reserved.
Review
"Data-mining and statistical analysis have suddenly become cool.... Dissecting marketing, politics, and even sports, stuff this complex and important shouldn't be this much fun to read."—Wired
"[Ayres's] thesis is provocative: Complex statistical models could be used to market products more intelligently, craft better movies, and solve health-care problems—if only we could get past our statistics phobia."—Portfolio
"When statistics conflict with expert opinion, bet on statistics....Businesses, consumers, and governments are waking up to the power of analyzing enormous tracts of information."—Discover
"Super Crunchers shows that data-driven decisionmaking is not just revolutionizing baseball and business; it's changing the way that education policy, health care reimbursements, even tax regulations are crafted. Super Crunching is truly reinventing government. Politicians love to tout policy proposals, but they rarely come back and tell you which ones succeeded and which ones failed. Data-driven policy making forces government to ask the bottom line question of 'What works.' That's an approach we can all support."—John Podesta, President of the Center for American Progress
"A lively and yet rigorously careful account of the use of quantitative methods for analysis and decision-making.... Both social scientists and businessmen can profit from this book, while enjoying themselves in the process."—Dr. Kenneth Arrow, Nobel Prize winning economist, and Professor Emeritus at Stanford University
“Ayres’ point is that human beings put far too much faith in their intuition and would often be better off listening to the numbers.... The best stories in the book are about Ayres and other economists he knows, whether they are studying wine, the Supreme Court or jobless benefits.... Ayres himself is one of the [statistical] detectives. He has done fascinating research.”—The New York Times Book Review
"Ian Ayres [is] a law-and-economics guru."—Chronicle of Higher Education
“Lively and enjoyable.... Ayres skillfully demonstrates the importance that statistical literacy can play in our lives, especially now that technology permits it to occur on a scale never before imagined.... Edifying and entertaining."—Publishers Weekly
"Super Crunchers presents a convincing and disturbing vision of a future in which everyday decision-making is increasingly automated, and the role of human judgment restricted to providing input to formulae."—The Economist
"Insightful and delightful!" —Forbes
About the Author
Excerpt. © Reprinted by permission. All rights reserved.
Who's Doing Your Thinking for You?
Recommendations make life a lot easier. Want to know what movie to rent? The traditional way was to ask a friend or to see whether reviewers gave it a thumbs-up.
Nowadays people are looking for Internet guidance drawn from the behavior of the masses. Some of these "preference engines" are simple lists of what's most popular. The New York Times lists the "most emailed articles." iTunes lists the top downloaded songs. Del.icio.us lists the most popular Internet bookmarks. These simple filters often let surfers zero in on the greatest hits.
Some recommendation software goes a step further and tries to tell you what people like you enjoyed. Amazon.com tells you that people who bought The Da Vinci Code also bought Holy Blood, Holy Grail. Netflix gives you recommendations that are contingent on the movies that you yourself have recommended in the past. This is truly "collaborative filtering," because your ratings of movies help Netflix make better recommendations to others and their ratings help Netflix make better recommendations to you. The Internet is a perfect vehicle for this service because it's really cheap for an Internet retailer to keep track of customer behavior and to automatically aggregate, analyze, and display this information for subsequent customers.
Of course, these algorithms aren't perfect. A bachelor buying a one-time gift for a baby could, for example, trigger the program into recommending more baby products in the future. Wal-Mart had to apologize when people who searched for Martin Luther King: I Have a Dream were told they might also appreciate a Planet of the Apes DVD collection. Amazon.com similarly offended some customers who searched for "abortion" and were asked "Did you mean adoption?" The adoption question was generated automatically simply because many past customers who searched for abortion had also searched for adoption.
Still, on net, collaborative filters have been a huge boon for both consumers and retailers. At Netflix, nearly two-thirds of the rented films are recommended by the site. And recommended films are rated half a star higher (on Netflix's five-star ranking system) than films that people rent outside the recommendation system.
While lists of most-emailed articles and best-sellers tend to concentrate usage, the great thing about the more personally tailored recommendations is that they diversify usage. Netflix can recommend different movies to different people. As a result, more than 90 percent of the titles in its 50,000-movie catalog are rented at least monthly. Collaborative filters let sellers access what Chris Anderson calls the "long tail" of the preference distribution. The Netflix recommendations let its customers put themselves in rarefied market niches that used to be hard to find.
The same thing is happening with music. At Pandora.com, users can type in a song or an artist that they like and almost instantaneously the website starts streaming song after song in the same genre. Do you like Cyndi Lauper and Smash Mouth? Voila, Pandora creates a Lauper/Smash Mouth radio station just for you that plays these artists plus others that sound like them. As each song is playing, you have the option of teaching the software more about what you like by clicking "I really like this song" or "Don't play this type of song again."
It's amazing how well this site works for both me and my kids. It not only plays music that each of us enjoys, but it also finds music that we like by groups we've never heard of. For example, because I told Pandora that I like Bruce Springsteen, it created a radio station that started playing the Boss and other well-known artists, but after a few songs it had me grooving to "Now" by Keaton Simons (and because of on-hand quick links, it's easy to buy the song or album on iTunes or Amazon). This is the long tail in action because there's no way a nerd like me would have come across this guy on my own. A similar preference system lets Rhapsody.com play more than 90 percent of its catalog of a million songs every month.
MSNBC.com has recently added its own "recommended stories" feature. It uses a cookie to keep track of the sixteen articles you've most recently read and uses automated text analysis to predict what new stories you'll want to read. It's surprising how accurate a sixteen-story history can be in kickstarting your morning reading. It's also a bit embarrassing: in my case American Idol articles are automatically recommended.
Still, Chicago law professor Cass Sunstein worries that there's a social cost to exploiting the long tail. The more successful these personalized filters are, the more we as a citizenry are deprived of a common experience. Nicholas Negroponte, MIT professor and guru of media technology, sees in these "personalized news" features the emergence of the "Daily Me"—news publications that expose citizens only to information that fits with their narrowly preconceived preferences. Of course, self-filtering of the news has been with us for a long time. Vice President Cheney only watches Fox News. Ralph Nader reads Mother Jones. The difference is that now technology is creating listener censorship that is diabolically more powerful. Websites like Excite.com and Zatso.net started to allow users to produce "the newspaper of me" and "a personalized newscast." The goal is to create a place "where you decide what's the news." Google News allows you to personalize your newsgroups. Email alerts and RSS feeds allow you now to select "This Is the News I Want." If we want, we can now be relieved of the hassle of even glancing at those pesky news articles about social issues that we'd rather ignore.
All of these collaborative filters are examples of what James Surowiecki called "The Wisdom of Crowds." In some contexts, collective predictions are more accurate than the best estimate that any member of the group could achieve. For example, imagine that you offer a $100 prize to a college class for the student with the best estimate of the number of pennies in a jar. The wisdom of the group can be found simply by calculating their average estimate. It's been shown repeatedly that this average estimate is very likely to be closer to the truth than any of the individual estimates. Some people guess too high, and others too low—but collectively the high and low estimates tend to cancel out. Groups can often make better predictions than individuals.
On the TV show Who Wants to Be a Millionaire, "asking the audience" produces the right answer more than 90 percent of the time (while phoning an individual friend produces the right answer less than two-thirds of the time). Collaborative filtering is a kind of tailored audience polling. People who are like you can make pretty accurate guesses about what types of music or movies you'll like. Preference databases are powerful ways to improve personal decision making.
eHarmony Sings a New Tune
There is a new wave of prediction that utilizes the wisdom of crowds in a way that goes beyond conscious preferences. The rise of eHarmony is the discovery of a new wisdom of crowds through Super Crunching. Unlike traditional dating services that solicit and match people based on their conscious and articulated preferences, eHarmony tries to find out what kind of person you are and then matches you with others who the data say are most compatible. eHarmony looks at a large database of information to see what types of personalities actually are happy together as couples.
Neil Clark Warren, eHarmony's founder and driving force, studied more than 5,000 married people in the late 1990s. Warren patented a predictive statistical model of compatibility based on twenty-nine different variables related to a person's emotional temperament, social style, cognitive mode, and relationship skills.
eHarmony's approach relies on the mother of Super Crunching techniques—the regression. A regression is a statistical procedure that takes raw historical data and estimates how various causal factors influence a single variable of interest. In eHarmony's case the variable of interest is how compatible a couple is likely to be. And the causal factors are twenty-nine emotional, social, and cognitive attributes of each person in the couple.
The regression technique was developed more than 100 years ago by Francis Galton, a cousin of Charles Darwin. Galton estimated the first regression line way back in 1877. Remember Orley Ashenfelter's simple equation to predict the quality of wine? That equation came from a regression. Galton's very first regression was also agricultural. He estimated a formula to predict the size of sweet pea seeds based on the size of their parent seeds. Galton found that the offspring of large seeds tended to be larger than the offspring of average or small seeds, but they weren't quite as large as their large parents.
Galton calculated a different regression equation and found a similar tendency for the heights of sons and fathers. The sons of tall fathers were taller than average but not quite as tall as their fathers. In terms of the regression equation, this means that the formula predicting a son's height will multiply the father's height by some factor less than one. In fact, Galton estimated that every additional inch that a father was above average only contributed two-thirds of an inch to the son's predicted height.
He found the pattern again when he calculated the regression equation estimating the relationship between the IQ of parents and children. The children of smart parents were smarter than the average person but not as smart as their folks. The very term "regression" doesn't have anything to do with the technique itself. Dalton just called the technique a regression because the first things that he happened to estimate displayed this tendency—what Galton called "regression toward mediocrity"—and what we now call "regression toward the mean."
The regression literally produces an equation that best fits the data. Even though the regression equation is estimated using historical data, the equation can be used to predict what will happen in the future. Dalton's first equation predicted seed and child size as a function of their progenitors' size. Orley Ashenfelter's wine equation predicted how temperature and rain would impact wine quality.
eHarmony produced a formula to predict preference. Unlike the Netflix or Amazon preference engines, the eHarmony regression is trying to match compatible people by using personality and character traits that people may not even know they have or be able to articulate. Indeed, eHarmony might match you with someone that you might never have imagined that you could like. This is the wisdom of crowds that goes beyond the conscious choices of individual members to see what works at unconscious, hidden levels.
eHarmony is not alone in trying to use data-driven matching. Perfectmatch matches users based on a modified version of the Myers-Briggs personality test. In the 1940s, Isabel Briggs Myers and her mother Katharine Briggs developed a test based on psychiatrist Carl Jung's theory of personality types. The Myers-Briggs test classifies people into sixteen different basic types. Perfectmatch uses this M-B classification to pair people who have personalities that historically have the highest probability of forming lasting relationships.
Not to be outdone, True.com collects data from its clients on ninety-nine relationship factors and feeds the results into a regression formula to calculate the compatibility index score between any two members. In essence, True.com will tell you the likelihood you will get along with anyone else.
While all three services crunch numbers to make their compatibility predictions, their results are markedly different. eHarmony believes in finding people who are a lot like you. "What our research kept saying," Warren has observed, "is [to] find somebody whose intelligence is a lot like yours, whose ambition is a lot like yours, whose energy is a lot like yours, whose spirituality is a lot like yours, whose curiosity is a lot like yours. It was a similarity model."
Perfectmatch and True.com in contrast look for complementary personalities. "We all know, not just in our heart of hearts, but in our experience, that sometimes we're attracted [to], indeed get along better with, somebody different from us," says Pepper Schwartz, the empiricist behind Perfectmatch. "So the nice thing about the Myers-Briggs was it's not just characteristics, but how they fit together."
This disagreement over results isn't the way data-driven decision making is supposed to work. The data should be able to adjudicate whether similar or complementary people make better matches. It's hard to tell who's right, because the industry keeps its analysis and the data on which the analysis is based a tightly held secret. Unlike the data from a bunch of my studies (on taxicab tipping, affirmative action, and concealed handguns) that anyone can freely download from the Internet, the data behind the matching rules at the Internet dating services are proprietary.
Mark Thompson, who developed Yahoo! Personals, says it's impractical to apply social science standards to the market. "The peer-review system is not going to apply here," Thompson says. "We had two months to develop the system for Yahoo! We literally worked around the clock. We did studies on 50,000 people."
The matching sites, meanwhile, are starting to compete on validating their claims. True.com emphasizes that it is the only site which had its methodology certified by an independent auditor. True.com's chief psychologist James Houran is particularly dismissive of eHarmony's data claims. "I've seen no evidence they even conducted any study that forms the basis of their test," Houran says. "If you're touting that you're doing something scientific . . . you inform the academic community."
eHarmony is responding by providing some evidence that their matching system works. It sponsored a Harris poll suggesting that eHarmony is now producing about ninety marriages a day (that's over 30,000 a year). This is better than nothing, but it's only a modest success because with more than five million members, these marriages represent about only a 1 percent chance that your $50 fee will produce a walk down the aisle. The competitors are quick to dismiss the marriage number. Yahoo!'s Thompson has said you have a better chance of finding your future spouse if you "go hang out at the Safeway."
eHarmony also claims that it has evidence that its married couples are in fact more compatible. Its researchers presented last year to the American Psychological Society their finding that married couples who found each other through eHarmony were significantly happier than couples married for a similar length of time who met by other means. There are some serious weaknesses with this study, but the big news for me is that the major matching sites are not just Super Crunching to develop their algorithms; they're Super Crunching to prove that their algorithms got it right.
The matching algorithms of these services aren't, however, completely data-driven. All the services rely at least partially on the conscious preferences of their clients (regardless of whether these preferences are valid predictors of compatibility). eHarmony allows clients to discriminate on the race of potential mates. Even though it's only acting on the wishes of its clients, matching services that discriminate by race may violate a statute dating back to the Civil War that prohibits race discrimination in contracting. Think about it. eHarmony is a for-profit company that takes $50 from black clients and refuses to treat them the same (match them with the same people) as some white clients. A restaurant would be in a lot of trouble if it refused to seat Hispanic customers in a section where customers had stated a preference to have "Anglos only."
eHarmony has gotten into even more trouble for its refusal to match same sex couples. The founder's wife and senior vice president, Marylyn Warren, has claimed that "eHarmony is meant for everybody. We do not discriminate in any way." This is clearly false. They would refuse to match two men even if, based on their answers to the company's 436 questions, the computer algorithm picked them to be the most compatible. There's a sad irony here. eHarmony, unlike its competitors, insists that similar people are the best matches. When it comes to gender, it insists that opposites attract. Out of the top ten matching sites, eHarmony is the only one that doesn't offer same-sex matching.
Why is eHarmony so out of step? Its refusal to match gay and lesbian clients, even in Massachusetts where same-sex marriage is legal, seems counter to the company's professed goal of helping people find lasting and satisfying marriage partners. Warren is a self-described "passionate Christian" who for years worked closely with James Dobson's Focus on the Family. eHarmony is only willing to facilitate certain types of legal marriages regardless of what the statistical algorithm says. In fact, because the algorithm is not public, it is possible that eHarmony puts a normative finger on the scale to favor certain clients.
Product details
- Publisher : Bantam; 1st edition (August 28, 2007)
- Language : English
- Hardcover : 272 pages
- ISBN-10 : 0553805401
- ISBN-13 : 978-0553805406
- Item Weight : 1.03 pounds
- Dimensions : 6.25 x 1 x 9.3 inches
- Best Sellers Rank: #1,315,882 in Books (See Top 100 in Books)
- #2,061 in Probability & Statistics (Books)
- #10,701 in Business Management (Books)
- #16,096 in Success Self-Help
- Customer Reviews:
About the author

Ian Ayres is the William K. Townsend Professor at Yale Law School and the Yale School of Management, and is editor of the Journal of Law, Economics and Organization. In addition to his best-selling SuperCrunchers, Ayres has written for the New York Times, the Wall Street Journal, Financial Times, International Herald Tribune, and The New Republic. He lives in New Haven, Connecticut.Barry Nalebuff is Professor of Economics and Management at the Yale School of Management. His books include The Art of Strategy (an update of the best-selling Thinking Strategically) and Co-opetition. He is the author of fifty scholarly articles and has been an associate editor of five academic journals. He lives in New Haven, Connecticut.
Customer reviews
Customer Reviews, including Product Star Ratings help customers to learn more about the product and decide whether it is the right product for them.
To calculate the overall star rating and percentage breakdown by star, we don’t use a simple average. Instead, our system considers things like how recent a review is and if the reviewer bought the item on Amazon. It also analyzed reviews to verify trustworthiness.
Learn more how customers reviews work on AmazonCustomers say
Customers find the book insightful and well-written. They describe it as an entertaining read for anyone with a big data background. The author uses easy-to-understand prose while describing intriguing facts. However, some readers feel the book lacks substance and is not worth the money spent on it.
AI-generated from the text of customer reviews
Customers find the book insightful and well-written. They say it provides important concepts and intriguing facts about data mining. The prose is easy to understand, and the book encourages readers to seek more information. Readers describe it as an excellent introduction to data mining and reasoning patterns. It stimulates their minds and reinforces the importance of statistics.
"...that allow machines to use defeasible, abductive, and inductive reasoning patterns in order to operate in domains unheard of just a few years ago...." Read more
"...is a basic statistical test of causal relationship it's still a very powerful tool that I need to re-introduce in my analytical life...." Read more
"...He provides multiple historical and empirical examples of how humans are very bad at making decisions...." Read more
"...about the "New Way to Be Smart" and the potentials for vastly improved decision making that can enhance our lives, but we non-crunchers need to be..." Read more
Customers find the book engaging and instructive. They find it a great read for those with a big data background. The author provides clear explanations of using data to validate points. Overall, readers describe it as an excellent commute read.
"...of pure exhilaration, and proof again that this is the best time ever to be alive." Read more
"Great book on the importance of data-driven decision making...." Read more
"...All-in-all, I thought this book was okay. Interesting, but not great." Read more
"I thought this was one of the best books I've read this past year. I found it to be well written, entertaining and insightful...." Read more
Customers find the book easy to read. They appreciate the author's clear explanations of how using data to validate points works. The book is written for novices and provides good documentation with references. The non-technical language presents history and applications in a way that's accessible to non-experts.
"This short book, which can be read in a few hours, could be considered an apology or even a manifesto for mathematical and statistical modeling...." Read more
"...The writing is clear but unexceptional. But the book is incredibly thought provoking if you haven't been aware of the trends...." Read more
"...I found it to be well written, entertaining and insightful...." Read more
"...I like the way he presented standard deviation in a way that was easy to understand...." Read more
Customers find the book unimpressive, boring, and not informative. They say there is no substance, no story, no conclusion, and no main point.
"...All-in-all, I thought this book was okay. Interesting, but not great." Read more
"...There is no substance, no story, no conclusion and no "main point" in it. The book reads as glorified hype of something old and trivial...." Read more
"...that innocent readers like me were lured to purchase otherwise unimpressive work,..." Read more
"Not worth spending money on this book." Read more
Top reviews from the United States
There was a problem filtering reviews right now. Please try again later.
- Reviewed in the United States on September 8, 2007This short book, which can be read in a few hours, could be considered an apology or even a manifesto for mathematical and statistical modeling. Even those readers, such as this reviewer, who have been involved in "supercrunching" for many years will find some interesting anecdotes in this book that illustrate its power and limitations. But even more importantly, it discusses the reactions of many (and typically highly insecure) individuals against the practice of mathematical modeling, some of these bordering on the absurd but with most content with ridiculing its practice. There is no question that the "supercrunching" that the author describes will continue to have greater influence in the manner in which it is currently practiced, but it is also true that much more sophisticated approaches to modeling will arise, some of these using machine intelligence and highly advanced mathematics. Indeed, in the past decade the use of artificial intelligence has exploded in areas such as finance, network engineering, bioinformatics, and Internet security. The author has just scratched the surface of the vast number of tasks that are now being done by machine, sometimes without any intervention or supervision by humans.
As the author details in the book, many businesses and public institutions have jumped on the bandwagon of supercrunching, and their successes in doing so he documents well, with references given for readers who want more of the details. But many businesses that could profit greatly from this approach have refrained from its use, because of skepticism or distrust of quantitative reasoning. To paraphrase the author, they want to stay with the horse-and-buggy, while others are getting around in locomotives. There is still a great reliance on "experts" whose track record is weak and when compared with statistical modeling falls very short. It is an open question whether these businesses will find themselves in bankruptcy court because of their rejection of statistical modeling, but they are going to have to face stiff competition from the businesses that do.
For those involved in the supercrunching that the author describes, it is not surprising to hear that many important business decisions are being made based on the results of statistical modeling. Humans are to a large degree still "in the loop", but the author describes instances where the machines "are actually in charge." The author still wants both human and machine to be in a mutual symbiosis, with considerably more weight given to machine predictions. And along these same lines, he brings up the canonical question as to what place humans are to have as more responsibility for decision-making is given to the machines. Many may find their social and employment status shrink, becoming white elephants (or "potted plants" to use the author's terminology) in the process. The author tries to alleviate these anxieties by pointing to the need for humans to still do the groundwork that enables supercrunching to take place. Humans must still "hypothesize" he asserts, in that they must still make the decisions as to what variables are going to be used when the machines actually perform the statistical analysis.
Certainly if one remains within the statistical modeling paradigm, as the author does throughout the book, there will still be need for humans to "hypothesize." But if one chooses to go beyond this paradigm, the landscape changes considerably. The author gives a brief glimpse into how this is done in his discussion on neural networks. But he leaves out any discussion of the research in automated scientific and mathematical discovery that has taken place in the last two decades, some of this research showing remarkable progress. If this trend continues, and there is every reason to think that it will, then the machines will be able to hypothesize, theorize, and analyze in a manner that is similar to humans but may be vastly superior. In addition to these developments, significant progress has been made in artificial intelligence that allow machines to use defeasible, abductive, and inductive reasoning patterns in order to operate in domains unheard of just a few years ago. These domains include automated legal reasoning, computational creativity, rumor detection and propagation, virus recognition in data networks, musical composition, and automated mortgage underwriting. With these and their supercrunching abilities, they will certainly instill both admiration and fear, and using them will require extreme confidence to a degree that goes far beyond what the author describes in this book. It is disquieting to some that the machines will have this degree of intelligence and autonomy, but to others it is a source of pure exhilaration, and proof again that this is the best time ever to be alive.
- Reviewed in the United States on June 20, 2015Great book on the importance of data-driven decision making. While I have always been someone that has let the data do the talking, I haven't found an easy way to explain why. Super Crunchers is that easy way! Below I have summarized some of the important points of the book....
Super Crunching is crucially about the impact of statistical analysis on real-world decisions. Two core techniques for Super Crunching are the regression and randomization.
1. Regression will make your predictions more accurate (Historical approach):
It all starts with the use of regressions, and although this method is a basic statistical test of causal relationship it's still a very powerful tool that I need to re-introduce in my analytical life.
Regressions make predictions and tell you how precise the prediction is. It tries to hone in on the causal impact of a variable on a dependent. It can tell us the weights to place upon various factors and simultaneously tell us how precisely it was able to estimate these weights.
2. Randomization and large sample sizes (Present/Real-Time approach):
Reliance on historical data increases the difficulty in discerning causation. Large randomized tests work because the distribution amongst the sample are increasingly identical. Think A/B testing on steroids that allows you to quickly test different combinations! Boils down to the averages of the "treated and untreated" groups.
Government has embraced randomization as the best way to test what works. Statistical profiling led to smarter targeting of government support
With finite amounts of data, we can only estimate a finite number of causal effects
3. Neural network
Unlike the regression approach, which estimates the weights to apply to a single equation, the neural approach uses a system of equations represented by a series of interconnected switches.
Computers use historical data to train the equation switches to come up with optimal weights. But while the neural technique can yield powerful predictions, it does a poorer job of telling you why it is working or how much confidence it has in its prediction.
Super Crunching requires analysis of the results of repeated decisions. If you can't measure what you're trying to maximize, you're not going to be able to rely on data-driven decisions.
We humans just overestimate our ability to make good decisions and we're skeptical that a formula that necessarily ignores innumerable pieces of information could do a better job than we could.
- Reviewed in the United States on March 17, 2008The author has a two Bachelors (Russian Studies and Economics) from Yale, PhD (Economics) from MIT, JD from Yale. He is a bonifide genius and he sprinkles this book with much of his intellectual horsepower.
He provides multiple historical and empirical examples of how humans are very bad at making decisions. Human are even worse in determining the quality of their decisions (most people tends to exaggerate the correctness of their answers). Statistics, on the other hand, are mathetically sound way of predicting future results. Better yet, it provides a highly reliable way of determining the quality of the prediction. Generally speaking, the better the data, the more predictive statistics can become.
Recently, however, there has been a great convergence of statistics and technology. More specifically, informational technology which allows number crunching of multiple terabytes of data in a relatively short period of time has become available at a practical price.
The author believes whoever can take advantage of this relatively new phenomenon has a great competitive and informational edge. Having reliable data analysis tools that use statistical regression analysis enables an entitity to make better decisions (and also know the probability of making a bad decision).
This has become a game changing convergence. Indeed, it is changing the world and will become even more important in the future.
What should you do to be better prepared?
Learn statistics and get the best data mining/analysis tools and employees your company can buy. As the author states, we are seeing only the beginning of this mostly positive trend.
Top reviews from other countries
- Ashray ManoharReviewed in India on February 17, 2017
4.0 out of 5 stars Very interesting read, it touches upon daily life examples ...
Very interesting read, it touches upon daily life examples where number crunching has played a role to extract insight of business which can be used to make crucial decisions. But at times it becomes redundant.
- StephenReviewed in Canada on January 13, 2015
5.0 out of 5 stars Amazing stories. Give this out to all sorts of ...
Amazing stories. Give this out to all sorts of people. Love it!
- TimGReviewed in the United Kingdom on February 28, 2013
5.0 out of 5 stars Enjoyable and thought-provoking
I'm far from an expert in this field so can't make any comment on whether the contents of the book are accurate, but I found it very engaging to read, enjoyable, and certainly got me thinking about the implications of number crunching to evaluate the world around me. Recommended to me by a friend, and I'd strongly pass that recommendation on.
- DenReviewed in the United Kingdom on December 7, 2021
4.0 out of 5 stars The future is stats
Number crunching is the future. Want to get ahead? Study statistics. Great book on the brute force of number crunching and how it works.
- Vijay BahadurReviewed in India on June 30, 2019
4.0 out of 5 stars 4 Star
4 star