"“I want you to estimate,” Oliver began, “how much money you think Google makes daily from Gmail ads.” Oh. My. GOSH. Was he serious? The answer depended on so many different factors, none of which I had any clue how to guesstimate. “Um, you mean a hard number? Maybe…$70,000?” Oliver’s hearty laugh told me my response was foolish. ... Now I was asked for an exact amount of revenue. “Say each G-mail user opens seven new e-mails a day. They would see 28 ads. If they click on ¼ of those ads, then only seven ads are clicked. If all advertisers are charged $0.05 per clicked ad, then the amount of revenue would be whatever $0.05 x 7 ads x the number of G-mail users is. Does that make any sense at all?” “Kind of.” Oliver sounded confused. “You lost me at the ‘only clicking on ¼ of the ads’ comment. Let’s move on.” " - Paul Buchheit from Bookmarklet

Does everyone get those estimation questions? I didn't get anything like that in the interviews I had at Google. - Benjamin Golub

There's a reason I'm not a Google employee... - Robert Scoble

Here's a hint: if they ask you to estimate something, don't just make up a random number. Also, don't assume that people click on 1/4 of all ads :) - Paul Buchheit

@bgolub She was interviewing to be an APM. I imagine those interviews are quite different from Eng interviews. I can't be sure having only done Eng interviews myself. - EricaJoy

Its unfortunate, because the interviewers need to filter, but some people don't do well when put on the spot under real-time stress, but in a job situation could solve similar problems being left to think about them. The kinds of skills needed to win lightning math competitions or TopCoder, are not necessarily the skills best needed to work on a product. - Ray Cromwell

The problem I had with my interviews at Google were they were too literal. They were all questions that a recent college grad would be familiar with but anyone with any length of experience in the industry would have forgotten by this point. Going back and studying after I remember the answers now but I felt like I was being interviewed by someone just out of college (which was probably the case). - Jesse Stay

I've heard of one Eng interview question that could be answered quickly by estimation (but more slowly by just computing it). (I wasn't asked that question myself, nor did I ever ask it.) - Ruchira S. Datta

Priceless "You lost me at the 'only clicking on ¼ of the ads'" comment is priceless :) - Micah Wittman

Jesse, I wasn't asked any questions like that, nor did I ask any. I guess the interview process can be very variable. - Ruchira S. Datta

The "estimate something unknowable" question is something anybody interviewing for an even remotely technical position should expect these days--even though you might not get it--and you should understand the interviewer is asking to hear the process you go through, not the answer. As far as the 1/4 click through rate goes... well... that's classic. - Ken Sheppardson

Ha-ha, clicking on ¼ of the ads seemed like a ridiculously high estimate to me. I never click on an ad. And now I have ad block pro installed so I can't even see them. - Laura Norvig

A Google interviewer asked me similarly ridiculous questions. Why ask me about low-level database algorithms when i'm interested in java and web positions?! We danced around one question for 10 minutes while i tried to answer it to his satisfaction. It should have been obvious to him i'm knowledgeable enough about databases and database programming to just move on to the core interview. - ·[▪_▪]·

Ray, it's not a math problem -- it's a problem solving problem. - Paul Buchheit

Paul, there are different kinds of problem solving as well. To take mathematicians as an example, von Neumann was said to be very quick, whereas Hilbert was rather slow. So Hilbert might not have done well in an interview. - Ruchira S. Datta

The classic variations I've heard are "How many piano tuners are there in Chicago?" and "How many hamburgers does McDonalds sell each day?" - Ken Sheppardson

Wow... how would you answer either of those questions? - SAM

One of the people I interviewed with didn't speak very good English, and I spent 3/4 of the interview arguing with her how I thought Friend Connect could improve. I was then given 5 minutes at the end of the interview to try and answer her technical question which was about binary trees, something I hadn't studied or played with since college. 5 minutes wasn't enough to recall. The questions I was asked were math problems (and I graduated with a 3.95 GPA in college, A in stats and Algebra). - Jesse Stay

I could never survive an interview process like that. So I won't even try. - Laura Norvig

Jesse, sorry to hear that. I would usually ask one 45-minute interview question or two 20-minute ones. I was only asked 45-minute ones. And no non-technical questions till the end. - Ruchira S. Datta

These are almost the same questions I was asked (which I could answer easily now): http://courses.csail.mit.edu/iap... - amazing that Google hires MIT grads, considering MIT has an entire course around "hacking the Google interview". - Jesse Stay

SAM: You'd just dive in with an estimate of *something* and walk your way through it. E.g. How many people do you know who own pianos? What % of the population do you think owns pianos? How often do they get their pianos tuned? How long does it take to tune a piano? How many people live in Chicago? It doesn't matter so much whether your numbers are right, it's the fact you know how to combine all the component estimates into an overall estimate. - Ken Sheppardson

Not being able to do the math is fine, thinking that 1/4 of ads are clicked on is not. It's the approach and rough estimates that matter. - Paul Buchheit

Yeah, the two important pieces of info the interviewer got in this situation are from (1) the initial random guess vs walking through some estimating process and (2) the lack of understanding of click through rates - Ken Sheppardson

Those problem solving skills are useful when you have to model real-world systems. How many concurrent users to you estimate will we have at peak load? How many expensive vs. cheap queries are they going to perform? It's basically the same process - figure out reasonable estimates for each of the factors, do some basic stats and math and get a ballpark of how many servers you'll need vs. whether your problem is even viable. - Matt Mastracci

Ruchira my technical questions were all at the end (except for 45 minutes on arguing with someone I could barely understand how Friend Connect could improve). I got the non-technical, easy. Going back and studying I would have gotten the technical too, which is silly considering how fast I was able to find the answers (and understand them). I think that's why non-technical should be weighed higher than technical. It's easy to find the technical if you can prove you have the experience to find the right answer. - Jesse Stay

When I interview, I look for clear, informed thinking. I'm sure that's what Google looks for, too. It's the odd personal interaction that turns an interview into something hasty and pressured. I'm fine giving a person a question, and time to mind-map or outline -- tho I do prefer someone who can do the thinking right there in the moment. Also, it's best to warm a person up to the type of question, so they're not panicking as they switch from one mode to something completely different (don't small talk and then turn to a hard analytical question). - Christopher Galtenberg

Jesse, the technical question is supposed to be about problem-solving. You see how someone solves problems by watching them do it. It's not about looking things up. - Ruchira S. Datta

Ruchira, 5 minutes to show the thought process in figuring out how to tell how many levels are in a binary tree isn't enough to determine if someone can problem solve. - Jesse Stay

Call me soft, but I don't think she should have been bounced for a bad guess on ad CTR under interview stress. She worked through the appropriate steps and setup the right equation, just one of her assumptions was garbage. If she wasn't under pressure, and was asked to prepare a spreadsheet modeling out ad revenues, would she have really picked 1/4? or would she had thought about it more and/or did research. Take the Piano tuner question, if someone doesn't know anyone who plays piano, pretty much all the assumptions will be garbage. - Ray Cromwell

Jesse, agreed. That's why I like the longer interview questions. E.g., I would often find that someone was good at writing code but bad at complexity, or could think of an algorithm but not program. - Ruchira S. Datta

RE "ridiculous questions", one of the things I try to do when interviewing somebody is ask a question to which the correct answer is "I don't know." That's one reason I might drill down on some sort of low level technical issue outside the candidates field of expertise. If you don't know, say so. Don't sit there and try to pretend you do. Be willing to admit there's something you have to learn. For extra credit, explain how you'd find the answer. - Ken Sheppardson

Ken, there's major game theory going on. Does the interviewee think the interviewer wants a best effort attempt or rather an admission of insufficient data/experience. - Micah Wittman

Another one I realized afterwords he was talking about linked lists, but he never mentioned linked lists. He had just asked how many edges it would take (asking for the exact formula, mind you - it's been 10 years since stats!) before a cycle could be made (or something like that). If he had just mentioned he was talking about linked lists I could have given him an answer much faster. - Jesse Stay

Micah: Yup. And I want to see how much somebody will thrash before they ask what it is I actually want to know. - Ken Sheppardson

you got to thin the herd. - Aron Michalski

The right answer may be both. A good communicator, person who knows how to 'read' the other will put it out all out on the table. Hopefully without being too wordy :) - Micah Wittman

The interviewer is comparing multiple interviewees. Google gets so many qualified applicants that it's very likely that even though she was OK, compared to others applying for the same job, she didn't do as well. - Piaw Na

The interview process is designed to have few false positives, but in the classic tradeoff it can therefore have many false negatives. Although Jesse, in your case it sounds like it was just executed badly. - Ruchira S. Datta

The funny thing is the best answer wasn't always the answer they were looking for. For some reason interviewers never want to see the hash table answer, which is almost always the answer in the real world. (yet they never ask you why a hash table may not always be the best answer) - Jesse Stay

I always want the hash table answer Jesse. People who use red-black trees or whatever often haven't written real software :) - Paul Buchheit

Paul I like your style :-) So much time is saved that way. - Jesse Stay

@Paul: Agreed. I also find a surprising number of people who think they're brilliant for inventing tries, when a hash table would solve the problem faster, and with less code. - Piaw Na

Not only that but in a search world that can make or break your search speed. We dealt with that in HIPAA transaction matching at UnitedHealth Group while I was there. The hashed digest made matching records so much faster. - Jesse Stay

I cringe when I find out the developer interviewing me just graduated from college 3 years ago. - Jesse Stay

I have never been asked for these estimation questions during my interviews but engineering positions need them too. Good estimators are good engineers. - Burcu Dogan

@Jesse, why does that bother you? I was interviewed at Google by 6 folks, 5 of which were way younger than I was --- one of them so young that I remember him from when I was his TA in grad school. It didn't matter --- the questions weren't particularly hard, and I had fun. I didn't think Google's interview was any tougher than Yahoo's, Microsoft's or any other tech company known for engineering excellence. - Piaw Na

My quickie solution to the Egg Drop problem is exponential doubling sequence + 2x linear search = 25 drops, best solution is 14. Doh! BTW, I've heard this problem stated before as a Cats with 9 lives (can survive 9 drops). - Ray Cromwell

Piaw because a recent college grad is only going to ask you what they learned in college - Jesse Stay

Try to solve my interview puzzle question: http://cromwellian.blogspot.com/2006... - Ray Cromwell

Ray, that's a cool problem. - Matt Mastracci

@Jesse So the 3 years in the "real world" mean nothing? - EricaJoy

@Matt, I specifically constructed it so that it would be impossible for people to search for a pre-existing solution. - Ray Cromwell

@Jesse: I'd been working in Silicon Valley for 10 years when I interviewed there. I did not feel that my experience disadvantaged me. - Piaw Na

I just take more of a "get it done" attitude when I write code. I'd rather focus on getting the problem fixed in the fastest manner possible rather than spend all this time on theory. I guess it all depends on the problem at hand as to whether that's the right choice, but a recent college grad is less likely to understand that than someone who's been in the field for awhile. I like the questions that focus around that issue more than I like the intense algorithm for spending all this time trying to find the absolute most optimized solution that most of the time doesn't even matter for the problem at hand. - Jesse Stay

BTW, if you're a startup that wants to take on Google that's the way to beat Google. It's why Twitter has grown so fast. - Jesse Stay

True, if you look at many of the successful Web 2.0 startups, a lot of them didn't solve interesting computer science problems, but executed well in other ways. Implementation speed becomes a priority, as they can always go back and fix stuff later or rewrite, once they reach a certain level. Twitter ran "ok" enough on a Ruby mishmash until they broke down, but they didn't really lose their users because of it. - Ray Cromwell

Glen, unless the speed isn't as important as getting the product out the door fast. - Jesse Stay

All of those formulas need a "T" element (time to write the code), along with an "M" (maintainability) element - Jesse Stay

I think the best way to get a job at Google is to build a business and get bought by them :-) - Jesse Stay

Its good to be retired... ;) just sayin - WarLord

@Glen: Are you seriously asserting that insertion into a tree (of any sort) is O( 1 / (n log n))? - Piaw Na

@Jesse: It does no good to acquire a company whose software wouldn't scale (or whose people can't make them scale) when Google turns the firehose of traffic at them. Some of the due diligence done before a company is acquired (by Google or anyone) is to make sure that everything's technically up to snuff, or the people being acquired are smart enough to get them up to snuff. - Piaw Na

I don't see anything "nightmarish" about her interviews at all...In fact it seems pretty amazing to me she couldn't figure out the "math" problem in the 2nd interview. - Bindu Reddy

Well just for grins, here are a myriad of "problem solvers" trying to figure out how many golf balls you can fit on a bus: http://www.acetheinterview.com/questio... My solution to this is tell me how long it will take you to hang my wet clothes on the line without slipping on the downward slope of a ravine on 5 acres of wood WHILE you are figuring out how many golf balls it will take to fill up a school bus. In my interviews, I want to assess brains and brawn. ;) - Melanie Reed

If the question was really about problem solving then why would you stop at 1/4? That's just a parameter. - Todd Hoff

You would stop at 1/4 because you've already interviewed 20 other folks, and some of them gave you a much better answer there. You would then go back to hire one of them, or keep interviewing more people because nobody was good enough so far. - Piaw Na

wow, very insightful but clearly she is not geeky enough for the job! - Loc

Todd: The interview didn't end when she said 1/4, they just "moved on". The point wasn't to get a specific dollar amount, it was to see her process. - Ken Sheppardson

It's true that 1/4 is just a parameter, but choosing something out by so much shows a pretty significant lack of knowledge in an area that has _some_ relevance to Google. I'd suspect there would have been candidates who'd been able to have sensible discussions about the likely CTR in Gmail based on known CTRs on other websites and the factors which influence it. Surely that's relevant? - Nick Lothian

Also, clicking on 1/4 of ads, with 4 ads per email means that people were clicking on an ad every time they read an email. That's pretty clearly wrong. - Nick Lothian

Yeah, the 1/4 answer shows that the candidate did not apply her own personal experience to the problem solution. That's a negative mark. - Gary Burd

Nick, then don't say it's about the process, because clearly it's not. The process was correct. If you care about the numbers then ask about the numbers. Ken, "you lost me" is not moving on. - Todd Hoff

“Kind of.” Oliver sounded confused. “You lost me at the ‘only clicking on ¼ of the ads’ comment. Let’s move on.” - Ken Sheppardson

How you came up with your estimates is part of the process. Also, the ability to do basic match (1/4 * 4 = 1) is part of the process. "Does that sound high or low to you" is part of the process. - Nick Lothian

I wish I had learned how to reason probabilistically much earlier than I did. I never even heard about Bayes theorem until after grad/professional school. - Victor Ganata

"@Matt, I specifically constructed it so that it would be impossible for people to search for a pre-existing solution" Ray, what's the last piece of useful technology you've worked on? - Bill de hÓra

@Nick "It's true that 1/4 is just a parameter" - well is it or isn't it? is the model wrong or not? "but choosing something out by so much shows a pretty significant lack of knowledge in an area that has _some_ relevance to Google" - You mean like a Web based email client, or an ad serving system? - Bill de hÓra

"How many golf balls can fit in a school bus?" - Don't care, bought a dump truck instead. These questions are to interviews as rail shooters are to video games. - Bill de hÓra

@Plaw "It does no good to acquire a company whose software wouldn't scale" - explain Jaiku and while you're at it, explain Chubby. - Bill de hÓra

you can click on the ads in gmail? learn something every day. - SuezanneC Baskerville

Paul Buchheit's assumption of a 25 percent clickthrough rate on a Gmail ad is way to high. Most advertisers report an average of 3 clickthroughs per 1000 impressions. That about one one-hundredth of the rate that Paul has entertained. I suggest that he experiment with Google or Facebook advertising before flaunting assumptions that are two orders of magnitude beyond the norm. - Rich Reader

@Rich Reader that's right you tell Paul not to flaunt his assumptions about Gmail. But maybe read his post and comments first. Oh, and maybe his bio. - Steve Crossan

@Bill, Google Web Toolkit. Prior to that, http://timefire.com/showcase. - Ray Cromwell

@Rich Reader: I can't quite discern whether your comment is satirical. Paul wasn't making that assumption, and I'm fairly confident he knows a bit about Gmail, Google advertising, and Facebook. :) - Simon

http://friendfeed.com/search... Paul I am having deflamatory comments about me with my first and last name used, could you please warn this user or something, it is effecting my business and is illegal, I have told brent TWICE, but he removed my account and placed it on private now which was quite rude to be honest, now I will see a lawyer about deflamation since it effects my views as a writer if she is not removed or given a warning - dawngordon

Just saw an interesting article on Gawker about Google hiring http://gawker.com/5392947... and then another where Peter says they misconstrued his statements: http://www.mv-voice.com/square... - Laura Norvig