It was a gentleman agreement to keep those faulty nvidia in macbook, while introducing a new chip side-by-side. In the long term, I believe Nvidia will come out winning, regaining it's reputation, while will allways the one that sells expensive machines that ultimately fails in a very short period of life. I really would like to see some numbers on this issue: how many macbooks were affected, and how many were repaired; what was the lifetime of this machines? Great article inquirer.

http://www.google.com/insights/search/#q=macbook%20pro%20faulty&cmpt=q

http://www.google.com/insights/search/#q=nvidia%20faulty&cmpt=q

according to article, there's no difference between g84 and g86 cores of videochips. i mean package and soldering method. my own investigations show, that problem is in the bad compound which dilates due heat, and thus destructs soldering of balls between core and substrate. you can blow on the core(using soldering hot air gun) using 200C temperature and chip will work again for some time.

You can see the problems in http://discussions.apple.com/thread.jspa?messageID=9959108&tstart=0#9959108

Has no one tested the durability of the motherboard? It doesn't sound like an nVidia problem. Try testing the motherboard, or better yet try testing the Intel chipset on the board.

Blaming nVidia is stupid.

My screen went black immediately after updating to OS X 10.5.7 which includes the firmware update that claims to have fixed the black screen problem... This is not a new update... I wait so I can give them time to resolve these issues... but i should have researched it first!
Apple Genius claims there is nothing wrong with the NVIDIA graphics card, and through a process of elimination deduced the logic board must be replaced AT MY COST, because by coincidence it went out at the same time of the update... me and hundreds of others, as evidenced by the complaints in the apple forums. I wonder why Apple encourages so many of these "replace the logic board" solutions? Doesn't that suggest that there is something faulty with Apple's logic boards, failing in less than 2 years? But I thought it was the NVIDIA chips affected? One tech source suggested that it was the NVIDIA chip that caused the logic board failure, though appearing unaffected itself.
If it is only a small percentage of people affected, Apple should be happy to admit a small fault. That seems to be their strategy: accepting only a small amount of cases strangely similar to a much larger group...
I am looking forward to the day that the price of computers is appropriate for their lack of durability.

"Note
In a statement just before publication, Nvidia's Mike Hara had the following comment on the situation. "The GeForce 9600 GPU in the MacBook Pro does not have bad bumps. The material set (combination of underfill and bump) that is being used is similar to the material set that has been shipped in 100s of millions of chipsets by the world's largest semiconductor company.""

Intel?!?

My mbp gpu is dead dead dead and " invidia" is replacing it though you have to take it into an apple store for the repair. So is apple paying the bill or inviria??? Anyways last mac I ever buy. For what you pay apple should be on top of this and not sell mbp that could have possible failures. I've only had the mbp for a year. When I get it back I think I'll buy one of those bs laptop coolers. Btw you could grill some good food with the heat mbp puts out maybe that's all it's really good for now LOL. Thankfully I've got an iPhone that drops calls to help me stay connected while I wait for the next three to four days for apple to replace my logic board with another possible bad gpu. :) thank you Steve Jobs FTW

This article gives a whole new meaning to "Computer Forensics" and I tend to agree with its findings. However, if Apple had developed a better architecture, the heat would not have become an issue. Just look at Toshiba's new Qosmios. They look like gigantic ships. Apple likes it this way though: smart and so f****** hot.

UPDATE on the ISSUE.

I have bought a 2.4Ghz Unibody Macbook Pro in January, and I have experienced the Black screen of death; along with some half black screen + horizontal lines freezes.

These freezes happened in both WinXP SP3 (via Bootcamp), and Mac OS X 10.5.6 playing iTunes/Quicktime or just with the screen saver being turned on.

The freezes have happened under windows using the 9600M GT card (it's the one being used by default under bootcamp); and under Mac OS X using the 9400M. (not even playing games; but as I said.. it happened while watching movies..)

So, to me ... this MacBook Pro is cursed; both graphic cards have issues. Euthectic/Non-euthectic, well.... good work TheInquirer. However, it's been MONTHS now, and Apple has still not released any statement/fixes for this.

I am so upset; because they won't replace the computer here in France. Apple Care told me to bring the computer to an authorized repair center... but of course, that way I am forced to 1. find a way to get there 2. wait a week with no machine

I have also tried spinning up the fans, using smcFanControl; and this does not have any impact on this macbookpro.

The machine is running cold and it still freaking dies.

I have also replaced the drivers in BootCamp (graphics, nForce, Audio... etc) as you have described here: in vain.

Just thought I'd post my experience.

what news on the 17" unibody MBP's ???

2.53 gHz, 9600m GT, 4GB RAM, etc
Purchased around December 12 '08
Fired up Oblivion the first time and cranked the settings, ran beautifully. Went to show my friends - CRASH. BKSOD EVERY TIME SINCE. Investigated and some posters suggested heat, so I checked the bottom...OUCH. Searching around I found this article and a few others. I've never read much Inquirer, but the namesake doesn't inspire confidence, however, Charlie's claims seem to match my experience. Also, the forums are FULL of this for the new MBPs.

Note: I'm fast becoming an Apple fanboi, but now with this, I'm so confused! Somebody hold me.

I think Anand missed physics class, lead is a softer and is more malleable than tin.

Looking at the images though i noticed the metal seems to have not melted properly, the mixture does not seem homogeneous. And it looks like there is pitting or trapped gases in the metal. I think that they are just not heating it properly, and maybe the environment needs to be controlled better

http://www.flipchips.com/apnotes/kyzen/Kyzenevaporative.pdf

That is an article that talks about high lead bumps.

Thank you for your concern. I will call apple care asap and see what they have to say about it. I also noticed that they recently released an update to the OS that has "graphics improvements". I'm going to play more games today and see if it crashes on me again.

Warbear, if the machine is faulty then get your money back. Give them absolute hell, you will get your money back.

The product is unfit for the purpose it was sold for. Get your money back.

Warbear, are apple stores denying there is a problem with these notebooks? Is apple part of the nvidia coverup scandal?

Nvidia-gate!

Charlie, you rock. Keep up the excellent work.

I currently have the late 2008 MBP and have had some issues with mine. I know the time that you have had the machine may affect performance, however I have had mine for 1 week and have had it give me the black screen of death 3 times when playing games (All using the 9600m). I took it to my local apple store and was basically told, nothing we can do about it, hopefully they patch your game to increase the fan speed. Quite disturbing.

Hey, just to let you know the problem with the bad bumps, isn't about how many hours you have played games/video etiditing etc...

Due to the bad bumps + thermal cycling (computer on/off) results in cracking of the bump contacts hence giving unpredictable outcomes depending on which bumps were affected.

I suggest you read more about this problem. Your brand new Macbook (or pro, any nvidia based chipset/graphics chipset) will only develop this problem after a year of usage depending on the user's habit (how many on/off, hubernate, sleep cycles they do in a day).

I for one was affected with a a Hp DV6000 and a XPS M1530 waiting for trouble.

My suggestion, if it's not too late to return it, do so.

Very cool that you have access to the resources to perform such analysis. I understand that stress tests are going a bit far (thats something nVidia probably already did) but I would like to see some hard data (MTBF etc.) that the new kind of balls/package perform differently (not necessarily less reliable) before jumping to qualitative conclusions like 'bad' or 'good'.

Great work!

To:
posted by : Jack Wester, 10 December 2008

Charlie let me handle this one for you.

Jack,
A great piece of advice is don't post what you think. You come across stupid. Evidence was clearly submitted and you say it's lies with no proof that this article does that. You- nothing to show and all talk.

You only look like a stupid republican moron by posting your fan boi remarks and opinion which was a worthless waste of everyone's time. Go crawl back in a hole and come out when you have something to show this article is false.

Hey Tony,

Nvidia has stated that they are using both manufacturing material sets until the older "unreliable" one is depleted.

"NVIDIA will transition from using high-lead solder (95%Pb/5%Sn) to eutectic solder (63%Sn/37%Pb) flip-chip bump material for the G92 product family. During the transition period NVIDIA will be supplying both high-lead and eutectic bump until inventory is depleted. No other materials are being changed."

Last sentence here is pretty important "NO OTHER MATERIALS ARE BEING CHANGED." So I were to take it, the underfill has NOT changed.

Thanks for the heads up Charlie.
I was just about to buy a MacBook when I read this.
Went with a 24" Imac with ATI card instead.

After already having 2 x 8800 cards go tits-up I've had enough of the green monster for a while.

As NVIDIA have pointed out, it's the entire material properties (solder and glue) in combination that leads to the robustness in the design. The glue (underfill) and the solder together create the stability of the connection. They've used a more robust underfill to fix the problem; as a result they don't see the problem occurring any more. An analogy: imagine if your roofing felt keeps tearing off your roof in high winds. You replace it with the same type of felt but next time use a stronger glue to keep it on. Problem solved, by using a stronger glue not by using a different type of roofing felt. In your testing process you've only tested the felt and not tested the new stronger glue that's been used.

Anand think so at anandtech in his Jasper article:

http://www.anandtech.com/gadgets/showdoc.aspx?i=3472

"Traditionally GPUs had used high-lead bumps between the GPU die and the chip package, these bumps can carry a lot of current but are quite rigid, and rigid materials tend to break in a high stress environment. Unlike the bumps between the GPU package and a motherboard (or video card PCB), the solder bumps between a GPU die and the GPU package are connecting two different materials, each with its own rate of thermal expansion. The GPU die itself gets hotter much quicker than the GPU package, which puts additional stress on the bumps themselves. The type of stress also mattered, while simply maintaining high temperatures for a period of time provided one sort of stress, power cycling the GPUs provided a different one entirely - one that eventually resulted in these bumps, and the GPU as a whole, failing.

The GPU failures ended up being most pronounced in notebooks because of the usage model. With notebooks the number of times you turn them on and off in a day is much greater than a desktop, which puts a unique type of thermal stress on the aforementioned solder bumps, causing the sorts of failures that plagued NVIDIA GPUs.

In 2005, ATI switched from high-lead bumps (90% lead, 10% tin) to eutectic bumps (37% lead, 63% tin). These eutectic bumps can't carry as much current as high-lead bumps, they have a lower melting point but most importantly, they are not as rigid as high-lead bumps. So in those high stress situations caused by many power cycles, they don't crack, and thus you don't get the same GPU failure rates in notebooks as you do with NVIDIA hardware."

I've been doing VLSI design for over 20 years, have worked at IBM, Intel, TI, etc. Your focus on just to Pb bumps is flawed, it is the entire material set used in the connections as NVIDIA indicates.

If you were standing in front of a engineering postmortum team right now (who specialize in this area), they would most likely shoot you down for being incomplete and misleading.

Why don't you send your article to IEEE for publication. First it would have to pass the scrutiny of engineering experts in this area. As with a postmortem investigation, I would expect they would decline to publish your article.

Keep trying though... :-)

What you have shown is that the bumps on the 9400 and 9600 are different. Even if the PR guy said that they were supposed to be the same, but they are not, that alone doesn't mean that this difference could lead to failure. It's a plausible theory, but I don't see the cause-and effect link or smoking gun here. I certainly agree that it is very difficult or impossible to prove conclusively, unless you have an actual sample of a failure, and of course NVidia won't provide that to you. Maybe if you ask them nice...

To be frank this article epitomises bad science; that science that uses scientific methods to investigate "something" without regard to make sure that the particular "something" they're investigating is the right one

See http://news.cnet.com/8301-13924_3-10119277-64.html

"When you build a device, it's the material properties and everything in combination that leads to the robustness of the design. What we call the 'material set.' It's a combination of the underfill (a kind of a glue that helps hold the chip down) and the bump together that creates that stability in that connection,"

"What we did was, we just simply went to a more robust underfill. Stopped using that (previous) underfill, kept using high-lead bumps, but we changed the underfill. And now we don't see the problem."

"Intel has shipped hundreds of millions of chipsets that use the same material-set combo. We're using virtually the same materials that Intel uses in its chipsets,"

Charlie, can you look into the suspiciously high failure rates for desktop 8600/8800/9600 series ? Had to rma a board to XFX not long ago, and came across forums like this one:

http://www.bjorn3d.com/forum/forumdisplay.php?f=54

Where apparently my videocard's cause of death is quite common.

Allow me to add my praise and admiration to the chorus. I thought this type of detective work was dead and gone. I am absolutely gob-smacked at your tenacity and knowledge. Well done Sir - you deserve an extra turkey leg this Yuletide. Keep 'em coming.

Just wanted to comment on your work done here, simply astounding. Real journalism, taken to the next level.

Surely the bottom line is actual failure rates. Do we have any reliable evidence to suggest that these are high for the parts Charlie is speaking about? They could make bumps out of blancmange for all I care as long as they work.

No one realized this when Apple announced the use of nVidia GPUs?

Duhh! This isn't news at all! lol

nVidia is the suck.
Liars and cowards.

<3

I take issue with the premise that high lead bumps must be bad just because nVidia switched to eutectic bumps on the newer 9400M.

As Jmendes inferred earlier, high lead was commonly used for years before ROHS. Compared to eutectic, high lead is a more compliant material (meaning less stress on chip interconnects) and is less prone to electromigration (meaning better ability to retain its current capacity). The most authoritative explanation on this whole subject that I have seen is Pkgg Engr's comment to Part 1 of Charlie's 'Why Nvidia's chips are defective' series.

It is quite possible that the much higher performing 9600M requires such properties, and thus the reason nVidia stuck to high lead.

So this leads us to the underfill. There is no evidence that I can see that the combination of the new underfill with the old high lead bumps is prone to premature failure. The article infers that this must be the case because the nVidia chose eutectic bumps in the newer 9400m and because MacBooks have an overheating problem. This is simply speculation. I'm not saying it's not possible but the evidence is not yet there.

Don't get me wrong, Charlie. I do not favour either point of view on this whole affair. In fact I appreciate your previous work on uncovering and explaining the high failure rate of earlier chips, and find no reason to dispute it. However, in this case, I find your conclusion premature.

I must also respond to the comment about this being a 'witch hunt'. Who cares? The motivation behind an investigation is irrelevant as long as the truth is told.

Even though I'm not an nvidia fanboi, it still kind of gets on my nerve with Charlie constantly bashing them up without solid proof. Whether this article will be accepted or rejected is another story. But good work into the research and publishing of this article. One of the better reads on Inquirer for sure. Congrats!

I think the answer is, all consumers... I for one loves being informed. I'm sure plenty of others appreciate this kind of investigative work. AMD might prosper at the expense of Nvidia, but that's not Charlie's fault.

Oh, gosh, I wonder who this story could benefit if it turned out to be true? This one's really taxing... oh... maybe a huge rival GPU maker which is resurging in the GPU market at the moment, and just waiting for some cockup like this by its main competitor, NVidia?

That same company which Charlie is well known for preferring over NVidia?

Or, maybe a certain big CPU maker that does a bit of integrated graphics on the side, currently having a tetchy relationship with NVidia?

A lot of people stand to make a lot of money if reports like this turn out to be true. Charlie's gone too far to just be doing this for the fun of it. He kicked up quite the storm a few months earlier with much less scientific evidence, remember.

"Bad bumps" might not necessarily equate to the solder compound, but the email thread certainly implies otherwise.

Thorough work indeed Charlie, but even if we let aside the fact that you don't have any dignity or integrity to be an objective and worthy journalist, I have to ask: why do you see so much trouble to even use multimillion equipment to bring a down a company? Why? It would help nobody. Actually it would just bring trouble but you seem to have some kind of a personal agenda with this one.

Cheers

"bad bumps" do not equate to lead bumps or eutectic bumps. You proved the base materials of the bumps - so what? Nvidia says that the cracking bumps were displaced by a new material process. You have not proved that the bumps on your example had cracks. I see none - none were observed or noted. So the materials on the 9600 are "pure" lead - this has been in existence since cave man times and has been used in the industry excessively until the new RHOS requirements. Show me the cracks in the material, show me actual performance degradation due to the 9600 bumps - and maybe I'll believe some of what you have implied here - without irrefutable evidence - you have no leg to stand on.

Charlie,

This is the kind of periodism I respect! Holy sh*t! I just hope this is a bacth, because i just bought a laptop with a 9700m gt chip, which is virtually the same as the 9600.

KUDOS! Ahhh, and btw, uk.theinquirer.net doesn't work anymore, so I guess I wasn't the only one that was isolated from its favourite newspage until i tried another option. Andd... congrats for the new layout!

Charlie, I gotta give it to you mate, it's friggin brilliant!

You deserve a raise!

Nvidia + fanboys, HOW DO YOU LIKE THEM APPLES?

HAHAHAHAHAHAH

Excellent work Charlie, I'm really impressed you took this to the limit and really proved what's going on at Nvidia. I almost bought a 9600 m chip in a laptop 2 days ago. I thought that maybe they had managed to get the fix out for them by this time and held back the bad chips, but decided it wasn't worth the risk. The next day I read this article and was VERY glad I did not buy that laptop.
Thanks Charlie for helping me avoid a stinker. Unfortunately I own 2 8800 GTS, wonder when they will fail.
Keep up the good work.

Hara is over on CNET Lying about this. CNET has an article talking about this article on their site. It's one thing to make an accusation, but when you disassemble the product and prove that it has bad bumps, NVidia should just come clean and admit they screwed up, but you know that they won't. NVidia may as well just get in line at Congress to request a bail out because Bumpgate plus the RAMBUS lawsuit will hit their stock price and knock them off their feet.

Do me a favour and note his previous news. He's reports are more than 75% wrong.

New massive layoffs aften the first round? Still waiting.

A x86 CPU? Still waiting.

Massive failure rates on old chips? Just not true.

Abnormal failure rates on new macbook chips? Just not true. In fact, they are very reliable.

Tis still in effect but expires in 2010. Nvidia had better be figuring out how to use the "new" materials by then.

I just would like to congratulate the author, it is a brilliant piece of work/investigation. I can imagine that it must have been cost both lots of time and money to make this article but the reality is that only this type of work can reveal the truth about what companies are really up to. So well done charlie its work like this that needs to be made public more often.

Well done Charlie.

This new layout is more formal, not like INQ at all.

Bumps are exempt from RHOS, so it's legal in the EU....

http://www.rohs.gov.uk/content.aspx?id=17

Basically, they're not stupid, you know...

nice article - nice layout. I like it!

nice artcile, shitty new layout

Whatever you think of Charlie, whatever you think of the Inq... what you have now is INCONTROVERTIBLE evidence of the lies and cover-ups of nVidia. These parts have been REPEATEDLY proven to be at higher risk of failure... yet, the knee-jerk reaction of fanbois is to defend, defend, defend to the end. Sad, really. I guess people just hate the thought of thinking that yes, a company would take their hard earned cash despite the problems that exist.

I found this article both interesting and informative. Keep it up Charlie D.

I've got a Dell Vostro with a 8400M in it, and on these cold days has been booting up into corrupt graphics. It's fine after a reboot...but for how long?

Dun dun duuuuun :D

Lead content is a violation of ROHS. Non ROHS equipment is not allowed in Europe.
I wonder how they skipped ROHS(green) tests..

Hi Charlie

http://www.dvhardware.net/article31847.html

Please check that they have changed the underfill, Hi Lead bumps is not a problem by it self.

I dont usually read long articles, and I know the Charlie is sometimes biased, but I thought this was a very interesting and fair article. Its amazing to see them constantly lie! Ah well cant fault nVidia for consistency.

So it looks like the Nvidia 9300/9400 (as used in the MacBook) is ok then

So you'll probably be wanting to publish a correction to your previous article:
"Nvidia heat causing Macbooks to fail"
Right?

Plus, they may actually be doing the right thing by using stronger glue
- have to done any analysis on chips from other manufacturers (Intel, AMD etc) to see how the nVidia Materials Set compares to these ones?

I found this article interesting. Bought a 15" Macbook Pro 2.4 GHz on the day it was released from the Apple Store on Regent Street, London. I've played Spore for hours, Civ IV for days, and done a lot of video editing. I always have the GPU enabled. And I have NEVER experienced even on lock up as described. That's not to say that there aren't problems as decribed above, but I just wanted to assure people that not all of the Macbook Pros are faulty.

I though Charlie had been quiet for a while... this article explains what hes been up to...

Hi Charlie

Love your work, but it seems that your stories are being ignored by other web medias ?

Is that because they are afraid of nvidia or is it because your stories are wrong :-)

Wkr.
DK_Charlie_fan

nVidia said in their previous statement that they would fix their material set to correct the problem. You found bad bumps, but nVidia claims they're using a stronger glue this time around. So technically, they've switched to a "more robust material set", but they obviously haven't fixed the root of the previous problem - bad bumps. Glue might work, but the solder might crack nonetheless. Maybe you guys should crank up the heat on the boards and see how the chips handle themselves.

Clever of nVidia though - they don't actually say "bad bumps" at all in their statements or emails.

This is great! Now INQ readers can skip the telly, and get their CSI fix (Charlie Semiconductor Investigation) right here on the INQ.

Thanks for writing this great article (as requested). Most excellent!