Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Community > MacNN Lounge > Intel CPU flaw. Kernel memory leak.

Intel CPU flaw. Kernel memory leak.
Thread Tools
Thorzdad
Moderator
Join Date: Aug 2001
Location: Nobletucky
Status: Offline
Reply With Quote
Jan 3, 2018, 08:11 AM
 
     
andi*pandi
Moderator
Join Date: Jun 2000
Location: inside 128, north of 90
Status: Offline
Reply With Quote
Jan 3, 2018, 08:34 AM
 
At one point, Forcefully Unmap Complete Kernel With Interrupt Trampolines, aka ****WIT, was mulled by the Linux kernel team, giving you an idea of how annoying this has been for the developers.
     
Thorzdad  (op)
Moderator
Join Date: Aug 2001
Location: Nobletucky
Status: Offline
Reply With Quote
Jan 3, 2018, 11:26 AM
 
Oh, this just gets better. It seems that Intel's CEO just recently sold enough stock to take his holdings down to the minimum he's required to hold, as mandated by Intel's corporate bylaws.
     
reader50
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Jan 3, 2018, 12:22 PM
 
I'd like more specifics on which CPUs are affected. They're saying those made in the last decade. A lot of us have been holding off upgrading because the later Macs lack expansion options, or because CPU speed improvements have been small. It's possible we'll get a performance reward for doing so, if we're before the bug.

Originally Posted by Thorzdad View Post
Oh, this just gets better. It seems that Intel's CEO just recently sold enough stock to take his holdings down to the minimum he's required to hold, as mandated by Intel's corporate bylaws.
He'd better be able to prove the sale was planned before he learned of the bug. Otherwise he's in violation of securities laws forbidding use of inside information.
     
And.reg
The Mighty
Join Date: Feb 2004
Location: Well the sports issue was within arm's reach but they closed up shop and kicked me out. And I'm out of toilet paper.
Status: Offline
Reply With Quote
Jan 3, 2018, 12:34 PM
 
This may require an extra supply of popcorn...well done, Intel, well done.
This one time, at Boot Camp, I stuck a flute up my PC.
     
subego
Clinically Insane
Join Date: Jun 2001
Location: Chicago, Bang! Bang!
Status: Offline
Reply With Quote
Jan 3, 2018, 12:50 PM
 
I bet Intel is looking for ways to blame the engineers from VW.
     
Thorzdad  (op)
Moderator
Join Date: Aug 2001
Location: Nobletucky
Status: Offline
Reply With Quote
Jan 3, 2018, 12:58 PM
 
Originally Posted by reader50 View Post
I'd like more specifics on which CPUs are affected. They're saying those made in the last decade. A lot of us have been holding off upgrading because the later Macs lack expansion options, or because CPU speed improvements have been small. It's possible we'll get a performance reward for doing so, if we're before the bug.
As best as I can suss, if it's a 64-bit Intel processor, it's affected. That would make my late-2009 Core2 Duo iMac affected.
     
OreoCookie
Moderator
Join Date: May 2001
Location: Hilbert space
Status: Offline
Reply With Quote
Jan 3, 2018, 06:21 PM
 
These flaws are apparently not exclusive to Intel systems (from 1995 onwards), but many ARM chips and potentially also AMD chips are vulnerable, too.
According to Macrumors CPUs with speculative execution are affected (that includes all high-performance parts and excludes a whole bunch of parts for embedded systems). ARM has stated in a press release that they are working with AMD and Intel on a software fix to these issues. AMD claims that there is “near zero risk” they are affected.

Usually I don't link to MacRumors for this type of stuff, but since they include quotes from press releases and it is early in the morning here
I don't suffer from insanity, I enjoy every minute of it.
     
sek929
Posting Junkie
Join Date: Nov 1999
Location: Cape Cod, MA
Status: Online
Reply With Quote
Jan 3, 2018, 06:22 PM
 
A good reminder I have no idea about how computers actually work.

How on earth did it take so long for this flaw to surface?

Classist bigot and incurable ideologue
     
Thorzdad  (op)
Moderator
Join Date: Aug 2001
Location: Nobletucky
Status: Offline
Reply With Quote
Jan 3, 2018, 07:54 PM
 
FWIW, per Wikipedia...
Intel's processors implementing the Intel64 architecture include the Pentium 4 F-series/5x1 series, 506, and 516, Celeron D models 3x1, 3x6, 355, 347, 352, 360, and 365 and all later Celerons, all models of Xeon since "Nocona", all models of Pentium Dual-Core processors since "Merom-2M", the Atom 230, 330, D410, D425, D510, D525, N450, N455, N470, N475, N550, N570, N2600 and N2800, and all versions of the Pentium D, Pentium Extreme Edition, Core 2, Core i7, Core i5, and Core i3 processors.
     
Brien
Professional Poster
Join Date: Jun 2002
Location: Southern California
Status: Offline
Reply With Quote
Jan 3, 2018, 07:59 PM
 
So, like... everything?
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Jan 3, 2018, 08:25 PM
 
I have been amusing myself with reading the papers...

Both of the attacks work by using the cache to leak data.

Meltdown is the serious one, and only affects Intel for now - although there are hints that it may be possible on other designs. Program 1 creates an array and makes sure that it is not cached. It then spawns program 2. Program 2 makes sure that the kernel reads a piece of secret data into a register A. Program 2 then copies register A to register B and makes a memory read from that big array based on the contents of register B. As soon as the first copy instruction retires (ie, reaches the end of execution pipeline so its data should be stored and made accessible), program 2 will be terminated by the kernel and the data it held lost. Program 1 can now try to read different parts of its array and see how quickly it gets a result. Based on which read is fast, it can figure out what the contents of that register was - if the contents of the register was 5, program 2 will try to read the fifth value from the array, and program 1 can detect that value 5 loads faster. In practice, program 2 will only send one bit at a time - if that specific bit is 1, it will read an address, and if it is zero, it will do nothing. Program 1 will then spawn new copies of program 2 to read another bit, and so on. Continue until you have dumped the entire contents of RAM.

Spectre is arguably more clever. It sets up a branch like

If(x<sizeOfArray)
y=array2[array1[x]*256]

By running it many times where the If always evaluates to true, the branch predictor will assume that it should always be true. By giving it an X slightly larger than sizeOfArray, the CPU will make a load from array2 based on data stored just after array1, and the program can then figure out what the data was based on timing analysis during reads from array2. This affects all CPUs, but is a bit more limited in scope as you need to know the memory layout of the victim’s computer. It becomes a piece of an attack, not the whole thing.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
reader50
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Jan 3, 2018, 08:26 PM
 
Originally Posted by Brien View Post
So, like... everything?
We won't know until the details are released. Could be endemic to the Intel64 base, or when a particular branch prediction module was introduced. Or something else. The fact that they're not releasing which CPUs are affected implies we could determine where the bug is by checking past feature introductions.

It appears to be an issue of speculative execution misses vs (cache modification and/or a ring-privilege bug). Whatever it is, it can't be fixed with CPU firmware, or computer firmware.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Jan 3, 2018, 08:31 PM
 
Originally Posted by Brien View Post
So, like... everything?
There are two bugs, Meltdown and Spectre. The serious one, Meltdown, affects all Intel CPUs to as far back as anyone has been able to determine. Anything ever used in a Mac for sure. There are patches for it, but they hurt performance on some operations. As a regular user, this will be irrelevant, less than a percent on anything from Haswell and on and slews than 5% on anything older. It will hurt servers, though.

Spectre affects everything that has a branch predictor and a cache, which is truly everything from the last twenty years, from all manufacturers. That one is more limited in scope, however - you need to know the memory layout of your victim, and performance is limited. There are no patches, and there it does not appear to be possible to make one.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Jan 3, 2018, 08:34 PM
 
Originally Posted by And.reg View Post
This may require an extra supply of popcorn...well done, Intel, well done.
No, this one is bad. Intel has it worst, but no one is safe right now.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Jan 3, 2018, 08:38 PM
 
Originally Posted by sek929 View Post
A good reminder I have no idea about how computers actually work.

How on earth did it take so long for this flaw to surface?
It has been growing gradually. I know people have been talking about so-called side channel attacks for a year now, with some even predicting that there would be something like this, but they were unknown before that. This is an entirely new type of attack.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Jan 3, 2018, 08:42 PM
 
Originally Posted by reader50 View Post
I'd like more specifics on which CPUs are affected. They're saying those made in the last decade. A lot of us have been holding off upgrading because the later Macs lack expansion options, or because CPU speed improvements have been small. It's possible we'll get a performance reward for doing so, if we're before the bug.
Sorry, no. The patch for Meltdown doesn’t hurt user code that bad, and anyway it affects all Intel COUs ever used in a Mac, as far as we know.

He'd better be able to prove the sale was planned before he learned of the bug. Otherwise he's in violation of securities laws forbidding use of inside information.
Intel as a company knew about the bug for months. If the CEO somehow didn’t know, he isn’t doing his job. I think he expects to get fired over this and wanted to cash out, and this is the very definition of insider trading.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
reader50
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Jan 3, 2018, 09:12 PM
 
Originally Posted by reader50 View Post
He'd better be able to prove the sale was planned before he learned of the bug. Otherwise he's in violation of securities laws forbidding use of inside information.
Originally Posted by P View Post
Intel as a company knew about the bug for months. If the CEO somehow didn’t know, he isn’t doing his job. I think he expects to get fired over this and wanted to cash out, and this is the very definition of insider trading.
If that's his plan, he'll be going to jail. Insider trading is handled by the SEC, not Intel. It won't matter if he's fired or not.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Jan 3, 2018, 09:17 PM
 
Originally Posted by reader50 View Post
If that's his plan, he'll be going to jail. Insider trading is handled by the SEC, not Intel. It won't matter if he's fired or not.
How many of the bankers involved in the crash of ‘08 actually went to jail in the end?

1.

People get away with insider trading far too often.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
Thorzdad  (op)
Moderator
Join Date: Aug 2001
Location: Nobletucky
Status: Offline
Reply With Quote
Jan 4, 2018, 08:38 AM
 
My biggest question right now is how Apple will implement any patches, and how many older Macs are they going to leave twisting in the wind. My daily driver is a late-2009 iMac (10.9.5) which, for certain software reasons, isn't going to run any newer OS. I have this sinking feeling that, when Apple does issue patches, my iMac is not going to be invited to the party, even though it's absolutely affected by this problem.
     
OreoCookie
Moderator
Join Date: May 2001
Location: Hilbert space
Status: Offline
Reply With Quote
Jan 4, 2018, 10:21 AM
 
Apple has apparently already patched it with 10.13.2. With your 2009 iMac, you are probably out of luck.
I don't suffer from insanity, I enjoy every minute of it.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Jan 4, 2018, 11:09 AM
 
Apple has most likely patched not only 10.13, but 10.12 and 10.11 as well.

https://support.apple.com/en-us/HT208331

The text for the security updates in 10.13.2 is vague as it was written before the embargo was up, but it identifies who discovered the bug. The person at Google Project Zero who is the lead for Meltdown (Jann Horn) is noted as the discoverer for two bugs. The description for one is "An application may be able to read restricted memory", which could very well be Meltdown. The other is "A local user may be able to cause unexpected system termination or read kernel memory", which isn't too far off either. A couple more reference other people at Google Project Zero with interesting descriptions. All of those bugs were patched in 10.12 and 10.11 as well.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Jan 4, 2018, 11:14 AM
 
Originally Posted by OreoCookie View Post
Apple has apparently already patched it with 10.13.2. With your 2009 iMac, you are probably out of luck.
Not only that, but a 2009 iMac will be missing the "PCID" feature. That feature minimizes the performance impact of the Meltdown patch, at least on Linux, so a patch of a Mac that old would necessarily hurt performance more.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
reader50
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Jan 4, 2018, 12:26 PM
 
Assuming P's detective work is right, Apple's patching of 10.11 covers a whole lot of Macs. Basically everything back to 2008, and 2007 in some cases. It just doesn't cover anyone who needs to run an older OS.

If the older OS is not connected to the internet, you'd be fine. Possibly OK too if you only use a few apps, and restrict them. ie - modern browser, with flash and java / javascript turned off.

I'd like to see Apple patch further back on critical security flaws. Seems like they have a few times. I remember a late Safari update for Snow Leopard for example.

Edit: AppleInsider appears to have confirmed the detective work. Initial fixes in 10.13.2, with further fixes coming in 10.13.3.
( Last edited by reader50; Jan 4, 2018 at 12:46 PM. )
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Jan 4, 2018, 01:07 PM
 
I think that having supported multiple processor architectures with different page table designs over the years helped Apple respond quickly here. Both Windows and Linux have been x86 first for a very long time, and fundamental design decisions were made with its idiosyncratic single TLB design in mind.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Jan 4, 2018, 08:01 PM
 
https://support.apple.com/en-us/HT208394

Official statement from Apple. MacOS is indeed already patched for Meltdown, but crucially so is iOS and tvOS. Interesting - that implies that Apple’s ARM cores are affected.

Spectre will be mitigated by a Safari patch in the coming days, but work is ongoing on that one. There is also an LLVM patch out there that promises to reduce the threat at the compiler level.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
And.reg
The Mighty
Join Date: Feb 2004
Location: Well the sports issue was within arm's reach but they closed up shop and kicked me out. And I'm out of toilet paper.
Status: Offline
Reply With Quote
Jan 5, 2018, 12:47 PM
 
Originally Posted by P View Post
Interesting - that implies that Apple’s ARM cores are affected.
Yeah, that really surprised me.
This one time, at Boot Camp, I stuck a flute up my PC.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Jan 5, 2018, 01:48 PM
 
I have been thinking about that. The Meltdown paper describes a way to prevent it - basically to check if a memory read is allowed early in the pipeline, ideally straight after decode (=when it is “understood”). I initially thought that this is how AMD did it (because there was talk of a patent), but then I realized that that can’t be true. Intel and AMD crosslicense all patents every five years (for historical reasons - basically, IBM used to have some pull in this business) and if AMD patented this less than five years ago, many of their chips would still be vulnerable. The Meltdown paper describes a “toy” that shows was that pretty much all CPUs will speculatively excite without checking privileges. So why is Intel vulnerable?

Here is what I think. Warning: baseless speculation follows (not entirely unlike Intel CPUs)

When loading data from memory, there is always a lag. From L1 cache that can be covered in the reorder, but from anything further out than that, the CPU basically stops to wait for data. Part of Intel’s secret sauce is that they wait for data less often. The trick to doing this is that any load operation coming down the instruction flow will be elevated and sent to execute as soon as possible. This is why I think Meltdown hit them - whenever the rogue process had read the data it shouldn’t, the load instruction was already scheduled and ready to go at the load/store unit. This is called store-to-load forwarding, and is something that Intel improved massively in Conroe, the first Core 2. The load could execute and get as far as a cache before the illegal read got to retire, because Intel had prioritized that process so highly.

Apple’s CPUs are second only to Intel. That they managed that with A7 was impressive enough, but that wasn’t really close to Intel - it was just better than every other mobile CPU. A8 was a dud, and A10 and later basically only added cores and raised the clock a bit. A9, though - it recovered essentially all the IPC advantage Intel had without changing the core in any significant way. “All” it did was improve the memory subsystem and caches.

You see where I’m going with this. I think Apple copied Intel’s policies with regards to store-to-load forwarding and elevating loads. If I’m right, there is more where this came from. Pray that Apple’s App Store vetting processes are good.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
reader50
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Jan 5, 2018, 03:01 PM
 
For those who have to run macOS versions before El Capitan, there is hope. From this Ars article:
As an operating system vendor, Apple has updated both iOS and macOS to use the dual page table mappings that are the recommended solution here. For Apple, this is perhaps not such a big change; the 32-bit x86 versions of macOS already used a similar scheme. This was because Apple wanted to give 32-bit applications access to the full 4GB of virtual memory, rather than splitting that 4GB between the program and the kernel. While this imposes a performance cost, it provided better compatibility with the PowerPC macOS, which also gave applications the full 4GB.
So apps running in 32-bit mode are safe from Meltdown, as are apps running in PPC macOS. That means everything before 10.5 is safe - I believe 64-bit *nix processes were introduced in 10.4, but don't recall any apps actually using them. And nearly all apps are Meltdown-safe through 10.7. It was only with 10.8 that the OS always booted into a 64-bit kernel, and apps got recompiled in large numbers to be 64-bit native.

If you're careful to use 32-bit app versions, you'll have Meltdown protection even after 10.8 - I've lost track of when Activity Monitor stopped showing the bit-mode processes were running in.

This doesn't address Spectre on older OSes, but that one's harder to take advantage of. Make sure JS is off in your browser.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Jan 6, 2018, 05:32 PM
 
I had to look this up, because the history was fuzzy, but..

10.4 only ported System.lib to 64-bit.
10.5 ported all of Cocoa and all its underlying libraries.
10.6 shipped with a 64-bit kernel that was not always in use.

So if you are still on 10.4, you would be safe from Meltdown, I guess. I don’t think that running the 32-bit kernel on 10.5 or later will keep you safe, however.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
reader50
Administrator
Join Date: Jun 2000
Location: California
Status: Offline
Reply With Quote
Jan 6, 2018, 09:18 PM
 
As the vulnerability comes from apps (user processes, actually), you'd be safe enough if all apps run in 32-bit, as those get their own full memory maps. What mode the kernel runs in isn't directly relevant - I was using the 64-bit kernel and OS as an indicator of when developers likely recompiled their apps to run in 64-bit mode.

Activity Monitor in those OSes showed if an app was running as 64-bit or 32-bit. If you were careful about what apps you use, you could avoid the risk. Use 32-bit apps whenever possible. If you must use a 64-bit app, make sure it's both one you trust, and one that doesn't launch secondary code (mostly browsers).

Some non-browser apps like PhotoShop host plugins, but if memory serves, PS was limited to Carbon for a very long time. And Apple never released the 64-bit Carbon libraries, though people report those were developed internally.
     
Doc HM
Mac Elite
Join Date: Oct 2008
Location: UKland
Status: Offline
Reply With Quote
Jan 7, 2018, 11:10 AM
 
Originally Posted by reader50 View Post
If that's his plan, he'll be going to jail. Insider trading is handled by the SEC, not Intel. It won't matter if he's fired or not.
It will be interesting to see how this pans out. If there's no fallout from it then essentially the signal to CEO's and Wall Street is they can go crazy?

My guess is he'll offer up a lameomatic excuse about not reading some memo and everyone will pretend to believe him plus some weasel words about "learning from" and "going forward not looking back"
This space for Hire! Reasonable rates. Reach an audience of literally dozens!
     
OreoCookie
Moderator
Join Date: May 2001
Location: Hilbert space
Status: Offline
Reply With Quote
Jan 7, 2018, 09:35 PM
 
So far Intel's public reaction hasn't been great, The Register (as usual) has a hilarious takedown of their official response from the PR department.
I don't suffer from insanity, I enjoy every minute of it.
     
P
Moderator
Join Date: Apr 2000
Location: Gothenburg, Sweden
Status: Offline
Reply With Quote
Jan 8, 2018, 06:32 AM
 
Originally Posted by Doc HM View Post
It will be interesting to see how this pans out. If there's no fallout from it then essentially the signal to CEO's and Wall Street is they can go crazy?

My guess is he'll offer up a lameomatic excuse about not reading some memo and everyone will pretend to believe him plus some weasel words about "learning from" and "going forward not looking back"
His response is that this was always planned, and he just didn't change it, and anyway the stock price hasn't dropped any yet.

Originally Posted by OreoCookie View Post
So far Intel's public reaction hasn't been great, The Register (as usual) has a hilarious takedown of their official response from the PR department.
The Register has gained a bit of a second wind over this, I feel. They are irreverent for no good reason, which is what grates on me usually, but in a case like this it works great.

Originally Posted by reader50 View Post
As the vulnerability comes from apps (user processes, actually), you'd be safe enough if all apps run in 32-bit, as those get their own full memory maps. What mode the kernel runs in isn't directly relevant - I was using the 64-bit kernel and OS as an indicator of when developers likely recompiled their apps to run in 64-bit mode.

Activity Monitor in those OSes showed if an app was running as 64-bit or 32-bit. If you were careful about what apps you use, you could avoid the risk. Use 32-bit apps whenever possible. If you must use a 64-bit app, make sure it's both one you trust, and one that doesn't launch secondary code (mostly browsers).
You can do all of this - I think older OSes also showed if an app was 32-bit or 64-bit in Get Info, not certain - but any Meltdown exploit will come in the form of a trojan at this point. Just don't run apps you don't trust - OSes that old have enough privilege escalation exploits as it is.

Some non-browser apps like PhotoShop host plugins, but if memory serves, PS was limited to Carbon for a very long time. And Apple never released the 64-bit Carbon libraries, though people report those were developed internally.
Correct, Apple finished 64-bit Carbon and promised that it was coming, but killed the project weeks before WWDC, as they decided that they wanted to focus on one API going forward.
The new Mac Pro has up to 30 MB of cache inside the processor itself. That's more than the HD in my first Mac. Somehow I'm still running out of space.
     
   
Thread Tools
 
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Top
Privacy Policy
All times are GMT -4. The time now is 11:09 PM.
All contents of these forums © 1995-2017 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.8 © 2000-2017, Jelsoft Enterprises Ltd.,